Score actions, not manners
Text questions care about refusals and fabrication. Action questions care about the wrong tool, the wrong identity, data sent to the wrong place, and irreversible steps taken with no one in the loop.
LEAVE-BEHIND / PRE-PRODUCTIONCHENGDU, CN / 2026
Before go-live,
finish these thirty checks.
This is a leave-behind, not a white paper and not a customer report. Security teams do not need to become prompt engineers first: put the agent that is about to ship on the table and ask for evidence. Any question you cannot answer is a go-live blocker. The page is readable on its own; download the PDF when you need to print or countersign.
Book a 30-minute AI agent security architecture reviewExplore the workspace demoDownload the checklist PDF
Who this is for
- CISOs and security leads who must decide whether a Copilot, knowledge bot or business agent may touch production data
- Platform, application and data owners who actually attach tools, MCP servers and accounts
- Business owners moving expense, support, coding or operations agents from a trial into production
- Assessors and auditors who need to turn “we are careful” into evidence someone else can review
01An AI agent is not a chatbot with a nicer UI
An AI agent is not a chatbot with a nicer UI
Model security asks whether a system produces disallowed text, leaks training secrets, or gives up its system prompt. That work matters. It is not a production licence for an agent. An agent also has a goal, tools and state. It can turn “handle this invoice” into a chain of calls against internal systems. When those calls succeed, the business has already changed.
Output filters stop sentences, not tool calls
A denylist on the model’s mouth helps a chatbot. For an agent that speaks MCP, the action may already have left before the filter runs — or the model only says “done” after the tool has moved.
“We use an aligned model” is not a finding
Align the task scope, tool list, identity, approvals and logs. A perfect score on the subset can still push internal data off the boundary in week one.
02The paths that usually appear before go-live
The paths that usually appear before go-live
A classic pentest looks for break-ins: no legitimate identity, or a bypassed control. An agent assessment looks for the wrong door: the call is authorised, but this task should never have issued it. These paths show up in the lab without theatrical exploit strings.
Indirect prompt injection
Mail, attachments, web pages and ticket text enter the working context, so their authors gain a vote on this turn. A normal-looking travel PDF is enough to turn “pre-check” into “send outward for verification”.
RAG and memory overreach
If retrieval is similarity-only and memory is shared across users, a neighbouring team’s file and a planted “from now on always…” note become policy. The model did not decide to leak; it summarised what it was allowed to see.
Tool misuse and MCP supply chain
Put a summary task and a privileged API in the same toolbox and one sentence can switch tracks. If an MCP server is trusted by name, not by pinned version, a lookalike can register an extra export tool at runtime.
Excess privilege and irreversible actions
A “check reimbursement status” agent that can also export tables, send mail and edit vendors is no longer a lookup assistant. Send, delete, pay, change grants, run commands — if they can fire with no human confirm, “please be careful” in the prompt has no force.
03How to use it: “yes” without evidence is a fail
How to use it: “yes” without evidence is a fail
Do not send this as a survey for the business team to tick “yes”. Every item needs evidence: an account, a tool list, a short demo, a log line. The demo can be short. “We already thought about that,” with nothing to show, does not pass.
- 01
Pick one real agent
Write down who it acts as, which business it serves, which data and tools it holds. An abstract “our AI” cannot be checked.
- 02
Draw the boundary first
Model, agent, knowledge base, MCP, accounts, external inputs and memory on one page. Someone who did not build it should still see where data enters and leaves.
- 03
Three blockers stop production data
If three of the key questions resolve to “shared dev account, we will trim tools later, logs are not wired yet,” the agent may stay in an internal trial. It should not touch production data.
- 04
Fixes must be retested
A prompt edit with no regression item is not a fix. Every path you found should become a test you can walk again.
04Thirty pre-launch checks
Thirty pre-launch checks
How an AI agent differs from a classic LLM app
- It can call tools, change data or send messages with no human confirm — it is not text-only.
- The evaluation scores the action trace (what was called, as whom, where data went), not merely model refusals.
- On failure it stops and asks a person; it does not silently upgrade to a stronger tool to finish the job.
- “We use a safety-aligned model” is not written as this agent’s risk conclusion.
Assets and trust boundary
- Model, agent, knowledge base, MCP, accounts and external inputs are on one diagram.
- Someone who did not build the system can see where data enters and leaves.
- Memory is a node of its own, not decoration next to the model.
- An agent that can browse or read attachments does not share an all-powerful runtime with one that can touch internal lists.
Prompt injection
- Mail, attachments, web pages, ticket text and external APIs are listed as untrusted input.
- Untrusted text cannot rewrite the current goal or be pasted into high-privilege tool arguments.
- Browse or parse output is source-tagged and cannot become an internal query or a send body on its own.
- A secrecy line in the system prompt is never the only control.
RAG / memory
- Retrieval carries the caller’s identity and filters at fetch time — it is not “search everything, then ask the model to be discreet”.
- The vector store is split by tenant or department.
- Document IDs and permission labels used in an answer can be opened for review.
- Memory is isolated by user and task, with expiry, revoke and audit; cross-user reuse is treated as an overreach channel.
Identity & permission
- A tool call can be traced to a person, a team and a task — not only to “the assistant”.
- Initiator, authorisation scope, execution identity and the tool’s own credentials are recorded separately; there is no single “agent account” for every tool.
- The execution identity is not a shared dev or debug account, and is not reused by other business teams.
- The minimum permission to finish the main task is written down; anything extra has an owner, or it is removed.
Tool / MCP
- Tools the current task actually needs are listed; extras are removed by default.
- Maintainer, source, pinned version and update approval for each MCP / plugin can be shown; unclear origin is treated as unauthorised software.
- The tool list observed at startup is archived; adding or dropping an entry is a permission change, not a hot reload the model should “adapt to”.
- Credentials issued to MCP are scoped to the task: a search token cannot export, delete or administer.
Sensitive action
- Send, delete, pay, change grants and run commands cannot happen with no one confirming.
- Irreversible actions have their own approval; the model cannot fire them because it “seems necessary”.
- Writes outside the main task have been stripped.
- Memory or page content cannot be copied straight into high-privilege parameters.
Logging & evidence
- Input, retrieved snippets, tool request, tool response and human approval can be assembled into one task’s evidence chain.
- Application, gateway, model-session and MCP logs can be lined up to the same person and task.
- On incident you can cut tools, revoke tokens and freeze memory without shutting the entire model service.
- Paths already found are regression tests; after a fix, the same path is walked again.
Thirty pre-production checks
- 01 Tool calls trace to a person and a task: can a log line be followed to a named person, team and task, not just “the assistant”?
- 02 Execution identity is separate from development: are production tasks still using a dev, debug or laptop session?
- 03 Four identity layers are recorded apart: initiator, authorisation scope, execution identity, and the tool’s own credentials?
- 04 Write access can be cut in five minutes: can someone disable writes, revoke tokens and freeze memory that fast?
- 05 Retrieval is filtered by caller identity: do knowledge / ticket / customer stores fetch by identity, not a god service account?
- 06 Knowledge is isolated by tenant or department: is the vector store namespaced, not a mixed index for everyone?
- 07 Citations can be opened: can document IDs and permission labels used in an answer be opened on the spot?
- 08 Memory is isolated by user and task: is cross-user reuse forbidden, with expiry, revoke and audit?
- 09 External input is marked untrusted: do mail, files, pages, tickets and external APIs take an untrusted path?
- 10 Untrusted text cannot rewrite the goal: can outside content change the current task objective directly?
- 11 Browse output cannot become tool arguments: is parsed text blocked from becoming a query, recipient or command?
- 12 Prompt secrecy is not the only control: are there also tool cuts, approvals or retrieval filters?
- 13 The current task’s tool list is trimmed: are only tools for the main task attached, extras removed by default?
- 14 MCP origin, version and approval can be shown: maintainer, image or repo, pin, and who signs updates?
- 15 The live tool list is archived and change-controlled: are registered functions stored, and is every add/drop a permission change?
- 16 MCP credentials are least-privilege for the task: does a search server hold only a search token, not export or delete?
- 17 Failure must not auto-upgrade tools: on error, does it stop and ask, rather than call a stronger API?
- 18 Shell / command execution is sandboxed and reviewed: is a coding or ops agent in an isolated workspace with an allow-list?
- 19 Irreversible actions need a human by default: can send, delete, pay, grant-change or command-exec fire unattended?
- 20 Send / delete / grant-change cannot be the model’s private decision: are recipients, objects and scope bound to a directory?
- 21 Writes outside the main task are gone: can a lookup-only agent still export tables, send mail or edit vendors?
- 22 Memory or pages cannot fill high-privilege parameters: are they blocked from privileged tool arguments?
- 23 One task produces a full evidence chain: input, snippets, request, response and approval together?
- 24 Application, gateway, model and MCP logs line up: can four records be joined to the same task and person?
- 25 Found paths are regression tests: is the path a fixed test, not something mentioned once?
- 26 Fixes are retested on the same path: was the same input replayed, or was only the prompt edited?
- 27 There is a switch to cut tools and freeze memory: can the action surface stop without taking down the model service?
- 28 The asset and trust-boundary diagram is clear: model, agent, knowledge, MCP, accounts, external input, data in and out on one page?
- 29 An isolated environment has been walked with read-only tools: are identity, tool list, data and exit explained before any write?
- 30 Three or more “we will fix it later” means no production data: shared dev account, tools later, logs not wired — if three appear, is production data explicitly refused?
Assessment template
- Asset and trust-boundary diagram: someone who did not build it can follow data in and out.
- Risk register: object, precondition, impact, currently exploitable.
- Attack path: input → retrieval or tool call → result, short enough to retell.
- Evidence: input digest, critical context, tool request and response, policy hit or miss; a third person can check.
- Severity in business terms: whose data, which action, outage or leak, reversible or not.
- Remediation: cut tools, add approvals, filter retrieval, split identity; “stronger prompt” is never the only measure.
- Retest: each fixed item is walked on the same path.
- Management summary: in two pages, what was tested, the three things that matter, conditions for go-live.
05What an assessment should leave behind
What an assessment should leave behind
If the engagement ends on “overall risk is high” plus a product screenshot, leadership cannot decide and engineering cannot patch. This is the default pack. Missing any item means the assessment is not done.
Asset and trust-boundary diagram
Model, agent, knowledge base, MCP, accounts and external inputs on one page.
Risk register
Each row has an object, a precondition, an impact, and whether it is currently exploitable. “Prompt injection” with no system name is not a row.
Attack paths
Which input, through which retrieval or tool call, to which result. Short enough to retell.
Evidence
Input digest, critical context, tool request and response, whether policy blocked it. A third person must be able to check.
Severity in business terms
Whose data, which action, outage or leak, reversible or not.
Remediation and retest
Cut tools, add approvals, filter retrieval, split identities. Walk the same path again on items marked fixed.
Management summary
In two pages: what was tested, the three things that matter, the conditions for go-live.
06Suggested walkthrough
Suggested walkthrough
- 01Pick one agent that is about to ship. Write down its task, data and tools — not the company AI strategy.
- 02Draw the asset and trust boundary. Treat memory and MCP as their own nodes.
- 03Work the eight mechanism groups, then the thirty tickable checks.
- 04Accept only “yes + evidence”. Yes without evidence, no, and “later” all count as blockers.
- 05Turn blockers into go-live conditions. If you need a third party to read the diagram with you, book a 30-minute architecture review.
Explore the KUROSHIO workspace
Switch between Scope, Explore, Validate and Report in an interactive workspace. See how authorized scope, task orchestration, linked evidence and a report summary fit together, using sample data to illustrate a general assessment workflow.
07Questions
Before you start.
01Is this the same as our annual pentest?
No. A pentest is good at exposure, weak credentials and broken API auth. Many agent incidents use calls that were already authorised; only the goal was rewritten. A clean pentest and a week-two leak of a neighbouring team’s files can both be “correct” — they measured different things.
02We have no MCP, only an internal Copilot. Do we still need this?
Yes. If it can search a knowledge base, call an internal API, write memory or send a message, it is already not a chat window. MCP only makes tool discovery faster. The risk is the action.
03The system prompt already says “do not exfiltrate”. Is that enough?
No. A system prompt is advice, not a kernel enforcement point. When a document’s instruction is more specific and more like the current task, the model follows the more specific line. Egress is decided by the tool list and approvals, not by a promise to keep secrets.
04If all thirty items are “yes”, may we go live?
Only with evidence and a defined scope. “Yes” without an account, tool list, demo or log is still a fail. The checklist finds blockers; connecting production data still depends on irreversible actions and a kill switch.
05What happens in the 30-minute architecture review?
We walk this list against one real agent of yours: identity, tool list, untrusted input, sensitive actions, and whether logs can be lined up. You leave with the obvious blockers and a yes/no on whether a full evidence-backed assessment is warranted. It is not a product demo and not a verbal assurance.
Want to see whether your agent would pass?
Bring one agent that is about to ship, its tool list, and a boundary sketch. In thirty minutes we close the obvious blockers and decide whether a reviewable assessment is worth running.