First, give conclusions that can be used for decision-making
Business Agent usually handles user input, web page, mail, document and tool output at the same time, all of which may be subject to malicious or conflicting instructions. If privileges rely only on words such as “do not read other customer data”, the model, once miscalculated, may call for a high-risk tool. The correct design is to put user identity, Agent identity, role, resource range, action line, and approval rules into a definitive system; the model only proposes a candidate action, and the service decides whether to allow it.
What conditions need to be identified before judgement is made?
The same question may have different answers under different business, data and project phases. It is suggested that the following conditions be checked and that the common findings on the web be incorporated into their own projects.
Suggested order of advance
First, we'll be clear about the target and the border.
List and rank all tools and data.
Validation Key Dependence
Create independent identities and minimum permissions for users and Agent.
Development of assessable outcomes
Validation of resources, parameters, scales and operational status on the tool level.
Make sure you decide the next step with the real results.
Increase manual approval, audit and emergency decommissioning of high-risk operations.
How do you understand it in the actual business?
The mail assistant read an email containing malicious instructions, and the model attempts to call on the client to export the tool. If the tool service believes only in the model, it can cause data leak; if the service provider executes the verification based on current employee status, client affiliation, and export approval, the request is denied and security events are recorded.
The easiest pit to step on.
Considers a longer system hint equal to a stronger privileges control
Multiple Agents share a super-admin account.
Record only the model answers, not the tool parameters and the results of the implementation
How should we end up receiving and confirming?
Security tests should cover injections, over-authorization, tampering with parameters, return of tools to contamination, repeated execution and clearance. Even if the model output error command is not followed, external authorization levels must stop movement and leave a complete audit chain of users, Agent, tools and results.
When preparing to communicate with suppliers or internal teams, it is recommended that current processes, representative samples, existing systems, planning time and budget levels be brought. First, the unknown items are clearly marked, and then the decision is made to use diagnostics, PoC, fixed-range projects or ongoing research and development, which is usually more reliable than a direct demand for a price and duration without borders.