Most AI agent data leaks happen when one agent combines three things: access to private data, exposure to content you don't control, and a way to send information outside your organisation. Remove or tightly control any one of those three, and the most common leak path closes.

Why this matters now

In April 2026, the Cloud Security Alliance and Token Security reported that 65% of organisations had experienced at least one cybersecurity incident caused by AI agents in the past year. Of those, 61% involved sensitive data exposure. The same research found that 60% of organisations couldn't terminate a misbehaving agent, and 67% lacked audit trails across all the data channels their agents could reach.

Most of that data comes from the US, where agent adoption is further along. That gives Australian organisations a useful advantage: the chance to learn from these incidents before our own agents reach the same scale.

The pattern behind most leaks

OWASP's 2026 reporting found prompt injection is still the main cause of agent security failures in production. The reason is structural. A language model can't reliably tell the difference between an instruction from you and an instruction hidden inside an email, a document or a web page it has been asked to read.

That's why security researchers talk about a "lethal trifecta". An agent that can read private data, process untrusted content, and communicate externally can be turned into an exfiltration tool by a single well-placed piece of text.

Real incidents often start somewhere less dramatic. In April 2026, Vercel disclosed a breach in which attackers compromised a third-party AI tool that an employee had connected with OAuth access, then used that access to move into Vercel's systems.

What to do about it

The practical controls are mostly about design, not products:

1. Give every agent its own identity with the least access it needs. On Microsoft, that means managed identities and Entra rather than shared keys.
2. Separate the agents that read untrusted content from the agents that touch sensitive data, wherever you can.
3. Put an AI gateway between agents and models, so every call is authenticated, logged and rate-limited. We wrote about why this layer matters in We Nearly Shipped the Wrong AI Architecture.
4. Require a human approval step before an agent sends data outside the organisation.
5. Make sure you can switch an agent off, and that you can see what it touched.

FAQ

What is prompt injection? It's when text inside content an AI reads, such as an email or document, is interpreted as an instruction. The model follows it because it can't reliably distinguish data from commands.

Is Microsoft 365 Copilot affected by this? Any AI that reads content is exposed to some degree. Copilot only surfaces data a user can already access, which limits the damage, but permissions still need to be right.

Does the Privacy Act cover AI agents? If an agent uses personal information to make, or substantially help make, decisions that significantly affect people, new transparency obligations apply from 10 December 2026.

Planning your first production agent? Our AI Readiness Assessment reviews identity, network and gateway design before anything goes live.

About the author: Timotius Devin is AI Lead at PerData Technology Partners in Melbourne. At PerData Devin helps Australian organisations design, govern and deploy AI on the Microsoft platform.

Sources

1. Kiteworks, citing Cloud Security Alliance and Token Security, AI Agent Security Incidents Hit 65% of Firms in 2026: https://www.kiteworks.com/cybersecurity-risk-management/ai-agent-security-incidents-2026/
2. Help Net Security, Prompt injection still drives most agentic AI security failures in production (OWASP): https://www.helpnetsecurity.com/2026/06/11/owasp-prompt-injection-ai-security-failures/