Question 1

What is Shadow AI?

Accepted Answer

What is Shadow AI?

Shadow AI is any AI tool or service used inside your organization without security or IT approval and oversight: an employee pasting a contract into ChatGPT, a developer routing source code through an unsanctioned coding assistant, or an AI feature quietly switched on inside a SaaS tool you already pay for. It's the AI-era successor to Shadow IT, and it spreads faster because the tools are free, instantly useful, and a single browser tab away.

The reason it matters isn't the tool, it's the data. The work people hand to these assistants is often the most sensitive you hold (customer records, code, contracts, strategy), and once it leaves your perimeter you've lost the ability to control, log, or prove what happened to it.

Question 2

How is Shadow AI different from Shadow IT?

Accepted Answer

Shadow AI is a sharper-edged subset of Shadow IT. Classic Shadow IT is an unapproved app or service; the main risk is that you don't manage it. With Shadow AI the interaction itself is the risk: the act of using the tool moves sensitive data out of your control, and the tool may process, retain, or learn from it. It's also harder to spot, because the AI is increasingly a feature buried inside software you've already sanctioned, not a separate app someone installed.

Question 3

How widespread is Shadow AI, really?

Accepted Answer

How widespread is Shadow AI, really?

More than most security teams assume. Because adoption is bottom-up and invisible, it consistently runs ahead of policy: people start using AI to get their work done long before anyone writes a rule about it, and they rarely volunteer that they're doing it. The honest answer for almost any organization is that the real number is higher than the one your current tooling can see.

Published prevalence figures vary widely by survey, industry, and how it's measured, so treat any single headline percentage with caution. What's not in dispute is the direction: usage is broad, growing, and largely unmeasured unless you instrument for it.

Question 4

Why is Shadow AI a security and compliance risk?

Accepted Answer

Why is Shadow AI a security and compliance risk?

Four ways. Data leakage: sensitive inputs leave your perimeter and may be retained or used to train a third-party model. Prompt injection: assistants that read untrusted content can be steered into exfiltrating data. OWASP ranks prompt injection #1 among LLM risks (LLM01:2025), and EchoLeak (CVE-2025-32711, CVSS 9.3) showed a single crafted email turning Microsoft 365 Copilot into a zero-click data-exfiltration channel.¹²

And compliance: you can't evidence control over a system you can't see. Regulators increasingly expect an inventory of the AI in use and the data it touches. The EU AI Act (Regulation (EU) 2024/1689) sets security, oversight, and record-keeping obligations for higher-risk uses. Shadow AI is, by definition, the part of your AI footprint that no audit trail covers.³

Question 5

How do we actually detect Shadow AI in our organization?

Accepted Answer

How do we actually detect Shadow AI in our organization?

You can't secure what you can't see, and Shadow AI is built to stay out of sight. Discovery means combining signals, not running one scan: network and egress monitoring for traffic to known AI services and their APIs; endpoint or browser inspection to catch text pasted into web tools; OAuth and SaaS analysis to inventory which AI apps employees have connected; and identity analytics for anomalous access.⁴

The reason a plain asset scan comes back clean is that AI is increasingly a feature inside SaaS you've already approved and a paste into a browser tab, not a new app on a laptop. The inventory looks tidy while the exposure runs underneath it. Effective discovery has to see the AI interaction and the data inside it, not just match a domain list.

Question 6

We blocked ChatGPT at the firewall. Isn't that enough?

Accepted Answer

We blocked ChatGPT at the firewall. Isn't that enough?

No, and blocking alone tends to backfire. A blocklist is a list of the tools you already know about, while new assistants ship every week, AI features get embedded inside SaaS apps you've already approved, and people reach the tools anyway from personal devices, phones, or a different network. A hard block doesn't remove the demand. It pushes the same behavior somewhere you can't see it at all.

Visibility beats a blocklist. The defensible goal is to see every AI interaction and the sensitive data inside it, then apply policy where it matters, rather than pretending the activity stopped because one domain returns an error page.

Question 7

What does Shadow AI mean for compliance under GDPR, HIPAA, SOC 2, and the EU AI Act?

Accepted Answer

What does Shadow AI mean for compliance under GDPR, HIPAA, SOC 2, and the EU AI Act?

Shadow AI turns routine compliance obligations into open findings, because the data flow is undocumented. Under GDPR, an unapproved AI vendor handling personal data is an undocumented processor you haven't assessed or contracted. Under HIPAA, pasting PHI into a tool with no Business Associate Agreement can be a reportable disclosure, and you lose the access tracking the rule assumes. Under SOC 2, undocumented AI data flows undercut your monitoring and vendor-risk controls. None of these frameworks has an AI exemption. Existing duties apply to AI data flows as they stand.

The EU AI Act (Regulation (EU) 2024/1689) adds a phased timeline: prohibited practices and AI-literacy duties applied from February 2025, general-purpose AI model obligations from August 2025, and most high-risk-system and governance obligations from August 2026. It expects an inventory of the AI you use and records of the data it touches, which Shadow AI, by definition, can't provide.³

Question 8

What is data poisoning, and how worried should I actually be?

Accepted Answer

Data poisoning is when an attacker corrupts the data an AI model learns from, so the model makes predictable mistakes or carries a hidden backdoor. You should take it seriously: a 2025 study found a model can be backdoored with a small, fixed number of malicious documents, and a poisoned model behaves normally on standard tests, so routine validation misses it. The danger isn't that poisoning is loud; it's that it's quiet. A poisoned model can pass your benchmarks and still fail in an attacker-chosen way on a specific trigger. By the time you notice, it may already be in production, feeding decisions. OWASP lists “Data and Model Poisoning” as a top-10 LLM risk (LLM04:2025).

Question 9

How much data does an attacker need to poison an AI model?

Accepted Answer

Far less than most people assume. Research published in October 2025 by Anthropic, the UK AI Security Institute, and the Alan Turing Institute found that roughly 250 malicious documents can implant a backdoor in a large language model, and that number stayed nearly constant across models from 600 million to 13 billion parameters, regardless of how large the training set was. For a 13-billion-parameter model, those 250 documents were about 0.00016% of the training data. The practical takeaway: “our dataset is too big to poison” is false reassurance, scale doesn't dilute the attack the way intuition suggests. Honest caveat, the study demonstrated a narrow, low-stakes backdoor (the trigger made the model output gibberish), tested up to 13B parameters. The authors note it is not yet established that the same small, fixed document count produces more dangerous behaviours, or holds at frontier scale.

Question 10

We don't train our own models, can we still be affected by data poisoning?

Accepted Answer

Yes. You don't need to train a foundation model to be exposed. The two realistic paths for most organizations are fine-tuning on scraped or public data that already contains poisoned content, and poisoning the documents your retrieval (RAG) system reads at query time. Both can skew outputs without anyone touching your model's weights. This is the most common misconception about poisoning. “We just use an off-the-shelf model” doesn't remove the risk, it moves the attack surface to your fine-tuning data and your retrieval pipeline, which you control and are therefore responsible for.

Question 11

Can my RAG system or internal knowledge base be poisoned?

Accepted Answer

Yes. If an attacker or a careless insider can get a document into the knowledge base your AI reads from, a shared drive, wiki, ticketing system, or any external source you ingest, they can steer the model's answers on specific topics. This needs no access to your model and often no user interaction beyond someone querying the assistant. RAG poisoning is one of the most practical real-world AI attacks precisely because the entry point is mundane: anywhere your pipeline pulls documents from is a place poison can enter. The more connected and autonomous the assistant, the larger the blast radius.

Question 12

Can data poisoning be detected?

Accepted Answer

It's difficult with standard tools, which is exactly the core danger. A poisoned model typically scores normally on benchmark tests and only misbehaves on a hidden trigger. Reliable detection depends on data provenance and lineage tracking, statistical outlier and backdoor-trigger analysis, and adversarial red-teaming, not on accuracy metrics, which a competent attack is designed to pass. This is why “the model works fine in testing” is not evidence of safety against poisoning. Detection has to focus on the integrity of the data and on the model's behaviour under adversarial conditions, not on clean-test-set performance.

Question 13

How do I protect my AI or ML pipeline against data poisoning?

Accepted Answer

Treat your data pipeline as an attack surface, not just storage. The core controls: track data provenance and lineage from raw source to trained model, validate and screen training and fine-tuning data for anomalies, restrict who can write to your RAG corpus, and run adversarial testing against your own models. Continuous, automated monitoring matters because manual review doesn't scale. The underlying principle: you cannot trust a model more than you trust the data it learned from. Integrity controls belong at every stage, ingestion, storage, training, and retrieval, because poisoning can be introduced at any of them.

Question 14

What is prompt injection, and is it a real threat or just theoretical?

Accepted Answer

Prompt injection is when hidden instructions inside content a model reads override what it was actually told to do. It's real, not theoretical: it ranks #1 on the OWASP Top 10 for LLM Applications (LLM01:2025). In June 2025, researchers disclosed EchoLeak (CVE-2025-32711, CVSS 9.3), a zero-click flaw in Microsoft 365 Copilot where a single crafted email could make the assistant exfiltrate internal data. It's especially dangerous for AI agents that can act on what they read. The root cause is architectural: a model processes trusted instructions and untrusted content in the same context, with no built-in way to tell them apart.

Question 15

What's the difference between direct and indirect prompt injection?

Accepted Answer

Direct injection comes from the user typing to the model (“ignore previous instructions and…”). Indirect injection hides the malicious instruction inside content the model later reads, a web page, email, PDF, or even image metadata, so the victim may never see it. Indirect injection is the more dangerous variant because it can be zero-click and scales. EchoLeak was an indirect attack: the malicious instruction lived in an email the model processed, not in anything the user typed. As assistants gain access to more data sources, the indirect attack surface grows with them.

Question 16

What's the difference between prompt injection and jailbreaking?

Accepted Answer

They target different layers. Prompt injection attacks the application, it manipulates what the model does, such as leaking data or making unauthorized tool calls. Jailbreaking attacks the model's safety alignment, it bypasses what the model refuses to do, to produce restricted content. Attackers often chain the two, but the defenses differ, which is why practitioners increasingly separate them. OWASP currently groups both under LLM01:2025, but the distinction matters operationally: prompt-injection defenses focus on input isolation, least-privilege tooling, and output monitoring; jailbreak defenses focus on model alignment and content filtering. Defending one does not cover the other.

Question 17

What is an adversarial example or an adversarial patch?

Accepted Answer

An adversarial example is an input deliberately altered, often imperceptibly to a human, to make an AI model misclassify it. An adversarial patch is the physical-world version: a crafted sticker, pattern, or printed patch placed on an object or in an image that reliably fools a vision model, such as making it misread a sign or fail to detect a person. These attacks exploit the gap between how models “see” and how humans see. They matter most for systems acting on visual or sensor input, autonomous platforms, inspection, surveillance, ISR, where a single misclassification has physical consequences.

Question 18

What is shortcut learning, and is it a security risk?

Accepted Answer

Shortcut learning is when a model latches onto an easy, spurious correlation in its training data instead of the real underlying feature, for example, classifying medical X-rays by the scanner's watermark rather than the actual pathology. It isn't an attack by itself, but it's a genuine security and reliability problem: it makes models brittle, easy to fool, and prone to failing unpredictably. Attackers can exploit shortcuts they discover, and even with no attacker, a model relying on a shortcut fails silently the moment real-world data drifts away from the spurious pattern it actually learned. Standard accuracy testing tends to hide it, because on data that shares the shortcut the model still scores well.

Question 19

What's the difference between a data poisoning attack and a backdoor?

Accepted Answer

A backdoor is a specific, stealthier type of data poisoning. General poisoning degrades a model's overall accuracy or skews its behavior broadly. A backdoor leaves normal performance almost entirely intact but makes the model fail in an attacker-chosen way only when it sees a specific trigger, like a guard who waves through anyone wearing one particular item. That selectivity is what makes backdoors dangerous: there's no general accuracy drop to tip you off. The model is a sleeper that looks healthy until the trigger appears, which is why the 250-document finding (see the Data poisoning tab) is concerning specifically for backdoors.

Question 20

Who is liable when an AI system makes a bad or manipulated decision?

Accepted Answer

It's unsettled, and that uncertainty is itself a risk. Today, responsibility generally tracks back to the organization that deployed the system, but attribution is genuinely hard: AI agents often run under a human's credentials, so audit logs show the user taking an action, not the agent. Expect this to be shaped by regulation such as the EU AI Act and emerging case law. For now, the defensible position is governance you can evidence: know what each AI system can access, log and monitor its decisions, keep a human approval gate for high-impact actions, and be able to show the bounds you set on its autonomy.

Question 21

What does the EU AI Act require for AI security?

Accepted Answer

The EU AI Act (Regulation (EU) 2024/1689) requires “high-risk” AI systems to meet security and robustness obligations under Articles 9–15. Article 15 specifically requires resilience against attempts to manipulate the system, and names data poisoning, model poisoning, and adversarial examples among the threats high-risk systems must be designed to withstand. The Act applies extraterritorially: it covers providers and deployers whose AI output is used in the EU, regardless of where the company is based. High-risk obligations apply from 2 August 2026, with penalties up to €15 million or 3% of global annual turnover for non-compliance (and up to €35 million or 7% for prohibited practices). Note: amendments under the EU “Digital Omnibus” may adjust some high-risk deadlines, these were still in negotiation as of 2026, so verify the current date before relying on it.

Question 22

What does AI security governance actually involve?

Accepted Answer

AI security governance means knowing every AI system you run, what data each can access, and who can shut it down, then putting controls and evidence around that. In practice: an inventory of AI systems and agents, data-governance and provenance controls, logging and monitoring of AI decisions, human oversight for high-impact actions, and documented limits on each system's autonomy. The reframe driving boards in 2026 is moving from “how many AI tools do we have?” to “how much authority have we delegated, to which systems, under what controls?” Governance that can answer the second question is what regulators and boards increasingly expect.

Question 23

What is AI data leakage, and how does it happen?

Accepted Answer

AI data leakage is the exposure of sensitive information through interaction with an AI tool — most commonly when an employee pastes or uploads confidential content into a chatbot, assistant, or AI-powered feature, and that content leaves the organisation’s control. OWASP formalised this as LLM02:2025 Sensitive Information Disclosure in its 2025 Top 10 for LLM Applications. OWASP’s framing covers both directions: data flowing into a model (prompts, uploaded files, RAG context, system prompts) and data flowing out of it (a model that regurgitates training data, or surfaces one user’s information in another’s session). In practice, the dominant vector is mundane. Samsung’s 2023 incident — the most-cited real-world case — involved three separate leaks within roughly 20 days of employees gaining ChatGPT access: one engineer pasted semiconductor-equipment source code to debug it, another fed a recorded internal meeting transcript to generate minutes, and a third submitted code to optimise a chip-yield test sequence. Samsung banned generative AI on company devices shortly after. No system was hacked; the data left through a text box, by someone doing their job. A smaller class of leakage is platform-side. In March 2023 a Redis client bug in ChatGPT allowed some users to see other users’ conversation titles, and exposed name, billing address, card type, expiry date, and the last four digits of payment cards for approximately 1.2% of ChatGPT Plus subscribers active during a roughly nine-hour window. This is the rarer mechanism, but it illustrates that leakage is not only a user-behaviour problem — it also includes trusting a third party’s runtime. Honest caveat, “leakage” covers at least three distinct threat models: egress through prompts (a data-handling failure), vendor platform bugs (a third-party reliability failure), and model memorisation of training data (an AI-design failure). Most enterprise leakage is the first kind, which is a data-governance problem, not a uniquely AI-model problem. Conflating them tends to produce the wrong controls.

Question 24

What types of data are most commonly exposed through AI tools?

Accepted Answer

The categories that recur across independent vendor telemetry are source code, customer and client data, internal-only business documents, credentials and API keys, and regulated personal data (PII, PHI, financial records). Cyberhaven, analysing traffic from 1.6 million workers, found the top confidential categories pasted into ChatGPT were sensitive or internal-only data (319 incidents per week per 100,000 employees), source code (278), and client data (260). Harmonic Security’s Q2 2025 study — analysing one million prompts, 20,000 files, and over 300 AI applications — independently surfaced the same categories, with credit-card data, customer profiles, and employee PII prominent. The more important finding is that files are far more dangerous than prompts. Harmonic found roughly 22% of uploaded files contained sensitive data versus around 4.4% of prompt text, and files accounted for 79.7% of stored credit-card exposures, 75.3% of customer-profile leaks, and 68.8% of employee-PII incidents. People self-censor when typing but upload entire spreadsheets and documents without filtering. Most DLP and policy conversations over-index on prompt text and under-cover file uploads — which is where the majority of structured sensitive data actually moves. Honest caveat, every figure here is vendor telemetry from companies whose products detect AI data loss, drawn from their own customers’ traffic — not a neutral cross-industry sample. The categories are credible and corroborate across three independent vendors; the precise percentages should be treated as indicative rather than authoritative. There is no peer-reviewed, cross-industry baseline for AI data leakage prevalence.

Question 25

How is AI data leakage different from what a traditional DLP tool catches?

Accepted Answer

Traditional data loss prevention was designed around three chokepoints: email gateways, file storage, and endpoint file operations (USB transfers, downloads, attachments). Prompt leakage routes around all three. Pasting text into a browser-based AI tool produces no attachment, no recognisable file transfer, and no distinct network event — just encrypted HTTPS to a legitimate domain, indistinguishable at the perimeter from any other web request. The browser is the specific blind spot. Endpoint DLP can sometimes detect a clipboard-copy event, but it captures the paste without destination context — it cannot tell whether the text went into a local document or into a chatbot prompt, which is exactly the context needed for a policy decision. Where AI applications use WebSocket connections or certificate pinning, as many native desktop and mobile clients do, even SSL-inspecting proxies fail to decrypt the traffic. Broadcom’s own product documentation for Symantec DLP explicitly describes gaps in detecting ChatGPT via WebSocket transport and in the Firefox browser. Effective detection requires understanding that a request is going to an AI endpoint, decrypting and parsing the prompt, classifying its content, and linking it to an identity — a fundamentally different pipeline from filtering file transfers or matching email bodies against regex patterns. Coarser signals tell you only that an AI service was reached, not what was sent. This is a different control surface — the prompt boundary — not a configuration problem on legacy DLP. Honest caveat, modern DLP suites (Purview, Forcepoint, Zscaler, Netskope, and others) have added AI-aware browser coverage, so the claim that DLP cannot see AI traffic is increasingly an oversimplification. The accurate statement is narrower: DLP configured the traditional way — for email, file, and endpoint — misses the prompt layer, and even AI-aware DLP degrades on pinned native apps, personal accounts on unmanaged devices, and WebSocket transport. Coverage is improving; completeness is not solved.

Question 26

Can I stop sensitive data from reaching AI tools without blocking them entirely?

Accepted Answer

Yes — and the prevailing security guidance is explicitly not to blanket-block. Blocking drives usage underground: when AI tools are prohibited, employees switch to personal accounts on personal devices, which makes the same activity invisible to corporate controls. LayerX’s enterprise telemetry found that a majority of workplace AI usage already runs through non-corporate accounts — precisely the unmanaged path that sits outside any policy enforcement. Prohibition reduces visibility without reducing the underlying demand. The mechanism that enables governed AI use is inspection and policy at the prompt boundary: classify the content of each prompt or file upload in real time, then redact the sensitive element, warn the user, require a business justification, or block the specific sensitive content — while allowing the remainder of the prompt through unaffected. A prompt containing a customer record gets the record stripped or flagged; the same prompt with no sensitive content passes freely. OWASP’s LLM02 mitigations point in the same direction: input sanitisation, tokenisation and redaction, and access controls — not prohibition. Honest caveat, prompt-boundary enforcement works on the managed surface — corporate devices, managed browsers, sanctioned accounts. It cannot reliably cover an employee’s home laptop using a personal AI account without also controlling the endpoint or having network reach there. Classification is also imperfect: false negatives let some sensitive content through, and over-aggressive filters train users to ignore warnings. Selective prompt-boundary policy materially reduces leakage and improves auditability; it does not eliminate risk from the unmanaged path.

Question 27

How do I find out if sensitive data has already left through AI tools in my organization?

Accepted Answer

Retroactive discovery is hard, and you should be precise about what is actually knowable. The signals available after the fact are mostly circumstantial: DNS, firewall, and proxy logs show which AI domains were reached and at what volume; IdP and SSO logs show who authenticated to sanctioned AI applications; OAuth grant logs reveal which AI plug-ins employees have connected to your SaaS; CASB and SWG logs may show upload byte volumes. These tell you who used which AI tool and when — not what was inside the prompts, unless you were doing TLS-breaking content inspection at the time. Prompt content that was never captured cannot be reconstructed. For sanctioned enterprise tiers you have a real audit source. ChatGPT Enterprise and Team retain interaction logs accessible to admins. Microsoft 365 Copilot interaction data flows through Purview Audit, eDiscovery, and Communication Compliance, which can be searched retroactively. For shadow and personal-account usage those records sit in OpenAI’s or Google’s account, not yours. Discovery there relies on the same signals used to surface shadow AI more broadly: email metadata from signup confirmations, endpoint browser history, EDR clipboard and process telemetry, and identity-source correlation. In practice, you can build a reasonable inventory of exposure — which tools, which users, what volume — from logs you already hold, and audit content for sanctioned tools. What you generally cannot do retroactively is prove what sensitive content left through unsanctioned tools before purpose-built monitoring was in place. That data was never recorded. Forward-looking prompt-boundary inspection is what converts the answer from “we believe no significant leakage occurred” to “we can show it didn’t.” Honest caveat, a clean DNS log or quiet Purview audit does not mean no leakage occurred — it means no leakage occurred through the channels you were watching. Personal accounts, BYOD, native pinned apps, and any period before monitoring was deployed are structural blind spots. The defensible statement to leadership or an auditor is: here is what we can see and here is what we structurally cannot — not that the absence of evidence is evidence of absence.

	Shadow IT	Shadow AI
What it is	Unapproved apps and services	Unapproved AI tools and AI features
Core risk	Unmanaged software in your estate	Sensitive data leaving your perimeter through the interaction itself
Where it hides	Installed apps, signups	Browser tabs and AI features embedded in approved SaaS
Why it's hard to see	Discoverable by network/asset scans	The interaction looks like normal traffic; data exposure is in the content¹
Primary defense	Asset inventory, access control	Real-time visibility into AI interactions, data classification, policy at the boundary

Your team will use AI. The question is whether you'll know what it's doing with your data.

Protection that runs on the machine.

Questions, answered.

See what your team is really sending to AI.