Techniques

Adversarial methods affecting AI models and applications, with non-weaponized examples, mitigations, and framework mappings.

CATEGORYLAYERROWS

SHOWING 1–10 / 45 TECHNIQUES

PAGE 1 / 5

Prompt manipulationAAT-001

Direct prompt injection

User-controlled instructions change an AI system's behavior, priorities, or output in a way the application did not intend.

EXAMPLE SCENARIO

A user places conflicting instructions in a support request so the assistant ignores its assigned support scope and attempts an unrelated action.

applicationmodelpolicy bypassunauthorized behavior

Mitigations and mappings

Mitigations

Separate instructions from untrusted data
Enforce authorization outside the model
Constrain available actions and validate outputs

Frameworks and sources

OWASP LLM01:2025 · MITRE ATLAS AML.T0051.000

OWASP LLM01:2025

Prompt manipulationAAT-002

Indirect prompt injection

Instructions embedded in external content influence a model when an application retrieves, reads, or summarizes that content.

EXAMPLE SCENARIO

A document indexed by a research assistant contains hidden instructions that redirect the assistant when the document is retrieved.

applicationretrievalagentgoal redirectiondata disclosuretool misuse

Mitigations and mappings

Mitigations

Treat retrieved content as untrusted
Preserve data and instruction boundaries
Require approval for consequential actions

Frameworks and sources

OWASP LLM01:2025 · MITRE ATLAS AML.T0051.001

OWASP LLM01:2025

Prompt manipulationAAT-003

Jailbreaking

Conversational framing, transformations, or multi-step interaction are used to bypass model-level behavioral restrictions.

EXAMPLE SCENARIO

A user repeatedly reframes a disallowed request as fictional analysis until the model begins responding outside its intended safety policy.

modelapplicationsafety-control bypassrestricted output

Mitigations and mappings

Mitigations

Use layered input and output controls
Test multi-turn behavior
Keep authorization independent of refusal behavior

Frameworks and sources

OWASP LLM01:2025

Prompt manipulationAAT-004

Multimodal prompt injection

Adversarial instructions are delivered through images, audio, documents, or other non-text inputs processed by a multimodal system.

EXAMPLE SCENARIO

A low-contrast instruction embedded in an uploaded image influences an assistant that is expected only to describe the visible scene.

modelapplicationhidden instruction deliverypolicy bypass

Mitigations and mappings

Mitigations

Inspect every input modality
Limit cross-modal authority
Test imperceptible and transformed inputs

Frameworks and sources

OWASP LLM01:2025

Information discoveryAAT-005

System prompt extraction

A user attempts to recover hidden application instructions, policies, routing logic, or other system-context material.

EXAMPLE SCENARIO

A user asks the assistant to reconstruct its internal instructions through summaries, translations, or incremental disclosure.

applicationmodelinstruction discoverycontrol discovery

Mitigations and mappings

Mitigations

Do not store secrets in prompts
Assume system instructions can be inferred
Enforce controls in application code

Frameworks and sources

OWASP LLM07:2025

Information discoveryAAT-006

Sensitive context extraction

Model context, connected data, tool results, or conversation state are manipulated to disclose information outside the requester's authorization.

EXAMPLE SCENARIO

A shared assistant is prompted to summarize fragments retrieved from another tenant's documents after an isolation failure.

applicationretrievalagentdata disclosurecross-tenant access

Mitigations and mappings

Mitigations

Apply authorization before retrieval
Minimize context
Redact secrets and test tenant isolation

Frameworks and sources

OWASP LLM02:2025

Information discoveryAAT-007

Training data extraction

Repeated or targeted queries attempt to elicit memorized examples, rare strings, or sensitive material from model training data.

EXAMPLE SCENARIO

An evaluator varies prompts around a rare phrase to determine whether a model reproduces a memorized private record.

modelmemorized data recoveryprivacy compromise

Mitigations and mappings

Mitigations

Remove sensitive training material
Measure memorization
Rate-limit and monitor extraction patterns

Frameworks and sources

OWASP LLM02:2025

Information discoveryAAT-008

Membership inference

An attacker analyzes model outputs or confidence behavior to infer whether a particular record was present in training data.

EXAMPLE SCENARIO

A researcher compares a classifier's response patterns for known and unknown samples to estimate whether a person's record was included.

modeltraining-set discoveryprivacy compromise

Mitigations and mappings

Mitigations

Limit confidence exposure
Use privacy-preserving training
Evaluate membership leakage before release

Frameworks and sources

OWASP LLM02:2025

Information discoveryAAT-009

Model extraction

Systematic queries are used to approximate a hosted model's decision boundary, behavior, or functionality in a substitute model.

EXAMPLE SCENARIO

An actor collects a large set of API input-output pairs and uses them to train a lower-cost model that imitates the service.

modelapplicationmodel theftcapability replication

Mitigations and mappings

Mitigations

Rate-limit and monitor systematic querying
Reduce unnecessary output detail
Use contractual and technical access controls

Frameworks and sources

OWASP LLM03:2025

Model manipulationAAT-010

Adversarial example and evasion

Inputs are intentionally modified so a model misclassifies, overlooks, or incorrectly scores an otherwise recognizable sample.

EXAMPLE SCENARIO

Small visual changes cause an image classifier to assign the wrong category while the image still appears unchanged to a person.

modeldetection evasionmisclassification

Mitigations and mappings

Mitigations

Evaluate adversarial robustness
Use input transformations and ensembles where appropriate
Monitor distribution shifts

Frameworks and sources

MITRE ATLAS