LLM09:2025 - Misinformation

Misinformation is the ninth risk in the OWASP Top 10 for LLM Applications 2025. It occurs when models generate false but credible-sounding content (commonly called hallucinations) and applications consume this output as authoritative data without validation, confidence checks, or human review gates, leading to incorrect decisions, legal liability, and eroded user trust.

Overview

Large language models generate text probabilistically and have no inherent concept of truth. They can produce confident, well-structured responses that are factually incorrect, internally inconsistent, or entirely fabricated. This is a known limitation, not a bug, but it becomes a security and reliability risk when application code treats LLM output as verified fact. In critical domains such as healthcare, finance, and legal services, acting on hallucinated content can cause real harm. From a SAST perspective, the risk manifests as code patterns that blindly trust LLM output: storing it in databases, displaying it to users, or feeding it into business logic without any validation layer. The distinction between "the model is wrong" and "the application accepts wrong data" is critical. Misinformation becomes a vulnerability when the application architecture fails to account for the inherent unreliability of generated content.

What Radar Detects

• LLM output used directly as authoritative data in business logic.** Code paths where LLM responses are passed directly to decision-making logic, database writes, or calculations without intermediate validation against known data sources or business rules.

• Missing confidence thresholds before using LLM output.** API integrations that consume LLM responses without checking model confidence scores, logprobs, or similar quality indicators to filter out low-confidence or ambiguous results.

• LLM-generated content stored in databases or displayed to users without review flags.** Application flows that persist or render LLM output without marking it as AI-generated, pending review, or unverified, making it indistinguishable from human-authored or validated content.

• Absence of human review workflows for LLM-generated content in critical paths.** Critical business processes in domains such as medical, legal, or financial services where LLM-generated content flows directly to end users or downstream systems without a mandatory human review step.

• Missing source attribution on LLM responses.** LLM output presented to users without citations, provenance markers, or links to supporting sources, preventing users from independently verifying the accuracy of the information.

• LLM output used to make automated decisions without fallback or override mechanisms.** Automated workflows driven by LLM output that lack fallback logic, manual override capabilities, or escalation paths when the model produces unexpected or low-confidence results.

• Missing output schema validation on structured LLM responses.** LLM responses expected in structured formats (JSON, XML, specific data schemas) that are parsed and used without validating that the output conforms to the expected schema, field types, and value constraints.

• LLM-generated content used in compliance or regulatory outputs without audit trail.** Generated content that flows into compliance reports, regulatory filings, or audit documentation without traceability markers indicating that the content was AI-generated and has not been independently verified.

Related CWEs

CWE-345 (Insufficient Verification of Data Authenticity), CWE-707 (Improper Neutralization).

See the CWE Reference for details.

Overlap with OWASP Top 10 Web

Misinformation indirectly relates to A08:2025 Software or Data Integrity Failures in the traditional OWASP Top 10. Both categories address the risk of trusting unverified data from an external source. Where A08 focuses on unsigned updates, deserialization attacks, and CI/CD compromise, LLM09 concerns the integrity of data generated by a probabilistic model, treating the LLM as an untrusted data source whose output requires verification before use.

Prevention

• Implement validation layers for all LLM output used in business logic. Cross-reference generated content against known data sources, databases, or rule engines before acting on it.
• Add confidence scoring and enforce minimum thresholds. Reject or flag LLM responses that fall below acceptable confidence levels for the given use case.
• Require source attribution for all LLM-generated content. Include citations, references, or links to supporting documents so users can verify claims independently.
• Implement mandatory human review workflows for LLM output in critical domains such as healthcare, finance, legal, and compliance.
• Flag LLM-generated content clearly in the user interface. Distinguish AI-generated text from human-authored or verified content through visual indicators or metadata.
• Provide manual override and escalation mechanisms for automated decisions driven by LLM output, ensuring a human can intervene when the model produces incorrect or ambiguous results.
• Validate structured LLM output against expected schemas before parsing. Enforce field types, required fields, and value constraints on JSON or structured responses.
• Maintain an audit trail for all LLM-generated content used in compliance or regulatory contexts, recording the model version, prompt, timestamp, and review status.

Next Steps