Secure AI document generation with untrusted content

Treat every source document, retrieved passage, template field, image URL, and generated sentence as untrusted data. The model may propose document intent, but it should not gain ambient authority to fetch arbitrary resources, choose destinations, weaken validation, or release files. Secure AI document generation separates instructions from content, validates a typed intermediate representation, executes inside bounded capabilities, and requires evidence before release.

The document is part of the attack surface

An AI document workflow combines user instructions, uploaded files, retrieval results, templates, brand assets, model output, format libraries, storage and delivery integrations. Untrusted text can cross several of these before it reaches a final PPTX, DOCX, XLSX or PDF.

Prompt injection is one risk, but not the only one. A malicious source can also attempt to trigger external fetches, inject hyperlinks, reference local paths, expand into a resource-exhausting document, exploit a parser, hide text outside the visible page, or persuade a reviewer to approve a different artifact.

OWASP’s current LLM prompt-injection guidance recommends input validation, clear separation between instructions and data, output monitoring, least privilege, human controls for high-risk actions, and comprehensive monitoring. A document system needs to apply those controls beyond the prompt itself.

Draw the trust boundaries before choosing a model

A practical threat model distinguishes at least six zones:

The model never moves data from one zone to another by assertion. It cannot label a source “trusted,” authorize an external fetch, or declare its own output approved.

Keep instructions and content structurally separate

Do not concatenate system rules, retrieved passages, and user files into one undifferentiated prompt. Mark source material as data and give the model a constrained output schema.

A good intermediate representation contains document intent: sections, blocks, table data, chart series, style tokens, citations, and semantic locators. It should not accept raw shell commands, arbitrary code, unrestricted file paths, or renderer-specific XML fragments unless the workflow explicitly requires and isolates them.

Reject unknown fields. Constrain lengths, counts, enums, URLs, media types, and numeric ranges. Normalize Unicode and dangerous control characters according to a documented policy. Validation cannot prove the model’s statements are true, but it can prevent a prose answer from becoming executable authority.

Make resource acquisition a separate operation

External images, fonts, linked documents, and templates create network and parsing risk. A renderer should not fetch arbitrary URLs embedded by the model.

Use an acquisition layer that:

• allows only approved schemes and destinations;
• blocks loopback, private-network, metadata-service, and local-file targets;
• enforces redirects, byte limits, MIME checks, timeouts, and decompression limits;
• scans or parses in isolation where appropriate;
• stores content under a hash and passes the renderer a bounded reference;
• records the source and transformation history.

The renderer receives approved bytes, not permission to browse.

Bound document complexity

File generation is computational work. Attackers can request enormous tables, deeply nested structures, high-resolution media, too many pages, pathological formulas, or archives that expand far beyond their compressed size.

Enforce limits at several layers:

• request-body bytes;
• nodes, pages, slides, sheets, rows, columns, and cells;
• text length per field and per artifact;
• image count, dimensions, decoded pixels, and total bytes;
• archive entries and expanded size;
• render wall time, memory, and output bytes;
• retries and concurrent jobs per identity.

Return a typed resource-limit error. Do not partially generate and present the result as complete unless the operation contract explicitly supports partial artifacts and reports what was omitted.

Treat the generated file as untrusted output

A generated Office package or PDF can contain relationships, hyperlinks, metadata, embedded files, scripts, forms, or hidden content. Validate the exact bytes before delivery.

Checks should include the package graph, external relationships, allowed part types, declared losses, suspicious metadata, unexpected active content, and semantic requirements. For PDFs, distinguish syntactic parsing, visual rendering, accessibility structure, forms, attachments, and JavaScript policy. For Office files, distinguish Open XML structure from real-application acceptance.

Sanitize only under an explicit transformation contract. Silent removal can change meaning. If the system strips macros or external links, return those changes as losses.

Separate candidate creation from release

Generation should produce a candidate. Release is a different authority.

A release gate can require:

1. the exact artifact hash;
2. structural and semantic validation results;
3. declared losses and policy outcome;
4. a preview generated from the same bytes;
5. reviewer identity or automated-policy authority;
6. an approved destination;
7. an immutable release record.

If the file is regenerated after review, the hash changes and the earlier approval becomes stale. If the destination changes, authorization should run again. A model should not be able to substitute a new file between preview and delivery.

Apply least privilege to every tool

An AI agent that can draft a deck does not automatically need access to customer storage, email, CRM records, or organization-wide templates. Give it narrowly scoped document operations and short-lived access to approved resources.

For MCP tools, the official tools specification says servers must validate inputs, implement access controls, rate-limit invocations, and sanitize outputs; clients should show sensitive calls, validate results, apply timeouts, and log usage. Treat tool annotations as untrusted unless the server itself is trusted.

The model can choose among allowed operations. The server owns authorization, quotas, idempotency, and destination policy.

Log decisions without leaking document content

Security telemetry should answer who requested what operation, which policy and implementation ran, which artifact hash resulted, which controls fired, and whether release occurred. It should not copy entire confidential prompts or document bodies into a broadly accessible log system.

Prefer identifiers, hashes, counts, stable error codes, policy versions, and redacted locators. Put sensitive debugging material in a separately controlled evidence store with retention and access rules.

NIST’s AI Risk Management Framework frames AI risk work across governance, mapping, measurement, and management. That lifecycle is useful here: define the document job and harms, measure technical and workflow controls, and manage the residual risk instead of treating one model filter as the security boundary.

Exercise the threat model before release

Turn the trust-boundary diagram into tests. Feed the workflow a document that contains text resembling system instructions, a URL to a disallowed host, an oversized embedded object, a relationship that escapes the package boundary, and content that asks the agent to reveal credentials. Each control should fail at its owning boundary with a stable code and without performing later network, storage, approval, or delivery actions.

Add positive controls that use ordinary customer content with punctuation, code snippets, URLs, and multilingual text. An over-broad filter that deletes legitimate source material is also a security and product defect. Structural separation should carry most of the defense: trusted instructions belong in typed configuration, while untrusted document content belongs in data fields that cannot become tool authority.

Test the generated artifact independently from the model response. Reopen its package, enumerate relationships, reject unexpected active content, enforce media and expansion limits, and compare the output with the declared mutation scope. Run preview generation in the same network and filesystem restrictions used in production. If a format requires a less isolated converter, expose that dependency as a distinct binding and policy decision.

Finally, rehearse deletion and incident response. Confirm that operators can identify the affected logical job from hashes and low-sensitivity metadata, revoke download access, prevent release, expire temporary storage, and preserve only the evidence required by policy. Verify that routine logs do not contain source paragraphs, generated document text, signed URLs, or secrets. A secure generation flow must remain explainable during failure without turning telemetry into another copy of the customer’s document.

Production checklist

• Model inputs and retrieved sources are explicitly marked as untrusted data.
• The model emits a strict intermediate representation, not executable code by default.
• Unknown fields and out-of-range values fail validation.
• External acquisition is allowlisted, isolated, and content-addressed.
• Rendering runs with bounded time, memory, network, and filesystem access.
• Outputs are checked for unexpected relationships, active content, and declared losses.
• Candidate, reviewed, approved, and released states are separate.
• Approval and delivery bind to the exact artifact hash.
• Tool authorization and rate limits are server-enforced.
• Logs preserve decisions and evidence without duplicating sensitive content.

Runstamp’s agent tools, processing records, review controls, and deployment guide show how these controls fit together. Check the current operation and deployment availability before designing the integration.