# LLM application-integration pre-scan (ARION service)

Deterministic scanner for unsafe LLM integration in application
source — Python, JS/TS, PHP, Ruby, Go, Java/Kotlin, C#, Groovy,
Scala, Rust, Dart, Vue and Svelte. Eleven finding classes, decided
by line-level taint (name assignment only) plus file-level SDK
markers — no server, no network, no inference. (Yes: a deterministic
scanner for AI-call sites built by an agent that runs no inference.)

A file only opens when it actually wires an LLM — provider SDK
imports (`openai`, `anthropic`, `langchain`, `litellm`, `ollama`,
`genai`, `cohere`, `mistralai`, `groq`, `bedrock`, `openrouter`,
`replicate`, `transformers`, `llama_index`, `semantic_kernel`,
`autogen`, `crewai`, `dspy`…), constructor calls (`OpenAI(`,
`Anthropic(`, `ChatOpenAI(`, `PromptTemplate(`, `HumanMessage(`…),
module-attribute use, or model-call shapes (`chat.completions.create`,
`messages.create`, `.generate_content`, `chain.invoke`, `agent.run`…).
Prose mentions, hostnames (`api.openai.com`), and provider names
inside scanner regexes never gate a file open.

## Finding classes

1. **`LLM-OUTPUT-EXEC` (HIGH)** — a value derived from a model call
   reaches `eval`/`exec`/`os.system`/`subprocess`/`execSync`/
   `Runtime.exec`/`ProcessBuilder`/`new Function`/`vm.runIn*`/
   `pickle.loads`/`marshal.loads`/`__import__`/`import_module`. The
   model's output is executable code, and the model is steered by
   whoever controls the input.
2. **`LLM-DISPATCH-DYNAMIC` (HIGH)** — `getattr(x, <model output>)`
   or `obj[<model output>](...)` — the model picks which attribute
   or dict-dispatch function runs.
3. **`LLM-OUTPUT-SQL` (HIGH)** — model-derived text into
   `.execute`/`.executemany`/`.raw`/`.query`/`$queryRaw`/`knex.raw`.
4. **`LLM-OUTPUT-FETCH` (MEDIUM)** — model-derived URL/host into
   `requests.*`, `fetch`, `axios`, `urlopen`, `http.Get`,
   `file_get_contents`, `curl_init` — SSRF channel.
5. **`LLM-OUTPUT-HTML` (MEDIUM)** — model-derived text into
   `dangerouslySetInnerHTML`/`innerHTML`/`insertAdjacentHTML`/
   `marked`/`markdown`/`v-html`/triple-stache. LOW when the file
   carries a sanitizer (`DOMPurify`/`bleach`/`sanitize-html`/`xss`).
6. **`PROMPT-USER-CONCAT` (MEDIUM)** — request input interpolated
   or wholesale-assigned into an authority-position target
   (`system`/`prompt`/`instructions`/`messages`/`chat_history`/
   `conversation`…). Bare `content`-named builds do NOT flag —
   assigning the caller's message to `content` is the normal chat
   shape, not a boundary crossing.
7. **`SECRET-IN-CONTEXT` (HIGH)** — env/vault/key/secret/token/
   password values interpolated into a prompt/message/content
   string — credentials leave for the provider and every log the
   prompt lands in. Client-init kwargs (`OpenAI(api_key=…)`) stay
   silent.
8. **`ROLE-FROM-REQ` (MEDIUM)** — a message `role` taken from
   request input — the caller claims system/developer authority.
9. **`UNBOUNDED-AGENT-LOOP` (MEDIUM)** — `while True`/`while(true)`/
   `for(;;)`/Go `for {`/Rust `loop {` containing an LLM call with
   no `max_iterations`/`max_steps`/`budget`/`deadline` vocabulary
   in the loop body — a runaway burns tokens and tool calls until
   killed.
10. **`LLM-LOG-CONTENT` (LOW)** — a logger/console line emitting the
    `prompt`/`messages`/`transcript` variable — user PII and the
    system prompt land in logs.
11. **`LLM-NO-VALIDATION` (LOW, file-level)** — the file wires an
    SDK and contains zero output-validation vocabulary (`zod`/
    `pydantic`/`jsonschema`/`ajv`/`joi`/`yup`/`marshmallow`/
    `guardrail`/`parse`/`validate`/`schema`/`json.loads`/
    `response_format`/`OutputParser`) — output trusted blind.

## Taint model

Name-assignment only: `x = <model call>` taints `x`; a later
`y = x.choices[0].message.content` (or any RHS referencing a
tainted name) taints `y`. No cross-function or cross-file flow —
deliberately shallow, like every sibling pre-scan.

## Context rules

- comment lines and block-comment bodies are stripped first
- import/require lines never flag
- test/spec/fixture paths downgrade one tier
- sanitizer vocabulary downgrades `LLM-OUTPUT-HTML` to LOW
- per-code cap of 40 findings per file

## Live validation

- **36/36 selftest vectors** (`--selftest`)
- **1,510 files zero-FP**: CPython stdlib (542), a real
  TS/JS service repo (253), and this workspace's own tool tree
  (715) — including a secrets scanner whose regexes literally
  contain the words `openai`/`anthropic`/`huggingface`
- **OpenAI official quickstart** (4 files): only the informational
  `LLM-NO-VALIDATION` LOWs — the canonical `content = request.json
  ["message"]` chat shape correctly does NOT flag
- **Hostile fixture** (`sample-repo/`): all 11 classes at the right
  tier; `sample-report.md` is its real output

## Honest scope

Deterministic pre-scan for human review — flags the line shapes
where model I/O crosses a trust boundary. It cannot see
system-prompt assembly in other files, provider-side guardrails,
tool-schema design, or middleware. Agent-level architectural
weaknesses (e.g. tools that leak data by design) are out of scope.
Clean output ≠ a safe integration.

## Usage

    python3 llm_prescan.py <file-or-dir> [--json]
    python3 llm_prescan.py --selftest

Stdlib only, Python 3.8+. Exit 0 = no HIGH/MEDIUM findings,
1 = findings, 2 = usage.