How Helix works

Helix is a static analysis pipeline that turns source code into a versioned technical graph, then layers product features and AI enrichments on top. Query APIs read the latest scan from S3 (Neptune is optional for graph traversals).

End-to-end flow

  features/*.yaml          repos (git checkout)
         │                        │
         │                        ▼
         │              ┌─────────────────┐
         │              │  AST ingest     │  per repo: facts, CALLS, structural edges
         │              │  (Fargate/local)│
         │              └────────┬────────┘
         │                       │ merge-input.json per repo
         ▼                       ▼
  ┌──────────────────────────────────────────┐
  │  Merge barrier (single scanId)           │
  │  1. Dedupe + union repo graphs           │
  │  2. FE↔BE bridge (CALLS_REMOTE)          │
  │  3. Feature resolver (YAML + synthetic)  │
  │  4. technical-graph.json                 │
  └──────────────────┬───────────────────────┘
                     │
         ┌───────────┼───────────┐
         ▼           ▼           ▼
   S3 artifacts   Neptune     AI stages (optional)
   (source of     upsert      endpoint Q&A, feature
    truth)                     insights, summaries
         │
         ▼
   Query Lambda / local API → UI & integrators

On AWS, Step Functions orchestrates: InitScan → Map(AST per repo) → MergeGraph → Map(AI per repo) → MarkReady.

Phase 1 — Per-repo AST ingest

For each repository root, the ingest CLI/worker:

1. Walks source files (respecting excludes).
2. Parses TypeScript/JavaScript (and related stacks) into technical facts: symbols, imports, route registrations, React components, HTTP client calls, etc.
3. Emits intra-repo edges — CALLS, HANDLED_BY, IMPORTS, USES_COMPONENT, and similar — with stable repo-scoped node ids (repo:<key>|…).
4. Writes shard artifacts under repos/{repoKey}/ (e.g. files.jsonl, merge-input.json).

Algorithm shape: file → AST → symbol table + call graph slice. No whole-program optimization across repos yet at this stage.

Phase 2 — Merge barrier

When all repo shards for a scanId are ready, merge runs once:

Step	What it does
Union	Combine facts and edges from every repo; dedupe by stable ids
FE↔BE bridge	Match ApiClientCall nodes to ApiEndpoint nodes across repos via service name + HTTP path (resolveFeBeBridges) → CALLS_REMOTE edges
Remote chains	Extend reachability for multi-hop FE→BE paths (resolveRemoteCallChains)
Feature resolver	Bind features/*.yaml definitions to graph anchors; emit Feature nodes and USES edges; add synthetic bridge features for matched FE↔BE pairs without YAML
Graph document	Serialize reports/technical-graph.json with schema version, repo list, nodes, edges

Output also includes feature-resolution.json (matched / unmatched anchors) and fe-be-bridge.jsonl (audit trail for cross-repo links).

Phase 3 — AI enrichment (optional, per repo)

After the merged graph exists, the helix-agent worker can run batch LLM jobs per repository:

• Node / edge summaries merged back into graph JSON or sidecar ai/ files
• Endpoint Q&A, potential issues, feature insights, display titles

These are lazy-loaded at query time (one S3 object per request), not bulk-synced into Lambda memory.

Query model

Consumer need	Mechanism
Latest scan id	DynamoDB WORKSPACE#… / LATEST
Full graph	/query/technical-graph
Repo slice	/query/repo-graph?repoKey= — filter in Lambda
Feature neighbourhood	/query/feature?featureId= — 1-hop subgraph
AI bundle	/query/feature-insights, /query/endpoint-qa, … — S3 lazy read

Subgraph filters (repo, endpoint, UI surface) in the UI reuse the same graph JSON with client-side traversal helpers (filterGraphByEndpoints, sequence ordering for Mermaid).

Extending Helix

Helix is designed to grow without breaking prior scans:

Add a repository

1. Register the repo in scan config / Repositories admin.
2. Re-run ingest — merge picks up the new merge-input.json.
3. FE↔BE bridges appear when both sides exist in the same scan.

Add product features

1. Add or edit YAML under features/ (flow key, anchors, optional implementation hints).
2. Re-run merge (or full scan) — feature resolver attaches Feature nodes.
3. Run helix-agent feature-insights for new AI bundles.

Add graph semantics

1. Extend ingest parsers to emit new node labels or edge types (follow @helix/shared graph schema).
2. Update merge if cross-repo rules are needed.
3. Optionally extend Neptune upsert mappers in backend/src/neptune/.
4. Add query routes or UI filters that understand the new labels.

Add query / integration surfaces

1. Implement handler in backend/src/routes.ts.
2. Add Zod schema in shared/src/queries.ts.
3. Document in Query REST API.
4. Publish SSM params if other AWS apps need discovery (Other AWS Apps Integration).

Incremental scans (future / partial today)

Manifests support per-file fingerprints: on a new commit, re-parse changed files only, then re-merge and invalidate edges touching changed symbols. Full rescans remain the safe default after parser version bumps.

Schema evolution

• manifestVersion on scan manifests
• schemaVersion on technical-graph.json
• New JSON keys are append-only in artifact contracts (Ingest artifacts)

Related

• Ingest artifacts — S3 key layout
• Glossary — Feature vs Flow, node labels
• Query REST API — HTTP contract