NestWeaver Benchmarks: Why Your AI Coding Agent Needs a Knowledge Graph

AI coding agents are expensive. Not in the “enterprise software” sense — in the literal, per-token sense.

Cursor’s telemetry shows that input tokens account for 82-92% of non-cache token volume in coding workflows. The input-to-output ratio climbed 152% in early 2026. That means the vast majority of what you’re paying for isn’t the model reasoning about your code — it’s the model reading your code.

Most of those tokens go toward orientation. Figuring out where things are. Understanding how components connect. Tracing a call chain through three layers of abstraction. Not reasoning about the fix — just finding the right place to look.

The tools claiming to solve this problem make big promises. Fewer tokens. Faster queries. Better context. But what happens when you actually measure them?

We benchmarked NestWeaver against two of the most popular code intelligence tools — Graphify (71K GitHub stars) and GitNexus (43K stars) — across four open-source repositories spanning TypeScript, Rust, and Java. The results tell a clear story about what matters in code intelligence: it’s not just speed or token reduction. It’s the quality of what comes back.

The Context Problem

Developers spend 58-70% of their time on code comprehension, not writing code. This isn’t a new finding — it’s been consistently observed across studies spanning decades, from Minelli et al. at ICPC 2015 to Xia et al. in IEEE TSE 2018. At a loaded cost of $173K per developer, that’s $100K-$121K per year spent understanding existing code.

AI coding tools are supposed to help with this. But they have a retrieval problem.

The ContextBench study found that state-of-the-art agents achieve only 0.35 F1 at line-level context retrieval. They find the right files, but drown in noise at the block and line level. Worse, SWE Context Bench showed that bad retrieval is actually worse than no retrieval — autonomous retrieval dropped resolution rates below the no-context baseline.

The Qodo State of AI Code Quality report surveyed 609 developers and found that 65% say their AI assistant “misses relevant context.” And the METR study demonstrated that AI tools actually make experienced developers 19% slower on familiar codebases — because without good context, the tool becomes a liability rather than an asset.

This is where graph-based retrieval enters the picture. Regolo’s benchmark showed that graph-based retrieval uses 11x fewer tokens with 4x better accuracy compared to standard RAG. CodeRAG-Bench demonstrated that perfect retrieval triples a weak model’s accuracy — from 31.7% to 94.5% on HumanEval.

The retrieval layer is the bottleneck. Not the model. Not the prompt engineering. The thing that decides what goes into the prompt in the first place.

What We Built

NestWeaver builds a structural knowledge graph over codebases. Not keyword search. Not embedding similarity. A real graph with symbols, edges, and cross-file relationships.

The pipeline works like this: Tree-sitter parses source files across 32 languages, extracting functions, classes, types, and their relationships. A cross-file resolver traces imports, call chains, and type usage across module boundaries. Personalized PageRank ranks every symbol by structural importance — the same algorithm Google used to rank web pages, adapted for code topology. A daemon keeps the graph in memory with Metal GPU acceleration on macOS.

For AI agents, NestWeaver exposes 40 MCP tools. Ask for context around a symbol and you get the ranked subgraph — callers, callees, dependencies, types — all within a token budget. Not a file dump. A focused answer.

The key insight is that code has structure that text doesn’t. A function calls other functions. A class implements an interface. A module exports symbols that other modules import. These relationships are explicit in the source — they don’t need to be inferred by an embedding model or guessed by a keyword search. They can be parsed, resolved, and ranked.

That’s what NestWeaver does. It reads the structure that’s already there and makes it queryable.

The Benchmark

We tested against Graphify and GitNexus — the two most popular open-source code intelligence tools with graph-based approaches.

Four repositories, chosen for language diversity and scale:

All tools ran on Apple M3 Pro (36 GB). NestWeaver in daemon mode — its production configuration. Equal warm-up queries for all tools. Same 10 queries per repo: 5 keyword searches and 5 structural context queries using verified symbol names.

Metrics: p50 query latency, symbols extracted, edges resolved, result relevance (manual inspection of returned nodes against ground truth).

Every benchmark is in the repo. Run it yourself:

git clone https://github.com/Kehl-io/nestweaver
bash benchmarks/run.sh

Query Speed

The headline numbers: NestWeaver answers queries 17x faster than Graphify on Deno and 34x faster on Next.js.

The interesting pattern: NestWeaver gets faster on bigger repos. 157ms on Tailwind, 82ms on Deno, 71ms on Next.js. That’s counterintuitive until you understand the architecture.

The daemon keeps the graph in memory. Larger graphs mean better cache locality for structurally hot paths. PageRank has already identified which symbols matter, so the query walks a pre-ranked graph rather than doing BFS over the raw structure. The work happens at index time, not query time.

Competitors go the other direction. Graphify degrades from 185ms on Tailwind to 2,440ms on Next.js — a 13x slowdown as the repo grows. GitNexus follows a similar curve: 700ms to 1,116ms. Both tools do more work at query time as the graph grows, which means latency scales with repo size.

On Elasticsearch (44K files, 511K symbols), Graphify could not complete the benchmark. It crashed during query execution. NestWeaver handled it at 617ms p50 — slower than the smaller repos because of the sheer edge count (14.8 million), but still well within interactive latency.

For an AI coding agent making 10-50 queries per task, the difference between 71ms and 2,440ms per query is the difference between a tool that feels instant and one that dominates the task’s wall-clock time.

Speed Without Accuracy Is Meaningless

Fast queries that return garbage aren’t useful. Here’s what each tool actually returns when you ask for context around a symbol.

NestWeaver returns approximately 30 connected symbols per query, ranked by Personalized PageRank. Each result includes the function signature, file path, line number, and its structural relationship to the query seed. Ask for SearchService in Elasticsearch and you get the class definition, its constructor, TransportExplainAction (a caller), SearchTransportService (a dependency), and the full call chain into SearchQueryThenFetchAsyncAction — 482 connected nodes across 261 classes.

That’s not a list of files. It’s a subgraph. The agent can read it and understand how SearchService fits into the system without opening a single file.

Graphify returns 29-65 nodes per query, but they’re file-level entries, not individual symbols. And 9-15% of those nodes are garbage stubs — vitest config files, node_path module references, framework dependencies with no actual source code. There’s no relevance ranking; results are ordered by BFS distance, which means structurally close but semantically irrelevant files rank equally with critical callees.

When you ask Graphify about SearchService, you get the file that contains it, plus adjacent files in the directory tree, plus files that happen to share import paths. Some of those are useful. Many aren’t. The agent has to figure out which is which — which means more tokens spent on triage.

GitNexus finds symbol definitions but returns 0 callers and 0 callees. It locates where SearchService is defined but can’t tell you what calls it, what it depends on, or how it fits into the request-handling pipeline. It’s a glorified grep with a graph data structure behind it that doesn’t actually resolve cross-file relationships.

What the competitors claim

Graphify advertises “71.5x fewer tokens per query.” That number comes from a built-in benchmark that uses 5 hardcoded questions (“how does authentication work”, “what is the main entry point”) and compares against sending the entire repository as context. No real tool does that. Independent reviews found the realistic range is 5-15x on pure code repositories — still useful, but a long way from 71x. Graphify publishes zero accuracy or quality metrics.

GitNexus claims “88% fewer tool calls, 74% token savings.” That comes from 3 queries by 1 person on 1 codebase — the author’s own personal project. The only independent replication found 68% fewer tool calls and 43% token savings. GitNexus was actually slower than the baseline in that test. Their repo contains a SWE-bench evaluation harness, but the results have never been published.

NestWeaver’s benchmarks are in the repo. Every number is reproducible. MIT licensed. We’d rather show our work than claim 71x improvements on a cherry-picked benchmark.

Graph Depth

The reason NestWeaver returns better results: it extracts a deeper graph.

On Deno, NestWeaver extracts 142,753 symbols and 280,491 edges. Graphify extracts 75,931 nodes and 176,970 edges — roughly half. On Elasticsearch, NestWeaver builds a graph with 511,058 symbols and 14.8 million edges.

That’s not a symbol index. It’s a knowledge graph.

More edges mean more cross-file relationships resolved. More resolved relationships mean PageRank has richer signal for ranking. Richer ranking means queries return more relevant results. Each layer compounds: a graph that resolves 2x more edges doesn’t return 2x better results — it returns qualitatively different results, because it can trace paths that a shallower graph simply can’t see.

The difference shows up most clearly on large codebases. On Tailwind (542 files), all three tools produce roughly comparable graphs. On Elasticsearch (44K files), NestWeaver’s graph has 6.8x more edges than Graphify’s. At scale, the gap widens because cross-file resolution is a superlinear problem — the number of potential relationships grows faster than the number of files.

Incremental re-indexing

A deep graph is only useful if you can keep it current. After a single file change, NestWeaver re-indexes without rebuilding the entire graph.

Competitors require a full re-index on every change. On Next.js, that’s 111 seconds for Graphify or 176 seconds for GitNexus — every time a file changes. NestWeaver does it in 4 seconds.

For an AI agent iterating on a fix — edit, re-query, edit again — the difference between 4 seconds and 111 seconds per cycle determines whether the graph stays useful or becomes stale.

Why This Matters

The AI coding tools market is valued at $9-11 billion in 2026, growing at 53% CAGR — the fastest of any generative AI modality. GitHub moved all plans to usage-based billing in June 2026, generating 400+ comments and 900 downvotes. Token costs are structural, not temporary.

Research shows agentic coding tasks consume 1,000x more tokens than code chat. The retrieval layer — what goes into the prompt — is the dominant cost lever. Not model selection. Not prompt compression. What you choose to include.

Graph-based retrieval addresses this directly. Not by compressing tokens, but by selecting the right ones. CodeRAG-Bench showed that giving GPT-4o gold-retrieved documents improves SWE-bench resolution from 2.3% to 30.7% — a 13x improvement from better retrieval alone. Code-Craft demonstrated that graph-based hierarchical retrieval improves Pass@1 by 82% on large codebases, with the advantage growing as codebases get bigger.

The comprehension tax — $100K+ per developer per year at 58-70% of working time — is the single biggest leverage point in developer tooling. A tool that reduces comprehension time by 10% saves more than eliminating all code-writing time entirely. Peter Hallam from the Microsoft C# compiler team made this observation years ago, and it’s only become more true as codebases grow.

The tools that win this market won’t be the ones with the best model access — everyone has that now. They’ll be the ones with the best retrieval. The ones that can answer “what calls this function and why does it matter” in 71 milliseconds instead of 2.4 seconds, and return 30 relevant symbols instead of 65 file-level stubs with 15% noise.

Get Started

NestWeaver is open source, MIT licensed, and compiles to a single Rust binary.

npm install -g @kehl-io/nestweaver
nestweaver index --repo .
nestweaver setup  # auto-configure Claude Code, Cursor, Copilot, and 13 more

Run the benchmarks yourself:

git clone https://github.com/Kehl-io/nestweaver
cd nestweaver
bash benchmarks/run.sh

If you’re building AI coding tools and your retrieval layer is grep and file reads, you’re leaving accuracy and money on the table. NestWeaver gives your agents a knowledge graph to query instead of a filesystem to read.

GitHub | Documentation | Benchmarks