A NUL byte no longer costs a file, and --explain counts the row budget in rows.
A test that feeds a raw NUL byte to a parser, written into a list of keys as the byte itself, cost its Dart file everything but one function name: the grammar’s string scanner stops on a NUL, and recovering from the unclosed string it gave up the file from its first line. The same list cost a Python file the definitions after it, with no warning at all. vexp now reads a NUL as a space when it parses, in every language it parses with a grammar (the script blocks of Vue, Svelte, Astro and Razor files aside), and the warning for a partly parsed file names the line where the reading broke, not only where the unread region starts. The rest of the release comes from measuring how a typed question’s words are counted on large indexes: --explain now prints, in rows, what the row budget counted and what it left out, and three ranking changes built and measured this week ship behind switches, off, because on questions they were not tuned on they did not hold up. How vexp ranks is unchanged by default. And a daemon an editor window started no longer outlives that window, with the idle stop now named where you look for it.
A NUL byte in a string no longer costs the file around it. Tree-sitter’s lexers do not take a NUL as a character, so a string literal holding one never closes and the grammar recovers. In a short statement the recovery stays inside it. In a Dart list of about ten entries or more it read the entries as list patterns and gave up the file from line 1, keeping only a function’s signature, with no body, no test names and no calls. In Python the same list cost the two definitions that followed it, without a warning, since the partly-parsed check reads Dart only. The parser now reads every NUL as a space, one byte for one (the script blocks of Vue, Svelte, Astro and Razor files are not covered yet), so every position still points into the file as written, and names, signatures and bodies keep the NUL. On 342 variants of the reported shape, the Dart files lost whole went from 208 to none that are valid Dart.
A partly parsed file names the line where the reading broke. A partly parsed file was reported by where its unread region starts, and the region is what the grammar gave up, which can start far above the cause: a NUL two thirds down a file was reported as lines from line 1. The report now adds the line where the reading broke, the first token the grammar could not take, when it is not the region’s first line, as in “29 line(s) from line 1, the reading broke at line 14”. It is said the same way in the index warning, the summary of vexp index, vexp index --status, the daemon’s status and vexp doctor, and coverage.json carries it as broke_line. A Dart file an older version listed as partly parsed is read again on the daemon’s first start after the update, with no rebuild.
--explain says what the row budget counted, in rows. A typed question’s words are counted by the coverage pass rarest first, while the rows of their posting lists fit a budget of 8,000; a word past it is not counted. --explain listed those words as over the row budget, without the rows that decided it, and gave the same reason for words the 62-group mask cut. A new line gives each word’s rows, the rows the counted words take, the first word left out and the sum it would have made, and apart from those the words cut by the 62-group limit of the coverage mask: “word rows (budget 8000): counted 7912: …; first left out: … (would make 9324)”. It shows how many rows each left-out word carries, so you can see whether a question’s subject words were dropped for being common in the index or because rarer words had used the budget first. The JSON form carries the same numbers. VEXP_EXPLAIN_ROWS=0 restores the previous output.
The same question names the same test on every run. When a test was found through the functions it calls and two callees tied on call sites, the label “test calling X” and the window shown around the first call site were picked in a different order on every run. Ties are now broken the same way every time, so two runs of one question give the same capsule.
Three ranking changes ship off, behind switches. Measured this week on blind questions over four large open-source repositories and two indexes shaped like a field report: a file may hold at most two of the thirty places the text search keeps (VEXP_CUT=1), two words typed apart reach the one word the code writes (VEXP_VOCAB_JOIN=1, “read only” reaching readonly), and a word over the row budget is counted again over the candidate files when at most 30% of the index’s files carry it (VEXP_POOL_OVER_BUDGET=1). Together they raised the blind development questions from 24 to 27 of 46. On 240 typed questions they were not tuned on, the file share and the join together lost 7 top-3 answers and gained 1; the over-budget reads raised the query time, by 95% at the median on questions full of common words, and on 59 untuned questions gained one top-3 answer and lost one. So all three ship off, and the default ranking is that of 3.3.6. VEXP_POOL_ROW_BUDGET shrinks the row budget to see a question as a larger index would.
A window’s daemon goes with its window, and the idle stop is in the help. A daemon the VS Code extension starts for a window took itself for that window’s for as long as it ran. When the window went without stopping it, an editor that crashed or was killed, the daemon ran on, the idle stop passed it by, and its first log line to the pipe nobody read any more brought it down, leaving its socket and its registry row behind. It now stops within seconds of its window, the way a closing window stops it, and a log line that cannot be written is dropped. vexp daemons says “owned by VS Code” only while that window is there, and “VS Code’s engine, no window” for a daemon an agent started with the extension’s engine, as a ~/.claude.json entry naming the extension’s server does; that one stops after idle_timeout_minutes like any other. The setting is now in vexp --help, vexp daemons --help, vexp stop --help and the README, and vexp doctor says whether it is on and, when a project’s daemon stopped for idleness, says that instead of a warning.
Upgrade note: nothing to rebuild. No index is rebuilt and no setup needs running again. Dart files an older version could not read whole are read again on the daemon’s first start after the update. A Python or other file already indexed with a NUL in it is read again when it changes, or with vexp index. Update the VS Code extension together with the CLI and reload each open editor window once.