Contextual Chunk Retrieval — Controlled Experiment
October 9, 2026

Files

context-fixtures.json: 64 records, 16 families. Eight development families,
eight evaluation families. Four contexts per identical focus sentence,
two independently written specific queries per context, one generic query
per family. Contains the clean and seeded noisy variants actually used.

intent-fixtures.json: 32 records, eight additional evaluation families.
These change the intended operation on the same subject, rather than just
the meaning of a word. The weight chosen on the first development block
was carried into this independent block.

results.zip contains context-results.json and intent-results.json: all 15 methods and four
conditions, per-query rankings, tie-adjusted metrics, generic-query coverage,
development selection, and the original corpus hashes. The two corpora
were searched separately, with all records in the relevant block eligible.

summary.json: combined evaluation figures and family bootstrap intervals.
There are 128 specific evaluation queries and 16 generic evaluation queries,
across 16 evaluation families. Each condition reuses those queries with its
specified noise variants; the conditions are not independent query sets.

Method

Model: text-embedding-3-small. Dimensions: 1024. Retrieval: exact cosine.
Normalize each base embedding. For focus F, context C, and weight a, use
unit(a * unit(E(F)) + (1-a) * unit(E(C))). Context includes the focus text.
Whole-context uses a=0, sentence-only uses a=1. Symmetric blending uses the
same weight for both stored and query vectors. All methods use one model
and one dimension. Flavor labels, IDs and target labels were not embedded.

The generic query contains the shared focus only. Its relevant set is all
four contextual occurrences; coverage is measured at four returned records.
For ties, top-1 success is 1/tie_count when no candidate scores above the
target. Top-four coverage and reciprocal rank are also tie-adjusted.
Tolerance for equal scores: 1e-9.

The 80/20 blend was specified before the fixture experiment. Symmetric 95/5
was selected using development families only, via the harmonic mean of
specific-query top-1 accuracy and generic-query coverage averaged across
conditions. The blog reports the prespecified 80/20 result, with its baselines.

Noise adds five unrelated sentences while retaining the original text.
The fixture files include those exact sentences and their placement.

Limits

Synthetic corpus with manually chosen focus spans. This experiment does
not evaluate automatic segmentation, grouped prompt injection, production
latency, approximate search, graph traversal, or end-to-end answer quality.
Late chunking and LLM-generated contextual enrichment were not tested.

The results corpusHash hashes JSON.stringify(parsed_fixture), not the
original file bytes. Source hashes in summary.json use that representation.
The bootstrap resamples the 16 evaluation families with replacement,
10,000 draws, seed 101009; intervals describe variation in these fixtures.