Per-arm results
| Arm | Accuracy | 95% CI | Macro F1 | Avg latency | Adds |
|---|---|---|---|---|---|
| plain | 30.0% | 24.1–36.7% | 29.7% | 6.4s | baseline chunk RAG |
| plain_rr | 37.0% | 30.6–43.9% | 35.2% | 6.6s | + cross-encoder reranker |
| graphwinner | 59.5% | 52.6–66.1% | 50.5% | 7.5s | + parent-paper expansion |
| graph_concepts | 57.5% | 50.6–64.1% | 50.0% | 40.8s | + MeSH concept hop |
Significance (paired McNemar)
“Significant?” uses the raw (unadjusted) McNemar p-value at α=0.05; this summary.json predates Holm adjustment.
| Contrast | Δacc | Gains / losses | p-value | Significant? |
|---|---|---|---|---|
| plain → plain_rr (reranker) | +7.0pp | 35 / 21 | 0.0814 | no |
| plain_rr → graph (parent expansion) | +22.5pp | 71 / 26 | <0.0001 | yes |
| graph → graph_concepts (concept hop) | −2.0pp | 26 / 30 | 0.6889 | no |
Honest summary
The graph arm's win is one specific mechanism, not "a knowledge graph" in the abstract: parent-document expansion changes accuracy by +22.5pp over the reranked baseline (paired McNemar p<0.0001). The further MeSH concept-hop arm does not clear significance against the plain graph arm (p=0.6889) and costs ~5.4x the latency, so it is not the recommended arm to ship. See RESULTS.md for the full write-up.