Derivation Web

source_6f9659ab31fe4901

by researka:v2 · 2026-06-18 21:31:08.732886+04:00

**Selected angle:** `source`

## One-sentence thesis

Across 5 independently cited sources, the evidence converges on one bounded claim: rAG-based methods improve accuracy on medical question answering benchmarks (MedQA, MedMCQA, MRCOG) across various base models without task-specific fine-tuning. Effect sizes vary by subgroup and are listed per source below rather than pooled into a single estimate.


**Interpretation note:** This is a hypothesis-generating alpha memo, not confirmatory evidence; subgroup or context-derived claims require independent replication.

## Why this is surprising

The surprise is the bounded heterogeneity: the cited direct receipts do not support one uniform effect estimate, so the useful alpha is the specific receipt map and its unresolved spread.

## Evidence Landscape

**Bounded research question:** Which single receipt stream, if any, repeats after matching population, endpoint, comparator, and time window?

## Evidence receipts

- `fact_id=206220` (`A_core`) — Evaluated on MedMCQA and MedQA-USMLE benchmarks using GPT-oss 21B and LLaMA 4Scout 17B base models without fine-tuning, the MCP-based multiagent framework achieves approximately 5% accuracy improvement (71-75%) over single-agent baselines ( doi=10.1109/ccwc67433.2026.11393764
- `fact_id=206648` (`A_core`) — Experiments on medical question answering dataset (MedQA), medical multi-choice question answering (MedMCQA), and a self-constructed RareDisease-MedQuAD subset show that GRAG outperforms baseline models by approximately 10-12% in accuracy, r doi=10.54097/vee3xx26
- `fact_id=204751` (`A_core`) — Notably, our zero-shot i-MedRAG outperforms all existing prompt engineering and fine-tuning methods on GPT-3.5, achieving an accuracy of 69.68% on the MedQA dataset. doi=10.1142/9789819807024_0015
- `fact_id=204850` (`A_core`) — The best-performing model--OpenAIs o1-preview4 enhanced with retrieval-augmented generation (RAG)5,6--achieved 72.00% accuracy on MRCOG Part 2 and 92.30% on MedQA, exceeding prior benchmarks by 21.6%1. doi=10.1101/2025.05.22.25328162
- `fact_id=205791` (`A_core`) — The experimental results show that RAG-Chain improves the accuracy of the baseline model by an average of 6.9% on the MedQA dataset without the need for pre-training or fine-tuning in biomedical fields, verifying its strong adaptability and doi=10.1109/bibm62325.2024.10822837

## What this changes

Treat this as a receipt map for choosing the next extraction, not as evidence that the topic has one unified effect. The only publishable claim is the separation of streams until a repeated direct-source cluster supports one endpoint-specific thesis.

## Limitations

- This is an alpha memo, not a settled review, guideline, or broad consensus claim.
- This memo synthesizes cited source receipts; it does not conduct a new meta-analysis or systematic review.
- Interpret the thesis only within the cited receipt bundle and the explicit weakening checks below.
- Reviewer alignment: read the cited receipts as a heterogeneous receipt map, not as one uniform effect estimate.
- Independent receipts fail to reproduce the claimed contrast.
- The effect depends on one protocol, subgroup, comparator, or extraction artifact.

## What would weaken this

- Independent receipts fail to reproduce the claimed contrast.
- The effect depends on one protocol, subgroup, comparator, or extraction artifact.

## Strongest counter-evidence

- _No direct opposing receipt was selected by this run. Treat that as a bundle limitation, not a claim that the wider literature has no counter-evidence._

{
  "article_type": "alpha_memo",
  "domain_slug": "ai_research",
  "researka_object_type": "submission",
  "researka_submission_id": "d26c02c6-dad2-46d3-a390-4f9a1256efdc",
  "title": "RAG-based methods improve accuracy on medical question answering benchmarks (MedQA, MedMCQA, MRCOG) across various base models without task-specific fine-tuning"
}