Run 19 vs Run 16

The two runs scored on the same queries. The cards show each run's average across all queries; the table below is one row per query. Each Δ (delta)column is run B's score minus run A's — green means B did better on that query, red means A did. Click a column header to re-sort; click a query to see both ranked lists.

What do these scores mean?

Each backend is scored against NFCorpus — a benchmark of 323 medical search queries, where humans have judged which documents are actually relevant to each query. Every quality score runs 0–1 and higher is better; for latency, lower is better.

nDCG@10: Overall quality of the top-10 ranking — it rewards putting more-relevant documents nearer the top. This is the headline metric.
precision@10 (P@k): Of the 10 results shown, the fraction that are relevant.
recall@10: Of all the relevant documents that exist for a query, the fraction that made it into the top 10.
MRR: How high the first relevant result lands, on average (1.0 = always at rank 1).
p50 / p95 ms: Response time: the median (p50) and the slow-tail 95th percentile (p95), in milliseconds.

A: run 19 · hybrid

nDCG 0.3557 · P 0.2613 · recall 0.1719 · MRR 0.5501

B: run 16 · vector

nDCG 0.3428 · P 0.2554 · recall 0.1618 · MRR 0.5272

query	Δndcg	Δprecision	Δrecall	Δmrr
Childhood Constipation and Cow’s Milk	+0.392	+0.400	+0.190	0.000
endocrine disruptors	+0.150	+0.200	+0.020	0.000
prenatal vitamins	+0.147	0.000	0.000	+0.857
Any update on the scary in vitro avocado data?	+0.140	0.000	0.000	+0.500
serotonin	+0.137	+0.200	+0.044	0.000
Acne & Cancer Connection	+0.135	0.000	0.000	+0.500
smoking	+0.133	+0.200	+0.005	0.000
What is Actually in Chicken Nuggets?	+0.121	+0.100	+0.018	0.000
Are Multivitamins Good For You?	+0.114	0.000	0.000	+0.500
Do Cholesterol Statin Drugs Cause Breast Cancer?	+0.111	+0.300	+0.125	0.000
Optimal Phytosterol Dose	+0.102	+0.100	+0.067	0.000
Who Should be Careful About Curcumin?	+0.095	+0.100	+0.017	+0.167
viral infections	+0.095	+0.100	+0.010	0.000
folic acid	+0.092	+0.100	+0.083	0.000
low-carb diets	+0.081	0.000	0.000	+0.500
Is Milk and Mucus a Myth?	+0.075	+0.100	+0.023	0.000
Optimal Phytosterol Dose and Source	+0.073	+0.100	+0.071	0.000
Chicken Salmonella Thanks to Meat Industry Lawsuit	+0.073	0.000	0.000	0.000
Olive Oil and Artery Function	+0.071	0.000	0.000	+0.167
Japan	+0.070	+0.100	+0.005	0.000
saturated fat	+0.066	+0.100	+0.003	0.000
Meat & Multiple Myeloma	+0.064	0.000	0.000	+0.167
Chronic Headaches and Pork Tapeworms	+0.064	+0.100	+0.023	0.000
cinnamon	+0.064	+0.100	+0.029	0.000
Atkins diet	+0.060	0.000	0.000	0.000
Stopping Heart Disease in Childhood	+0.060	0.000	0.000	+0.500
EPIC Study	+0.054	0.000	0.000	0.000
African-American	+0.054	+0.100	+0.029	0.000
Are Dental X-Rays Safe?	+0.054	0.000	0.000	0.000
Is Milk Good for Our Bones?	+0.039	+0.100	+0.014	0.000
heme iron	+0.038	0.000	0.000	0.000
How Citrus Might Help Keep Your Hands Warm	+0.038	+0.100	+0.020	0.000
Do Fruit & Nut Bars Cause Weight Gain?	+0.035	+0.100	+0.045	0.000
rhabdomyolysis	+0.032	0.000	0.000	0.000
Vitamin D: Shedding some light on the new recommendations	+0.032	0.000	0.000	0.000
cooking methods	+0.029	0.000	0.000	0.000
Breast Cancer & Alcohol: How Much is Safe?	+0.027	0.000	0.000	0.000
Plant-Based Diets for Psoriasis	+0.025	0.000	0.000	+0.133
American Dental Association	+0.021	0.000	0.000	+0.083
Relieving Yourself of Excess Estrogen	+0.021	0.000	0.000	+0.083
Turmeric Curcumin and Osteoarthritis	+0.020	0.000	0.000	0.000
Keeping Your Hands Warm With Citrus	+0.015	0.000	0.000	+0.083
Fresh fruit versus frozen--which is better?	+0.012	0.000	0.000	0.000
Is Caramel Color Carcinogenic?	+0.010	0.000	0.000	0.000
seafood	+0.010	0.000	0.000	0.000
Should We Avoid Titanium Dioxide?	+0.010	0.000	0.000	0.000
worms	+0.009	0.000	0.000	+0.024
Preventing Brain Loss with B Vitamins?	+0.009	0.000	0.000	0.000
Preventing Ulcerative Colitis with Diet	+0.007	0.000	0.000	0.000
Is vitamin D3 (cholecalciferol) preferable to D2 (ergocalciferol)?	+0.004	0.000	0.000	0.000
caloric restriction	+0.003	0.000	0.000	0.000
trans fats	+0.003	0.000	0.000	0.000
Stool Size and Breast Cancer Risk	+0.003	0.000	0.000	0.000
flax oil	+0.003	0.000	0.000	+0.011
Plant vs. Cow Calcium	+0.002	0.000	0.000	0.000
Coffee and Artery Function	+0.000	0.000	0.000	0.000
leeks	0.000	0.000	0.000	0.000
How to Reduce Exposure to Alkylphenols Through Your Diet	0.000	0.000	0.000	0.000
What’s Driving America’s Obesity Problem?	0.000	0.000	0.000	0.000
Exploiting Autophagy to Live Longer	0.000	0.000	0.000	0.000
What Do Meat Purge and Cola Have in Common?	0.000	0.000	0.000	0.000
Chronic Headaches and Pork Parasites	0.000	0.000	0.000	0.000
Food Dyes and ADHD	0.000	0.000	0.000	0.000
Starving Tumors of Their Blood Supply	0.000	0.000	0.000	0.000
How Should I Take Probiotics?	0.000	0.000	0.000	0.000
Diet and Cellulite	0.000	0.000	0.000	0.000
Best Treatment for Constipation	0.000	0.000	0.000	0.000
Increasing Muscle Strength with Fenugreek	0.000	0.000	0.000	0.000
How Chemically Contaminated Are We?	0.000	0.000	0.000	0.000
Treating an Enlarged Prostate With Diet	0.000	0.000	0.000	0.000
Mercury Testing Recommended Before Pregnancy	0.000	0.000	0.000	0.000
Breast Cancer and Diet	0.000	0.000	0.000	0.000
Can antioxidant-rich spices counteract the effects of a high-fat meal?	0.000	0.000	0.000	0.000
Didn't another study show carnitine was good for the heart?	0.000	0.000	0.000	0.000
What do you think of Dr. Jenkins' take on paleolithic diets?	0.000	0.000	0.000	0.000
What about pepper plus turmeric in V8 juice?	0.000	0.000	0.000	0.000
Is annatto food coloring safe?	0.000	0.000	0.000	0.000
Are krill oil supplements better than fish oil capsules?	0.000	0.000	0.000	0.000
How can you believe in any scientific study?	0.000	0.000	0.000	0.000
accidents	0.000	0.000	0.000	0.000
adenovirus 36	0.000	0.000	0.000	0.000
airport scanners	0.000	0.000	0.000	0.000
Alli	0.000	0.000	0.000	0.000
amnesia	0.000	0.000	0.000	0.000
aneurysm	0.000	0.000	0.000	0.000
antinutrients	0.000	0.000	0.000	0.000
apnea	0.000	0.000	0.000	0.000
Arkansas	0.000	0.000	0.000	0.000
ascorbic acid	0.000	0.000	0.000	0.000
avocados	0.000	0.000	0.000	0.000
bagels	0.000	0.000	0.000	0.000
beans	0.000	0.000	0.000	0.000
benzene	0.000	0.000	0.000	0.000
betel nuts	0.000	0.000	0.000	0.000
bioavailability	0.000	0.000	0.000	0.000
blood clots	0.000	0.000	0.000	0.000
BMAA	0.000	0.000	0.000	0.000
BRCA genes	0.000	0.000	0.000	0.000
breast pain	0.000	0.000	0.000	0.000
Bush administration	0.000	0.000	0.000	0.000
canker sores	0.000	0.000	0.000	0.000
chanterelle mushrooms	0.000	0.000	0.000	0.000
Chernobyl	0.000	0.000	0.000	0.000
cocaine	0.000	0.000	0.000	0.000
coma	0.000	0.000	0.000	0.000
cortisol	0.000	0.000	0.000	0.000
crib death	0.000	0.000	0.000	0.000
cumin	0.000	0.000	0.000	0.000
Czechoslovakia	0.000	0.000	0.000	0.000
deafness	0.000	0.000	0.000	0.000
DHA	0.000	0.000	0.000	0.000
dietary scoring	0.000	0.000	0.000	0.000
domoic acid	0.000	0.000	0.000	0.000
Dr. Dean Ornish	0.000	0.000	0.000	0.000
Dr. Walter Willett	0.000	0.000	0.000	0.000
ECMO	0.000	0.000	0.000	0.000
eggnog	0.000	0.000	0.000	0.000
energy drinks	0.000	0.000	0.000	0.000
ergothioneine	0.000	0.000	0.000	0.000
factory farming practices	0.000	0.000	0.000	0.000
fava beans	0.000	0.000	0.000	0.000
fenugreek	0.000	0.000	0.000	0.000
Fosamax	0.000	0.000	0.000	0.000
fructose	0.000	0.000	0.000	0.000
galactosemia	0.000	0.000	0.000	0.000
genetic manipulation	0.000	0.000	0.000	0.000
goji berries	0.000	0.000	0.000	0.000
halibut	0.000	0.000	0.000	0.000
hearing	0.000	0.000	0.000	0.000
hernia	0.000	0.000	0.000	0.000
hormonal dysfunction	0.000	0.000	0.000	0.000
IGF-1	0.000	0.000	0.000	0.000
industrial toxins	0.000	0.000	0.000	0.000
Iowa Women’s Health Study	0.000	0.000	0.000	0.000
junk food	0.000	0.000	0.000	0.000
lard	0.000	0.000	0.000	0.000
Lindane	0.000	0.000	0.000	0.000
liver disease	0.000	0.000	0.000	0.000
lyme disease	0.000	0.000	0.000	0.000
magnesium	0.000	0.000	0.000	0.000
maple syrup	0.000	0.000	0.000	0.000
mastitis	0.000	0.000	0.000	0.000
medical ethics	0.000	0.000	0.000	0.000
mesquite	0.000	0.000	0.000	0.000
Mevacor	0.000	0.000	0.000	0.000
milk	0.000	0.000	0.000	0.000
mouth cancer	0.000	0.000	0.000	0.000
myelopathy	0.000	0.000	0.000	0.000
National Academy of Sciences	0.000	0.000	0.000	0.000
Native Americans	0.000	0.000	0.000	0.000
neurocysticercosis	0.000	0.000	0.000	0.000
norovirus	0.000	0.000	0.000	0.000
nuts	0.000	0.000	0.000	0.000
okra	0.000	0.000	0.000	0.000
organotins	0.000	0.000	0.000	0.000
oxen meat	0.000	0.000	0.000	0.000
Parkinson's disease	0.000	0.000	0.000	0.000
Peoria	0.000	0.000	0.000	0.000
pesticides	0.000	0.000	0.000	0.000
plant-based diet	0.000	0.000	0.000	0.000
poisonous plants	0.000	0.000	0.000	0.000
polypropylene plastic	0.000	0.000	0.000	0.000
poultry workers	0.000	0.000	0.000	0.000
prolactin	0.000	0.000	0.000	0.000
prunes	0.000	0.000	0.000	0.000
pumpkin	0.000	0.000	0.000	0.000
quinine	0.000	0.000	0.000	0.000
rapamycin	0.000	0.000	0.000	0.000
red tea	0.000	0.000	0.000	0.000
rickets	0.000	0.000	0.000	0.000
shelf life	0.000	0.000	0.000	0.000
sirtuins	0.000	0.000	0.000	0.000
soil health	0.000	0.000	0.000	0.000
spearmint	0.000	0.000	0.000	0.000
Splenda	0.000	0.000	0.000	0.000
stevia	0.000	0.000	0.000	0.000
subsidies	0.000	0.000	0.000	0.000
taro	0.000	0.000	0.000	0.000
thiamine	0.000	0.000	0.000	0.000
Tufts	0.000	0.000	0.000	0.000
turnips	0.000	0.000	0.000	0.000
uterine health	0.000	0.000	0.000	0.000
veal	0.000	0.000	0.000	0.000
veggie chicken	0.000	0.000	0.000	0.000
tongue worm	0.000	0.000	0.000	0.000
vitamin K	0.000	0.000	0.000	0.000
walnut oil	0.000	0.000	0.000	0.000
weight gain	0.000	0.000	0.000	0.000
whiting	0.000	0.000	0.000	0.000
Yale	0.000	0.000	0.000	0.000
Zoloft	0.000	0.000	0.000	0.000
More Than an Apple a Day: Combating Common Diseases	0.000	0.000	0.000	0.000
Diabetes as a Disease of Fat Toxicity	0.000	0.000	0.000	0.000
The Actual Benefit of Diet vs. Drugs	0.000	0.000	0.000	0.000
Caloric Restriction vs. Plant-Based Diets	0.000	0.000	0.000	0.000
Infectobesity: Adenovirus 36 and Childhood Obesity	0.000	0.000	0.000	0.000
Does Cholesterol Size Matter?	0.000	0.000	0.000	0.000
Barriers to Heart Disease Prevention	0.000	0.000	0.000	0.000
Diabetics Should Take Their Pulses	0.000	0.000	0.000	0.000
Academy of Nutrition and Dietetics Conflicts of Interest	0.000	0.000	0.000	0.000
Do Vegetarians Get Enough Protein?	0.000	0.000	0.000	0.000
Eggs and Arterial Function	0.000	0.000	0.000	0.000
Treating Asthma With Plants vs. Supplements?	0.000	0.000	0.000	0.000
Alkylphenol Endocrine Disruptors and Allergies	0.000	0.000	0.000	0.000
Heart Disease Starts in Childhood	0.000	0.000	0.000	0.000
Artificial Food Colors and ADHD	0.000	0.000	0.000	0.000
Anti-Angiogenesis: Cutting Off Tumor Supply Lines	0.000	0.000	0.000	0.000
Cancer Risk From CT Scan Radiation	0.000	0.000	0.000	0.000
Eating Healthy on a Budget	0.000	0.000	0.000	0.000
Prolonged Liver Function Enhancement From Broccoli	0.000	0.000	0.000	0.000
Apple Juice May Be Worse Than Sugar Water	0.000	0.000	0.000	0.000
Preventing Strokes with Diet	0.000	0.000	0.000	0.000
Neurobiology of Artificial Sweeteners	0.000	0.000	0.000	0.000
Benefits of Fenugreek Seeds	0.000	0.000	0.000	0.000
More Antibiotics In White Meat or Dark Meat?	0.000	0.000	0.000	0.000
Filled Full of Lead	0.000	0.000	0.000	0.000
The Answer to the Pritikin Puzzle	0.000	0.000	0.000	0.000
To Snack or Not to Snack?	0.000	0.000	0.000	0.000
Boosting Good Bacteria in the Colon Without Probiotics	0.000	0.000	0.000	0.000
Human Neurotransmitters in Plants	0.000	0.000	0.000	0.000
Kiwifruit for Irritable Bowel Syndrome	0.000	0.000	0.000	0.000
Dietary Treatment of Crohn's Disease	0.000	0.000	0.000	0.000
Unsafe at Any Feed	0.000	0.000	0.000	0.000
Pharmacists Versus Health Food Store Employees: Who Gives Better Advice?	0.000	0.000	0.000	0.000
Preventing Cataracts with Diet	0.000	0.000	0.000	0.000
Cheese Mites and Maggots	0.000	0.000	0.000	0.000
Cholesterol and Lower Back Pain	0.000	0.000	0.000	0.000
EPIC Findings on Lymphoma	0.000	0.000	0.000	0.000
Sometimes the Enzyme Myth Is True	0.000	0.000	0.000	0.000
Vitamin C-Enriched Bacon	0.000	0.000	0.000	0.000
Out of the Lab Onto the Track	0.000	0.000	0.000	0.000
Better Than Goji Berries	0.000	0.000	0.000	0.000
How to Help Prevent Abdominal Aortic Aneurysms	0.000	0.000	0.000	0.000
The Difficulty of Arriving at a Vitamin D Recommendation	0.000	0.000	0.000	0.000
Amyloid and Apple Juice	0.000	0.000	0.000	0.000
Dietary Guidelines: From Dairies to Berries	0.000	0.000	0.000	0.000
Too Much Iodine Can Be as Bad as Too Little	0.000	0.000	0.000	0.000
Is Dragon Fruit Good For You?	0.000	0.000	0.000	0.000
Overdosing on Greens	0.000	0.000	0.000	0.000
Dietary Theory of Alzheimer's	0.000	0.000	0.000	0.000
Apthous Ulcer Mystery Solved	0.000	0.000	0.000	0.000
Update on Herbalife®	0.000	0.000	0.000	0.000
Saturated Fat & Cancer Progression	0.000	0.000	0.000	0.000
Fish Fog	0.000	0.000	0.000	0.000
Sexually Transmitted Fish Toxin	0.000	0.000	0.000	0.000
Veggies vs. Cancer	0.000	0.000	0.000	0.000
Is Coconut Milk Good For You?	0.000	0.000	0.000	0.000
Boosting Heart Nerve Control	0.000	0.000	0.000	0.000
The Healthiest Sweetener	0.000	0.000	0.000	0.000
Healthiest Airplane Beverage	0.000	0.000	0.000	0.000
Antioxidant Content of 300 Foods	0.000	0.000	0.000	0.000
Vitamin Supplements Worth Taking	0.000	0.000	0.000	0.000
Healthy Chocolate Milkshakes	0.000	0.000	0.000	0.000
The Healthiest Vegetables	0.000	0.000	0.000	0.000
How Doctors Responded to Being Named a Leading Killer	0.000	0.000	0.000	0.000
Avoiding Cooked Meat Carcinogens	-0.003	0.000	0.000	0.000
Flaxseeds & Breast Cancer Survival: Clinical Evidence	-0.003	0.000	0.000	0.000
muscle health	-0.004	0.000	0.000	0.000
coffee	-0.005	0.000	0.000	0.000
anisakis	-0.008	0.000	0.000	0.000
Titanium Dioxide & Inflammatory Bowel Disease	-0.010	0.000	0.000	0.000
Are Artificial Colors Bad for You?	-0.011	0.000	0.000	0.000
Alcohol Risks vs. Benefits	-0.013	0.000	0.000	0.000
pork	-0.015	0.000	0.000	0.000
Global Burden of Disease Study	-0.015	0.000	0.000	0.000
oral intraepithelial neoplasia	-0.019	0.000	0.000	0.000
insects	-0.022	+0.100	+0.015	-0.500
Is Distilled Fish Oil Toxin-Free?	-0.026	-0.100	-0.063	0.000
How Long to Detox From Fish Before Pregnancy?	-0.029	0.000	0.000	-0.167
salmon	-0.029	0.000	0.000	0.000
Preventing the Common Cold with Probiotics?	-0.031	0.000	0.000	0.000
St. John's wort	-0.043	0.000	0.000	-0.208
bone fractures	-0.048	-0.100	-0.019	0.000
peanut butter	-0.051	0.000	0.000	0.000
Foods for Glaucoma	-0.052	0.000	0.000	0.000
cauliflower	-0.052	0.000	0.000	0.000
Dragon's Blood	-0.061	0.000	0.000	0.000
alternative medicine	-0.065	-0.200	-0.006	+0.500
black raspberries	-0.065	0.000	0.000	0.000
Phytates for the Treatment of Cancer	-0.068	-0.200	-0.036	-0.125
Dioxins Stored in Our Own Fat May Increase Diabetes Risk	-0.069	-0.100	-0.016	0.000
phytic acid	-0.069	-0.100	-0.030	0.000
carcinogens	-0.073	-0.100	-0.004	0.000
Is Caffeinated Tea Really Dehydrating?	-0.073	-0.100	-0.019	0.000
Finland	-0.073	-0.100	-0.040	-0.143
leucine	-0.073	-0.100	-0.067	0.000
carrageenan	-0.074	-0.100	-0.111	0.000
NIH-AARP study	-0.075	0.000	0.000	-0.400
bronchiolitis obliterans	-0.078	0.000	0.000	0.000
grapes	-0.078	-0.100	-0.005	0.000
Are Organic Foods Safer?	-0.081	0.000	0.000	-0.500
Evidence-based medicine	-0.081	0.000	0.000	-0.500
sulfur	-0.090	0.000	0.000	-0.500
growth promoters	-0.095	-0.100	-0.016	-0.250
ultra-processed foods	-0.102	-0.100	-0.200	-0.111
Kuna Indian Secret	-0.102	-0.100	-0.091	0.000
Convergence of Evidence	-0.117	-0.200	-0.059	-0.143
tempeh	-0.118	-0.100	-0.111	0.000
suppositories	-0.118	0.000	0.000	-0.500
Aluminum in Vaccines vs. Food	-0.120	0.000	0.000	-0.500
Are Avocados Good for You?	-0.132	0.000	0.000	-0.500
memory	-0.133	-0.200	-0.027	0.000
Is apple cider vinegar good for you?	-0.135	-0.100	-0.200	0.000
Hiroshima	-0.135	0.000	0.000	-0.800
chickpeas	-0.149	-0.200	-0.022	0.000
molasses	-0.151	0.000	0.000	-0.667
Rutin	-0.151	0.000	0.000	-0.667
Bowel Movement Frequency	-0.154	-0.200	-0.118	0.000
hyperactivity	-0.159	-0.200	-0.167	0.000
BPA Plastic and Male Sexual Dysfunction	-0.168	0.000	0.000	0.000
pineapples	-0.177	0.000	0.000	-0.833
titanium dioxide	-0.179	-0.200	-0.182	0.000
philippines	-0.179	-0.200	-0.182	-0.333
chlorophyll	-0.195	-0.100	-0.250	0.000
Harvard Physicians’ Study II	-0.196	-0.200	-0.333	-0.143
Counteracting the Effects of Dioxins Through Diet	-0.212	-0.100	-0.071	-0.500
kidney beans	-0.224	-0.100	-0.042	-0.750
cadaverine	-0.263	-0.100	-0.167	-0.500
The Saturated Fat Studies: Buttering Up the Public	-0.290	-0.200	-0.286	-0.250
BPH	-0.302	-0.200	-0.105	-0.500
sweeteners	-0.308	-0.400	-0.069	0.000
Panama	-0.456	0.000	0.000	-0.667
kohlrabi	-0.631	-0.100	-1.000	-0.500