Your users search. Do they find?
Independent, statistically sound search and RAG evaluation where a wrong result is expensive.
Users judge what search shows them, and nobody sees what it missed.
You can't see what search didn't return
People judge search by what appears on the screen. What it missed never appears, so nobody complains about it.
In a classic study, searchers believed they had found at least 75% of the relevant documents; measured recall averaged about 20% (Blair and Maron, 1985). The caveat: an old Boolean system used in legal discovery. The point is the gap between belief and measurement, not the number.
In legal and financial work that blind spot is expensive. Tested AI legal research tools “each hallucinate between 17% and 33% of the time” (Magesh et al., 2025, Journal of Empirical Legal Studies).
Who this is for
By system
- Portals and intranets. Customer, member and advisor portals, and intranets that hold policies, procedures and product documents. Liferay DXP sites start here.
- Document search and RAG. Search or question answering over contracts, case law, regulations, filings, policies and research, on any stack. Start here for documents.
By industry
- Legal. Law firms, in-house legal and legal tech. A public database counted 1,058 US court cases involving improper generative AI use in filings as of the start of June 2026 (Charlotin database, as summarised by the UNC Law Library).
- Finance. Banking, insurance, wealth and asset management, and fintech. When firms use AI, “firms’ decisions remain the responsibility of management bodies” (ESMA public statement, 30 May 2024).
The measurement ladder
From “search is switched on” to “we know it works”, one rung at a time. This is TensorOpt's framing, not an industry standard.
| Group | Rung | What exists at this rung |
|---|---|---|
| Observe | 1 | Search is switched on. Nothing is measured |
| Observe | 2 | A search log: queries and the results shown. Top queries and zero-result queries become visible |
| Label | 3 | A sample of logged queries has relevance labels; precision, recall and nDCG can be computed |
| Label | 4 | Agreement between labellers is measured, so label reliability is known |
| Label | 5 | An LLM judge, checked against human labels, scores results at scale |
| Decide | 6 | Configurations compared by their average scores (the trap: two averages can differ by chance) |
| Decide | 7 | Configurations compared with confidence intervals, paired tests and multiple-comparison control |
| Connect | 8 | Search metrics tied to outcomes (time to find, tickets, errors caught) |
| Connect | 9 | Configurations A/B tested on live traffic |
Rungs 8 and 9 are not sold.
[OWNER] Link to the ladder blog post, once it is live.
How to start
Free: record and look
On Liferay DXP, a free plugin records user searches and the results shown. For other stacks, the open-source evaluation harness is for teams that want to run the statistics themselves. The search log guide shows what to look for in a log with no labels.
EUR 2,000
Readout
An independent reading of what your log, or 30 to 100 questions your experts write, means for your decision, what to measure first, and a fixed quote for the Evaluation.
EUR 7,500
Evaluation
One decision measured: labels from your experts, confidence intervals and paired tests.
EUR 2,000 / month
Regression Retainer
Up to two changes a month, measured against the same judgment set.
EUR 8,000
Team Workshop
For teams that want to measure their own systems: evaluation and statistics for search and RAG teams.
Prices in EUR, excluding VAT. Prices in GBP and USD on request.
See the packagesMeasure the search you can't see
Learn how to tell whether your search works, and how to measure a change before you make it. Five short emails over two weeks:
- Why misses stay invisible: users judge what is on the screen, never what is missing.
- The measurement ladder: nine rungs, and where your system sits.
- The first rung to climb: a log of each query and the results it returned.
- Getting started safely: the free plugin through security review on Liferay DXP, or, on other stacks, where accuracy is lost first.
- Two averages are not a decision: what to measure before you switch on semantic search or change a parser, embedding or model.
Then at most two short follow-ups. You also get practical files that are not on the site: a plugin rollout checklist (Liferay DXP) or a logging schema (other stacks), and a labelling kit.
DEVCON 2026
László Csontos and Tibor Lipusz present "How Good Is Hybrid Search, Really? An Independent, Statistically Sound Evaluation" at Liferay DEVCON 2026 in London on 4 November.
Evidence
The finance demonstration on the public FiQA dataset is being rebuilt and is due 15 December.
The evaluation harness is open source: github.com/TensorOpt/search-rag-evaluation. More on the method, and what each demonstration does and does not show, is on the methodology page.
Independence
- TensorOpt is paid only by the client that buys the work.
- TensorOpt is not affiliated with any search vendor, or with Liferay.
- TensorOpt measures and reports. It does not implement the changes it recommends.
- The free plugin sends TensorOpt nothing.
- Author of Designing Hybrid Search Systems
- Author of the forthcoming Measuring Search
- Author of the open-source Search Eval Logger for Liferay DXP and of the open-source evaluation harness