TensorOpt

Liferay DEVCON 2026, QEII Centre, London

How Good Is Hybrid Search, Really? An Independent, Statistically Sound Evaluation

4 November 2026, 13:50 to 14:35 GMT, Track B (St James, 4th floor)

  • Tibor LipuszTibor LipuszProduct Manager, Search @ Liferay
  • László CsontosLászló CsontosSearch & Retrieval Consultant @ TensorOpt

About the session

The session abstract, as submitted:

Modern, hybrid search experiences require blending traditional lexical search with AI-powered methods like semantic search. While hybrid search is easy to enable, evaluating its effectiveness remains a challenge.

In this Deep Dive, we measure hybrid retrieval techniques—combining BM25, embeddings, and Reciprocal Rank Fusion (RRF)—against lexical-only retrieval. We also explore relevant Liferay DXP and proprietary Elasticsearch capabilities available via a Liferay Enterprise Search (LES) subscription. Using the public WANDS e-commerce dataset, we apply bootstrap confidence intervals and paired significance tests to identify whether and where hybrid retrieval provides a statistically significant advantage.

This session is co-presented by Tibor Lipusz, Liferay's Search Product Manager, and László Csontos, an independent search-evaluation consultant from TensorOpt. Together, they provide the product configuration context and the detailed methodological analysis necessary to optimize modern searches.

The session asks how good hybrid search is, compared with keyword search and vector search, on a public dataset with graded relevance labels. It measures the query as the unit, with confidence intervals and paired tests instead of averages alone, and it shows how to start measuring search on your own content.

The 9 levels of the search measurement framework: where are you?

TensorOpt's framing, not an industry standard

Observe

  • Level 1: Nothing measuredSearch Insights: shows the query sent to Elasticsearch for Search Bar searches. Useful for testing, development and troubleshooting.Analytics Cloud: lists search terms and how many users searched for each, but does not store the results shown for each query.
  • Level 2: Search logThe free plugin records each query with the results it returned.

Log queries and the results shown (level 2).

Levels 3 to 8: the measurement you add.

Label

  • Level 3: Labelled sample
  • Level 4: Labeller agreement
  • Level 5: Calibrated LLM judge

Label a sample on a graded scale; check agreement (levels 3 and 4).

Decide

  • Level 6: Compare averages
  • Level 7: Compare with statistics

Fix the metric, label weights and comparisons before you run, then compare with confidence intervals, not averages (level 7).

Connect

  • Level 8: Business outcomes
  • Level 9: Online A/B testsAnalytics Cloud A/B testing: click-through, confidence intervals and probability to win for search configurations, using Page Experiences with a different Blueprint for each experience.

Tie search to one business outcome (level 8).

Each level in detail: what exists there, what you can decide, and the trap at that level. On the TensorOpt blog the framework is called the measurement ladder, and each level a rung.

Start at level 2: log your searches

Search Eval Logger is free and open source (Apache 2.0). It records each search together with the results shown, and never changes a search. It never contacts TensorOpt or anything else, from the server or the browser, and it never records user IDs, names, email addresses or IP addresses.

Base Liferay DXP keeps no stored record of each query together with the results it returned.

Collection is off until you switch it on. With the default settings the plugin tells you the log is ready to export after 30 days and 500 searches, and it keeps 90 days of records. The export is a zip file that stays on your machine.

The plugin sends nothing out: one zip file on your machine, no other destinationInside your Liferay server sits the Search Eval Logger plugin. One arrow leads from it to a zip file on your machine, made only when an administrator exports. A second arrow, to anywhere else including TensorOpt, is crossed out: the plugin sends nothing there.Your Liferay serverSearch Eval Logger pluginon exportneverZip file onyour machineAnywhere else,TensorOpt included
The plugin writes one zip file, only when an administrator exports. It sends nothing anywhere else.

Query text can still be personal data. The operator decides, and anyone who receives an export needs a data processing agreement.

Materials published after the session

The slides, the results and the code will be published on this page. None of it needs an email address.

Next step: measuring your Liferay search

Optional. Nothing else on this page needs an email address. Sign-ups from 4 November start at the email on search logs (email 3), since attendees have seen the first two in the talk. Earlier sign-ups start at the beginning.

Measure the search you can't see

Learn how to tell whether your search works, and how to measure a change before you make it. Five short emails over two weeks:

  1. Why misses stay invisible: users judge what is on the screen, never what is missing.
  2. The measurement ladder: nine rungs, and where your system sits.
  3. The first rung to climb: a log of each query and the results it returned.
  4. Getting started safely: the free plugin through security review on Liferay DXP, or, on other stacks, where accuracy is lost first.
  5. Two averages are not a decision: what to measure before you switch on semantic search or change a parser, embedding or model.

Then at most two short follow-ups. You also get practical files that are not on the site: a plugin rollout checklist (Liferay DXP) or a logging schema (other stacks), and a labelling kit.

What does your search run on?

About, and contact

TensorOpt designed and ran the analysis. Liferay reviewed statements about its products, not the findings.

TensorOpt measures search where wrong results are costly, independently of any vendor. Read about measuring Liferay search, or write to info@tensoropt.ai.

Disclosures

Liferay is a trademark of Liferay, Inc. TensorOpt is independent and not affiliated with Liferay.