AI Search introduction

​

Overview

Nobly AI Search combines two fundamentally different search technologies to find documents:

• BM25 (Text/Keyword Search): Traditional full-text search that finds documents containing the exact words you typed. Similar to how a search engine matches keywords — it looks for the specific terms in your query across document text, keywords, and metadata.
• Vector/Semantic Search: AI-powered search that understands the meaning behind your query. Even if a document doesn’t contain the exact words you typed, it can be found if it covers the same concept. For example, searching “medical expenses” could find documents about “health insurance claims” or “hospital invoices.”

These are combined with Reciprocal Rank Fusion (RRF), and can be further refined by two optional LLM stages: query rewrite before search and result reranking after search. See LLM reranker and query rewrite for details.

​

Search config

AI Search does not use named modes. Instead, the Search config section of the settings panel exposes three independent toggles that together determine how a query is processed.

​

Auto weights

When on, the server automatically picks the BM25 vs semantic weight split for each query. When off, two sliders appear and you set the weights yourself — they always sum to 100%, step in 5% increments, and default to 70% BM25 / 30% Semantic. Turning Auto weights off also disables LLM rewrite, because rewriting is only applied when the server is free to choose weights.

​

LLM rewrite

An LLM rewrites your query into BM25-friendly and embedding-friendly variants before search runs. Hidden when the feature is disabled on the server. See LLM query rewrite.

​

LLM rerank

An LLM reads the top candidate results and re-orders them by how well they actually answer your query, producing a 0–100 relevance score and a short reasoning text per result. Hidden when the feature is disabled on the server. See LLM reranker.

​

How results are ranked

Each search passes through a pipeline. The stages that actually run depend on your Search config and server availability.

​

Stage 1: Query rewrite (optional)

If LLM rewrite is on, the query is rewritten into BM25 and embedding variants before retrieval. The original query is always included alongside the rewritten forms.

​

Stage 2: BM25 keyword ranking

A full-text search ranks documents by how well they match your query terms. This considers:

• Term frequency (how often your search terms appear)
• Document length (shorter documents with the same matches score higher)
• Term rarity (rarer terms contribute more to the score)

​

Stage 3: Vector similarity ranking

Your query is converted into a high-dimensional vector and compared against pre-computed vectors for every indexed document chunk. Documents whose vectors are closest to your query vector rank highest. Measured as cosine similarity (0.0 = completely unrelated, 1.0 = identical meaning).

​

Stage 4: Reciprocal Rank Fusion (RRF)

BM25 and vector rankings are combined into a single ordering using RRF, weighted by the BM25/semantic split chosen by Auto weights (or your manual sliders). RRF ensures neither ranking system fully dominates — a document ranked #100 in BM25 but #1 in vector still surfaces.

​

Stage 5: LLM reranker (optional)

If LLM rerank is on, the top RRF candidates are read by an LLM that scores each one against your query (0–100) and produces a short reasoning text. The LLM score replaces the RRF score as the displayed relevance score.

​

Summary

​

Understanding the relevance score

Each result displays a relevance score from 0 to 100. The source depends on whether LLM rerank ran:

• With LLM rerank: the score is the reranker’s direct 0–100 estimate of how well the document answers your query.
• Without LLM rerank: the score is derived from the RRF fusion, capped by semantic similarity so the top score reflects actual match strength rather than just position in the result list.

Hovering the info icon next to the score shows:

The relevance score (0–100%) shows how well each result matches your search. It is capped by semantic similarity so the top score reflects actual match strength, not just ranking. Scores are relative to the best result in the current set — so the same document can have different scores depending on what else is returned.

Important properties:

• Scores are relative to the result set. The same document can show different scores depending on which other results were returned.
• The top result always has the highest score, but its absolute value depends on query quality. A strong match might show 92; a weaker query might top out at 55.
• Lower scores are not necessarily bad. A top score of 60 on a partial-term keyword query can still be the right document.
• Rerank-on and rerank-off scores are not directly comparable. A 70 from LLM rerank and a 70 from RRF are measured differently.

​

Match sources - Why did this result appear?

Hovering over (or clicking) a result’s relevance score reveals detailed information about why this result was returned. Each result can have one or more match sources:

​

TextContent

The document’s text content was semantically similar to your query. This means the AI understood a conceptual connection between your query and the document’s actual text, even if the exact words differ. When available, the match detail shows which page numbers contain the relevant text and an excerpt of the matching passage.

​

Keywords

One or more of the document’s Nobly Insight keyword values matched terms in your query. The match detail shows exactly which keyword type and value matched, and which of your query terms caused the match. Example: Searching for 847291 might show:

Keywords — Kundenummer: 847291

​

Metadata

The document’s metadata fields (document name, document type, or creator username) matched terms in your query. Example: Searching for Hansen might show:

Metadata — DocumentName: Policy-Hansen-2024.pdf

​

Multiple sources

A result can appear from multiple sources simultaneously. A result that matches via both Keywords and TextContent is generally a stronger match than one appearing from a single source — it means the document matched both on exact data and on conceptual meaning.

​

What gets searched

When you run an AI search, your query is matched against a composite of all available information about each document:

• Document name — the file/document title
• Document type name — the Nobly Insight document type
• Creator — the username that stored the document
• All keyword values — every keyword type and value assigned to the document (formatted as “type: value”)
• Full document text content — the entire extracted text from the document (PDF text, Word content, etc.)

All of these are combined into a single searchable index, so a keyword search for “Hansen” will match whether “Hansen” appears in the document name, a keyword value, the creator field, or within the document’s actual text. For semantic/vector search, the same composite content is used to generate document embeddings, so the AI model understands the full context of each document.

​

Search progress stages

While a search runs, a progress indicator shows what the system is doing. The exact stages depend on your Search config and server features:

• Resolving search options…
• Rewriting query… (only with LLM rewrite)
• Building embeddings…
• Searching…
• The AI is reading the results and ranking them by relevance… (only with LLM rerank)
• Finishing…

After results arrive, badges at the top of the result table show which LLM stages were actually applied for this query — this can differ from your toggle state if a server-side feature was unavailable.

​

Search query tips

​

For exact lookups (IDs, numbers, codes)

Just type the value directly. With Auto weights on, the server picks an appropriate BM25/semantic split for code-like queries.

2001701234 INV-2024-0847

​

For finding specific documents

Use a few distinctive terms — Auto weights will balance exact and semantic matching.

Hansen pension insurance policy 2024

​

For exploratory / conceptual search

Describe what you’re looking for in natural language. LLM rerank sharpens the ordering further when enabled.

documents about employee health benefits changes customer complaints regarding delayed payments

​

General advice

• Be specific when you can. More distinctive terms lead to better BM25 matches.
• Use natural language for broad topics. The semantic engine and the LLM reranker excel at understanding intent.
• Don’t worry about exact wording. LLM rewrite expands your query to cover variations.
• Try turning Auto weights off if the default split doesn’t surface what you expect — the sliders give you direct control.

​

Search operators and syntax

Currently, AI Search does not support advanced search operators. The following do not work: Your query text is used as-is, or — with LLM rewrite on — passed through an LLM before being sent to BM25 and semantic search. Because the search uses both BM25 and semantic matching:

• BM25 naturally handles multi-word queries by matching individual terms
• Semantic search understands phrasing and context, so typing health insurance claims will find documents about that topic even without phrase operators
• The BM25 tokenizer may apply stemming at the database level, which provides some automatic fuzzy matching for word variations

​

Advanced settings

The settings panel next to the search bar is organized into collapsible sections.

​

Search config

• Auto weights — on by default. When off, set Text weight and Semantic weight sliders manually (0–100%, step 5%, always summing to 100%).
• LLM rewrite — on by default. Requires Auto weights. Hidden when disabled on the server.
• LLM rerank — on by default. Hidden when disabled on the server.

See LLM reranker and query rewrite for details.

​

Document types

Restrict the search to one or more document types or document type groups. When empty, all accessible document types are searched.

​

Date range

Restrict results to documents with a document date between From and To. Either endpoint is optional.

​

Columns

Toggle which columns appear in the results table and drag to reorder. Available columns:

• Document ID
• Document name
• Document type
• Document date
• Created by
• File extension
• Reasoning (populated by LLM rerank)

You can also add Nobly Insight keyword types as additional columns to see keyword values directly in results without opening each document. The score column always remains first and cannot be toggled off or reordered.

​

Max results

Control how many results are returned: 10, 20, 50, 100, 200, 500, or 1000. Default is 50. Higher limits take longer to process — LLM rerank time scales roughly with this number. The server may enforce a maximum cap.

​

Understanding different score ranges

Here are practical examples of what different score ranges typically mean. Remember that scores differ meaningfully depending on whether LLM rerank ran — treat these as rules of thumb rather than fixed thresholds.

​

High scores (75-100)

Strong match. The document is highly relevant to your query.

​

Medium scores (45-74)

Moderate match. The document is related to your query but may not be a direct hit. Could be:

• A partial keyword match (some terms found, others not)
• A conceptually related document that covers adjacent topics
• A document where your terms appear but in different contexts

​

Low scores (20-44)

Weak match. The document has some tenuous connection to your query. Worth checking if the higher-ranked results didn’t have what you need, but don’t expect a strong match.

​

Very low scores (below 20)

Marginal match. Typically only appears when few results are available and the system is returning the least-bad options. Consider refining your query.

​

Security and permissions

AI Search respects Nobly Insight document security at all times:

• Only documents the user has permission to view are returned. Permission checking happens at the database level during search execution, not after.
• Security keywords (keyword types flagged for security) are synced to the search index and used for access control evaluation.
• User group permissions are synced periodically (every 30 minutes by default) from Nobly Insight to the search index.
• A user will never see documents in search results that they would not be able to access through normal Nobly Insight document retrieval.

If a user reports “missing” documents in search results, verify that:

1. The document has been indexed (indexing is a separate process)
2. The user’s group has permission to the document type
3. Security keyword restrictions are not filtering the document out

​

Frequently asked questions