Beyond fine tuning

//Beyond Fine-Tuning: Finding Smarter Ways to Retrieve Financial Information with AI 

From training better embedding models to navigating complex documents, two consecutive research projects at Notilyze explored how to make financial AI systems more accurate and efficient.  

The challenge: Finding the right information 

Imagine asking an AI assistant a question about a bank’s sustainability performance. The answer might be hidden somewhere in a 300-page annual report, buried under environmental risks, corporate governance, or sustainability disclosures. Different banks might even describe the same topic using different terminology.  

Since 2024, the EU’s Corporate Sustainability Reporting Directive (CSRD) has begun requiring companies within its scope to disclose information about their environmental, social, and governance (ESG) impacts. These disclosures follow the European Sustainability Reporting Standards (ESRS). However, because the standards are relatively new, reporting practices and terminology can differ between banks. The same requirement may appear in different sections of lengthy  annual reports, making information difficult to locate and limiting the availability of consistent training data.  

For AI, finding the right information is often harder than generating the answer itself. This is where Retrieval-Augmented Generation (RAG) comes in. RAG allows Large Language Models (LLMs) to retrieve information from external documents before generating the answer, rather than relying solely on their training data. But RAG introduces a fundamental dependency: the quality of the answer depends heavily on the quality of the retrieved information.  

At Notilyze, two consecutive research projects investigated how to improve this critical part of RAG systems, each approaching the problem from a different perspective.  

Step 1: Teaching the model to understand financial language 

The first project, conducted by Douwe Terlouw and Joep Bom, focused on fine-tuning embedding models for financial information dense retrieval.  

Dense retrieval uses embedding models to represent questions and document passages as numerical vectors. It then identifies relevant passages based on semantic similarity rather than exact keyword matches. This is very useful in finance, where the same concept may be expressed in several ways. However, general-purpose embedding models do not always recognize specialized financial terminology. To address this, the researchers fine-tuned embedding models using data from European bank sustainability reports. The results confirmed that domain-specific fine-tuning improves retrieval quality.  

The illustration explains how relevant examples are brought closer to a query, and irrelevant examples are pushed farther away. 

Fine tuning method


But left another question: What if the problem isn’t only how well the model understands the text, but how the documents are searched?  

Step 2: improving how the system searches 

That question became the focus of the next research project by Bente Verhoef and Noortje Peters. 

Fine-tuning helps a model recognize relevant information. But financial documents present another challenge: their structure.  

Traditional dense retrieval generally searches across individual text chunks, but annual reports are organized into chapters, sections, and subsections, and this hierarchy provides valuable context about where information belongs. This method resembles more about how a human would search for a document.  

Bente investigated whether incorporating document hierarchy could improve further retrieval. The researcher compared dense retrieval with PageIndex, a system that organizes documents into hierarchical trees and uses an LLM to navigate the tree and identify the sections most likely to contain the answer. 

But Bente also developed Hybrid PageIndex, which combines hierarchical document navigation with semantic embeddings. Instead of repeatedly asking an LLM which branch to explore, the hybrid approach uses embedding similarity to guide the search. The goal was to preserve the benefits of document structure while reducing the costs of LLM-guided navigation.  

Retrieval methods

What did the research reveal?  

Bente evaluated the approaches on two datasets: sustainability-related reports from 23 European banks and FinanceBench, a financial question-answering benchmark.  

The results revealed an important trade-off. PageIndex achieved the highest retrieval accuracy on both datasets. On the European banking sustainability dataset, it achieved a retrieval ranking score (MRR@5) of 73.9%, indicating that relevant information was consistently ranked near the top of the search results. However, this performance came at a cost: PageIndex required repeated LLM reasoning, resulting in substantially higher query-time latency. 

Hybrid PageIndex offered a different trade-off. It was significantly faster than PageIndex and outperformed the corresponding dense retrieval models on FinanceBench. But on the banking sustainability dataset, the hybrid approach performed worse than flat dense retrieval. 

This highlighted a crucial finding: adding document structure does not automatically improve retrieval. The way a system navigates that structure matters. 

Retrieval Results  Banking and Finance Comparison

Better accuracy or better efficiency?  

The research also revealed why retrieval performance cannot be measured through accuracy alone. PageIndex delivered the strongest results, but its LLM-guided search required significantly more time and computational resources. Dense retrieval and Hybrid PageIndex offered much lower query-time latency.  

For organisations deploying AI at scale, that difference matters. A system supporting high-stakes financial analysis may prioritise accuracy, while a high-volume application may need faster responses.  

Two approaches, one bigger lesson 

Together, these projects highlight two complementary ways to improve financial AI. Fine-tuning improves how a model recognizes relevant information. Hierarchical retrieval improves how a system organizes and navigates that information. 

Neither approach is a universal solution. The results show that structure can deliver significant improvements, but its effectiveness depends on the data and navigation strategy. Combining embeddings with hierarchy can help but does not guarantee better performance.  

For organisations working with complex financial documents, the takeaway goes beyond choosing one technology over another. Building reliable AI is not only about making models smarter. It is also about designing better ways for them to find the information they need.  

*This research investigation was focused on finance; however, these same methods can be extrapolated to other fields, where they use specific jargon. Theredore, if the RAG system is updated, it can perform in other fields. Because the goal is to improve the RAG system beyond its training data and default values.  

This article is based on the research projects “Fine-Tuning of Embedding models to Improve retrieval-augmented generation systems” by Douwe Terlouw and Joep Bom (2026), and “Combining hierarchical structure and semantic retrieval in financial RAG systems” by Bente Verhoef and Noortje Peters (2026), Erasmus University, Rotterdam. 


About Notilyze

We are Notilyze, a SAS® Partner with only one goal: to help organizations with data analytics, and become more data-driven and successful. Our aim is to bring data analytics within reach of every department and every company. We do this by offering SAS® as a Service. Notilyze Cloud is the quickest solution to start with AI on SAS® Viya®. In addition, we offer IT solutions to properly integrate analytics into existing processes, and the experience of our SAS® specialists.

// Contact

Notilyze B.V.
Stationsplein 45 A4.004
3013 AK Rotterdam
+31 (0)10 79 86 295
info@notilyze.com

// Ask a question