Berlin Buzzwords is a yearly conference for search technology. As with tech other conferences, it has been converging towards being mostly about AI in the 2020s. Most technologies presented are open source, speeches are technical and without annoying sales pitches from companies and consultants selling software or services.
These notes are probably not useful for anyone except myself.
Unpacking Digital Sovereignty: How to avoid fueling the nationalist rise
SEE_AGAIN? Litt TED-aktig. Mye GPP. Enig med det meste. Litt veeeel progressivt…
Digital sovereignty: No clear definition. Technological dependencies so strong that they become dangers.
Eurostack All levels from rare earths, all the levels up to AI tools. - 1. Buy european. Broad support from european tech industry (of course!). Easier to make - 2. Go open source.
Danger: Sovereignty may easily be captured by nationalist. Eg. EU draft fo r digital sovereignty written by nationalist -> subsidies.
Data centers on European soil marketed by US tech companies as “sovereign”.
Open source -> more decentralised -> more sovereign
Two aspects of open source 1. more democratically managed 2. easier to manage without licences.
Will digital sovereignty lead to more money to open source development?
Military investment -> More secrecy -> Hurdle for open source?
Open Source AI definition: “sufficient detailed information about the data used to train the system so that a skiled person can reproduce”
Popular political narrative: Bright future as AGI will be developed -> AI answer to all questions (ie quasi religion) -> invest hevaily in AI in order not to be left behind.
“Potemkin-AI” There is a long list of services that purport to be powered by sophisticated software, but actually rely on humans acting like robots. Autonomous vehicles use remote-driving and human drivers disguised as seats to hide their Potemkin AI.
Digital eugenics -> Humans may be improved through AI.
TODO - Break up giant corporations - Requlate + tax - Public led infrastructure
Going Local-First: A Primer
Speaker: Miloš Sutanovac
SEE_AGAIN? Fin intro til localfirst, nyttige lenker, hovedtema, lett å følge med
This seems to be the reversal of cloud computing!
“The local-first paradigm promises transformative benefits — user-owned data, seamless offline capabilities, and instant interactions. But how do you get started?”
- Fast no spinners
- Collaboration
Local first: You won’t even notice that a server is down! -> everything continue to work Great SW examples: Linear, Figma, Notion (Dropbox -data ownership) Excalidraw, Obsidian (not opensource), Anytype
Linear.app : A sync engine to make it easier to create local first software
McLuhan quote: “We shape our tools and our tools shape us”
CRDT Conflict free replicated data type: Datatype with version control. Automerge, YJS https://github.com/alangibson/awesome-crdt
JAZZ: Evertythin created is colloborative first. Podcast: Localfirst
When is local first not a good idea? Banking, social media…
Building a knowledge graph for climate policy
SEE_AGAIN? Engelsk morsmål+, usikkert hva de har fått av merverdi ved å lage kunnskapskraften. Likte at de brukte Wikibase.
“At Climate Policy Radar, we’re building an open-source knowledge graph for climate policy. In this talk, we’ll share how we combine in-house expertise with scalable data infrastructure to identify key concepts in thousands of global climate policy documents. We’ll also touch on ontology design, equitable evaluation, and the climate impacts of AI.”
Why knowledge graph? Domain is confusing no newcomers. NER can be automated a lot! Semantic search is like NER + superstructures of sub-categories, super-categories, hierarchies. Goal: Capture domain knowledge from human experts in category system.
Storage engine: Wikibase ❤️. Bonus: Norms from wiki community Example term in Wikibase. Q: Why not use Wikidata, instead of Wikibase? A: WS overwhelming -> keep simple and easy
A decade of lessons in Open Source licensing
Speaker: Justin Mclean
SEE_AGAIN? ikke-teknisk foredrag.
Goals of open source : - Reduce legal risk for SW users - Remove legal risk for contributors.
Top Misconceptions to debunk - “All OS licences are compatible” -> they’re not. Some are not compatible to mix, some only in one direction… - “its on Github so I can use it” - Developer often belive that anything can be scraped from the internet and used as test/trainging data.
Licence categories: A all good, B conditional, C Forbidden.
Apache Rat: Release management tool
Trends - EU cyber resilience act. The Cyber Resilience Act is the first European regulation to set a minimum level of cyber security for all connected products available on the EU market. - US SBOM mandates. Mandatory SBOM Requirements: Program Executive Offices (PEOs) and Program Managers (PMs) must incorporate contract language requiring vendors to generate and deliver SBOMs for all covered computer software, including Government-off-the-Shelf, contractor-developed, commercial, and open-source software. - EU AI act - AI generated code ? Unclear legal status -> Not copyrightable in EU/US/China
Q: Who is responsible for copyright violation.
Q: Can Apache Licence limit who can use the software? No! then its not a OS licence anymore.
“Drawing from experience reviewing over 1000 open-source releases, I’ll address common misconceptions, frequent compliance issues, and the evolution of policies to mitigate these challenges.. Attendees will gain practical insights to ensure smoother project releases and foster a compliant, collaborative, open-source community.”
Nick Burch – Self-hosting AI LLMs - a beginners guide
Overfiled speech: What you see is what you mean; intent based ecommerce search
Performance Tuning Apache Solr for Dense Vectors
Speaker: Kevin Liang Bloomberg (on prem internal Solr-as-a-service)
SEE_AGAIN? Klar tydelig case study av Solr vektorsøk.
Use case: Search that do not exclude relevant documents, and are not bloated with too many hits. User writes NL search, return summary from documents.
BERT with 1024-dimensions. Search with cosine similarity.
Tip: Normalize vectors to reduce computation expence of cosine similarity.
Optimization: Use JDK21 + “project panama” Make vector field non stored (???)
“While powerful, dense vector search is not a plug-and-play feature that will scale straight out-of-the-box, particularly when it comes to extracting the maximum performance from limited compute resources. Come learn how we tuned dense vector indexes for our 100M+ document dataset, and drastically sped up our queries.”
Best Practices for Running Databases on Kubernetes
Speaker: Peter Zaitsev
15 years ago, marketers used to compare Cloud services with electricity grids.
Many DB-as-a-service use Kubernetes underneath.
- Store production DBs on persistent storage, not inside some container
- Keep data per Pod small (not many XX TB)
- use appropriate node size
- Set resource limit
- …
- Tune DB Tip: Percona managment Open source tool.
- scaling.
- pod priority. After restarting a pod, it needs time for crash recovery or heating up cache.
- …
- Enable encryption.
- use Kubernetes secrets to pass credentials to you applications.
- backups
- Discover new generations of databases (post-mysql). Vitess = package mysql in cloud.
- CPU choises. Better performance by ARM chips?
- Pick correct kubernetes deployment.
- Monitor utilization
Coroot tool from Percona provides observability for Postgres
“Running open source databases on Kubernetes? Learn best practices for high availability, security, backups, and disaster recovery. Discover key pitfalls to avoid and see how Operators simplify database management for MySQL, MongoDB, and PostgreSQL in Kubernetes environments.”
How I Sidestepped ‘Being Glue’
Speaker: Fatima Taj > SEE_AGAIN? Motsatte problemet av meg, å bli satt til prosjektadministrasjon i stedet for noe som meritterer :-)
https://www.noidea.dog/glue
Qumat: Apache Mahout Quantum Compute
Speakers: Trevor Grant, Andrew Musselman
Apache Mahout is an open source project to create scalable, machine learning algorithms. Mahout operates in addition to Hadoop, which allows you to apply the concept of machine learning via a selection of Mahout algorithms to distributed computing via Hadoop.
Support added for (simulating) quantum comping.
Apache Mahout’s support for quantum computing, through its Qumat project, is designed for both simulating quantum computation and running on actual quantum computers. Qumat provides a vendor-neutral interface that allows users to write quantum circuits and algorithms once and then execute them on a variety of backends, including both simulators and real quantum hardware
End-to-End Semantic Search with Apache Solr 9.8 LLM Module
Speaker: Alessandro Benedetti
SEE_AGAIN? Ypperlig oversikt over LLM-modulen i Solr. Oversikt over fremtidig og nåværende funksjonalitet.
“Apache Solr 9.8 introduces the LLM module opening the doors of end-to-end natural language query support through vector-backed semantic search (K Nearest Neighbors). This talk explores the open source contribution from both the indexing and query angles and what’s coming next for Solr in terms of integrations with Large Language Models.”
Semantic similarity: - Same word different meaning - Different words same meaning
Vector search added to Apache Solr 9 in 2022.
Pre-filtering or Post-filtering? Vector similarity function query can be used in learning to rank
What was missing according to speaker? Vectorisation model that convert document to
Apache Solr LLM model implements LangChain4j Support EXTERNAL text vectorisation (so far)
Tip: Split indexing in 2: 1 LLM vectorisation 2. traditional solr indexing -> join
Future features - Local models - RAG - Better hybrid search Missed feature with high pri Vector quantization?
Contexts & Machines: How Document Parsing Shapes RAG results
Speakers: Alessio Vertemati, Andrea Ponti (onnofftech.xyz)
SEE_AGAIN? Testa 3-4 måter å chunke PDFer opp mot hverandre og ytelsesmålt dette.
Workflow Documents (often pdf) > text > chunking
Team used a 4 libraries for PDF parsing (PyMyPDF, PdfAct, LlamaParse) 2 Chunkers: LangChain (rule based), Chonkie (semantic)
Rule based: Create larger chunks Semantic: Large variance in size
Conclusions “Unstructured” highest quality for visually structured data like printed tables
“How different document parsing and chunking strategies impact RAG pipeline performance? Using real-life documents and LLM-generated question/answer pairs, we assess multiple methods – both open-source and commercial – showing that parsing quality significantly affects response accuracy and that the best approach may depends on the question type.”
Hybrid search on hybrid models, at scale
Speakers: Radu Pop, Pietro Mele
SEE_AGAIN? Søk i fransk Nasjonalbibliotek. Nylig gått fra leksikalsk søk i metadata til vektorsøk i innholdet. Søket bare tilgjengelig for biblioteksansatte, til forskjell fra Nasjonalbiblioteket.
“We present an extensible hybrid search solution using Elasticsearch, built on a multi-index architecture and allowing the integration of multiple embedding models. Our approach addresses the challenges of searching a vast and heterogeneous collection, using different chunking granularity and offering an alternative to reciprocal rank fusion.”
Make search on 30 million hours (!) of French language television programs from INA.
Transcription. Whisper Large V3 turbo Search: Elastic for lexical search.
Vectors but…. - Which embedding model? - Which chinking method? - Which chunking size? - How to optimize and compress? - How to combine lexical search and vector search? How to combine search score from two different search methods?
CamenBERT = French version of BERT :-D
3.2 TB of data yearly -> compress using quantification
Chunking techniques.
- Fixed size - Recursive chunking - Document based chunking - Semantic chunking (Supposedly best)
Reduce memory size from TB got GB size by using quantification.
Methods for reranking: - Reciprocal Rank Fusion (closed source Elastic) - Script score reranking. Compute score prior to aggregation.
Hybrid search: Split into lexical search and vector search.
Conclusions: Chunking important for search quality. To few users (only archivists) to evaluate.
miniCOIL: Sparse Neural Retrieval Done Right
Speaker: Evgeniya Sukhodolskaya
SEE_AGAIN? Veldig velforklart intro til en teknikk for embeddings.
Lexical search + Vector search = Hybrid search
Sparse neural retrieval: Understand semantic mening, but usually it doesn’t work :-( Dont work = Doesnt beat 50 year old algorithm BM25
So how to make Sparse nevral network useful (ie. beat basic algoritm BM25)
COIL = Contextual inverted list
Speaker: “Only simpel models work, the more complicated the less likely it will work”
miniCOIL released on Huggingface
When to use miniCOIL? - Ranking with semantic understanding
“In this talk, we present miniCOIL — our attempt to make a sparse neural retrieval model as it should be — combining the benefits of dense and lexical retrieval without propagating their drawbacks. We will share how to design and train a lightweight model that is performant on out-of-domain data and demonstrate its capabilities.”
Unconference sessions (no video?)
Optimizeing LLM prompts
Speaker: Vincent Koc DSpy Opik Optimizer SDK system for optimizing LLM prompts.
https://www.comet.com/docs/opik/ The Opik platform allows you to log, view and evaluate your LLM traces during both development and production. Using the platform and our LLM as a Judge evaluators, you can identify and fix issues in your LLM application.
Model Context Protocol
“the USB-C of AI apps”. The Model Context Protocol is an open standard, open-source framework introduced by Anthropic in November 2024 to standardize the way artificial intelligence models like large language models integrate and share data with external tools, systems, and data sources. Goal: Serving as a universal connector between language-model agents and external software
Source: Wikipedia: Model Context Protocol https://modelcontextprotocol.io/introduction
Video to text
Analyze content of videos to generate embeddings you can search in -> you can automate watching videos.
Vespa
Jupyter notebook for LLMs
YQL query language for LLMs?
Knit together LLMs and Solr. Query language https://docs.vespa.ai/en/reference/query-language-reference.html
The Navigator for Apache Solr - An Integrated Search & Discovery Environment (ISDE) solr.search-navigator.org
Automate UI testing
AI-improved Selenium uipilot.ai
LLMs with open source tools
Tips - Splitting: Rely on docuent structure (paragraphs etc) - Use vector for ranking and semantics, string search for matching.
Searchability for AI agents
What to do when our users are AI agents not, humans?
Are LLMs so powerful that search quality is obsolete? -> NO!
If you give LLMs more documents from lower down the SERP list, the quality of the AI summary gets worse -> counterintuitive. Really many -> worse than raw results.
How [not] to evaluate your RAG
Speaker: Roman Grebennikov
SEE_AGAIN? Tidvis morsomt om RAG-evaluering
How do you know if your RAG system is actually working? We’ll share a real-world case study on evaluating RAG in production—tackling messy data, chunking fails, and unexpected chatbot behavior—so you can measure quality with confidence.
User case: Building a support agent for support agents! 80% of questions are already answered on the FAQ.
Speaker: “The context length of a human is very short” :-D
R (in RAG) = search metrics
Translation of evaluation sets - Google Translate best (in 2025) for high resource languages - GPT o4 best for low resource languages.
Belebele a multiple-choice machine reading comprehension (MRC) dataset spanning 122 language variants.
Perplexity: Gemma3-27B
Speaker: Going from English to a lower resource language is like going 3 years behind in time
“LLM as a judge” = better than nothing If you have humans, go for them since we should
RAGAS Evaluation framework Contextual chunking FB Belebele framework Nixiesearch
Streamlining Search Quality: Search Relevance Workbench
Stavros Macrakis, Eric Pugh
SEE_AGAIN? Viste et bestemt verktøy fra OpenSearch for å sammenligne to søkerankinger side om side.
Search relevance methods - Query sets - Human judgements - Experiments - Search configerations
OpenSearch dashboards
“Implicit judgement” = check what user click on as evaluation of search quality
How to train a fast LLM for coding tasks
Speaker: Ivan Dolgov, JetBrains
Coding LLMs are now part of our daily work, making coding easier. In this talk, we share how we built an in-house LLM for AI code completion in JetBrains products, covering design choices, data preparation, training, and model’s evaluation.
Focal models = pupose build, single domain model. <10B
JetBrains Mellum purpose build 4B
Will release Mellum and MellomSDK on Huggingface soon.