Introduction

I was talking with a friend about the library of babel (the book, and the online library). Inspired by my short experiments with RAG, I decided to see if I could make something similar: an infinite Wikipedia, where each keyword leads to a page, and pages have links between them. What is tricky is making sure that the wiki stays consistent.

Description

It’s a pretty simple service: every top-level URL is valid and will return a webpage with content generated by an LLM. The prompt is based on the title and a “context” extracted from other articles to keep the wiki consistent. The article will contain links that might have been instantiated or not.

The articles that are already generated are persisted in a Postgres database and will be served on subsequent requests. It is interesting because it allows you to see the “instantiated” library grow over time if you go to the root page. Each article also get summarised by ChatGPT, and that summary is stored alongside the article and indexed for the context building.

Source code

The source code can be found on the wikiofbabel Github page. Unlike other experiments, I do not offer a public-facing version as it would require significant time investment to make it “production-ready”. The risks of cost overruns are… significant!

It is really a PoC at the moment, but it was fun to make. More details on the README.