Recently, my team encountered a performance issue with our Python service that uses FastAPI, gunicorn, uvicorn, and the OpenAI API. The service works fine in development, but when we load-tested it in our staging environment …
Blog
Automating Groundedness Evaluation in RAG Applications
As a software engineer working on a Retrieval-Augmented Generation (RAG) application, one of my primary concerns is the accuracy and reliability of the answers it provides. A RAG application works by first searching for content …
GenAI App: To Stream or Not to Stream?
Generative AI applications, particularly those with LLM-powered chat interfaces like ChatGPT, stream responses. We type our query, and the answer gradually generates, word by word. This streaming approach has become so commonplace that it's almost …
Thoughts on Personalising Retrieval-Augmented Generation
I've been thinking about personalising Retrieval-Augmented Generation (RAG) lately. This is partly because my team is now building a chatbot that answers questions using a RAG setup. It's a continuation of my work with a …
Mentoring toolkit
I had the privilege of mentoring two junior software engineers over the past year. It was my first time doing something like this, so there was a lot to learn! To be honest, I don't …
Notes from AIAI London 2024
I attended the AI Accelerator Institute's Generative AI Summit that happened yesterday in London - AIAI London in short. Here are my notes. These are by no means exhaustive. Just a bunch of things I found …
Thoughts about AGI
There's a lot of buzz these days around AI, and AGI in particular. It seems like everyone is busy sharing their thoughts on AGI on the internet. I would love to do the same on …
Starting a tech book club!
I'm kickstarting a technical book club! It's going to be based around a mailing list with one thread per week discussing the scope covered that week. I have some ideas for what to read, but …
A non-review of half of Designing Data-Intensive Applications
I read the first seven chapters of "Designing Data-Intensive Applications" before deciding to stop. It's about half of this hefty book. In the spirit of learning in public through this blog, I'll reflect on what …
Anthropic is onto something interesting with Artifacts
A few weeks ago, Anthropic released a new feature to their Claude chatbot — Artifacts. It was announced together with Claude 3.5 Sonnet. You can find more details in the section about the feature. It's …