In-browser LLM on Chrome Canary

The nightly version of Google Chrome ships with an on-device LLM. The model it runs is Gemini Nano. Some documentation about this can be found here.

It's not surprising to be honest, that tiny LLMs make their way into browsers. We have already heard that Mozilla is experimenting with automatically generating image alt tags with on-device AI. I'm sure we will see similar initiatives come to fruition in the upcoming months. What I do hope, though, is that some of these improvements will also make their way into web standards. That will probably take a bit more time.