AI in the Browser: On-Device Machine Learning

Running AI Models Directly in the Browser

Artificial intelligence is increasingly running on your device instead of in the cloud. Running AI models in the browser has several advantages: your data never leaves your computer (great for privacy), the app works offline, and there's no server cost for running inference. Modern browsers are powerful enough to run impressive AI models directly.

The Main AI Runtimes for the Browser

Several frameworks let you run machine learning models in the browser. TensorFlow.js is Google's flagship library, supporting multiple backends: WebGL for graphics cards, WebGPU for modern GPUs, and plain WebAssembly for CPU fallback. It can convert models from Keras and TensorFlow SavedModel formats.

ONNX Runtime Web is a cross-platform runtime that supports models from PyTorch, TensorFlow, and Scikit-learn through the ONNX format. It also uses WebGPU and WebAssembly backends. MediaPipe provides pre-built solutions for face detection, hand tracking, pose estimation, and object detection that work in real time.

For natural language processing, Transformers.js lets you run Hugging Face models directly in the browser. And for the most ambitious projects, WebLLM can run large language models locally, though this requires significant memory.

What You Can Build

Computer vision is one of the most practical applications. You can remove backgrounds from images in real time, track body poses for fitness apps, detect faces and emotions, read text from images (OCR), or scan barcodes — all without sending images to a server.

For language, you can classify text, recognize named entities, translate between languages, summarize articles, and even run chatbots locally. Audio processing includes speech recognition (like Whisper), keyword spotting, and text-to-speech synthesis.

Performance Considerations

Running AI in the browser is getting faster, but there are limits. WebGPU is the most performant backend for modern hardware, especially for transformer models. WebGL is more widely supported but slower. Quantization — reducing the precision of model weights from 32-bit to 8-bit — can make models 4 times smaller and faster with minimal accuracy loss.

Practical model sizes are around 100-500MB. Larger models take too long to download and use too much memory. For complex tasks, a common pattern is to run a small model on the device and fall back to a server-side model for difficult cases.

Privacy and Architecture

The biggest advantage of on-device AI is privacy. Data never leaves the user's device, so you don't need GDPR or CCPA compliance for the AI processing. There are no server inference costs. And the app works offline. The trade-off is that model weights are visible to users (they can inspect the downloaded files), device capabilities vary widely, and battery life can be affected.

A common architecture is to use a hybrid approach: run a small, fast model on the device for most cases, and send only the difficult cases to a server-side model. This balances privacy, speed, and accuracy.