Cross-tab WebGPU

How it works

Small language models can run directly in your browser with WebGPU. If several tabs open the same app, each tab can load its own copy of the model weights into GPU memory. Those copies add up and can exhaust GPU memory. This demo uses a Web Lock to choose one tab to run the model. BroadcastChannel lets the other tabs send prompts to it, avoiding duplicate model loads.

CHOOSE

The tabs use a Web Lock to choose one leader tab. Only that tab starts the WebGPU model worker.

SHARE

Other tabs send prompts to the leader through BroadcastChannel. The leader handles requests one at a time and streams answers back.

RECOVER

If the leader tab closes or refreshes, another tab takes over. Model files saved in private browser storage (OPFS) make reloading faster.

Prompts and model inference stay in your browser. See version 2 for a global network where browser instances share model work.