Private Voice AI agent on Edge Devices [Workshop]

Michael Yuan - Second State

Voice agents, conversational and real-time AI systems capable of listening, reasoning, speaking and actions, are rapidly becoming one of the most practical forms of AI agents. However, most existing voice AI solutions are closed, cloud-dependent, very expensive, and difficult to customize.

In this workshop, we explore using WebAssembly to run lightweight and portable AI inference servers for the full voice pipeline — speech-to-text, LLM reasoning, and text-to-speechn — directly on your own machines, supporting a large variety of AI models, OSes, CPU, GPU, NPU and consumer hardware. The goal is to enable private, controllable, and portable voice AI deployment.

Participants will walk through the architecture of a modern voice AI agent and learn how each component can be powered by Wasm-based cross-platform AI inference.

Topics we’ll cover:

• The two AI inference approaches for voice agents: ASR → LLM → TTS vs End-to-End voice models
• Running local, open-source ASR, LLM, and TTS models using WebAssembly and WASI-NN
• Designing a practical, real-world voice-agent workflow
• Integrating MCP so the agent can take real actions via external tools

## By the end of this session, you will:

• Understand how a modern voice AI agent is architected
• Learn how WebAssembly enables lightweight, portable, secure AI inference on consumer-grade hardware
• Build your own customizable, privacy-preserving voice AI agent

Workshop Requirements

Please bring your own laptop. We will provide an open-source voice device that connects to the AI services on your laptop.