A talking LLM that runs on your own computer without needing the internet.
Copy the command below to clone the repository to your machine.
Original article: https://blog.duy-huynh.com/build-your-own-voice-assistant-and-run-it-locally/
Updated May 2025: Now using Chatterbox TTS, a state-of-the-art open-source TTS model from Resemble AI!
The original implementation using Bark has been preserved in the
archive-2025-05-29branch for reference.
After my latest post about how to build your own RAG and run it locally. Today, we're taking it a step further by not only implementing the conversational abilities of large language models but also adding listening and speaking capabilities. The idea is straightforward: we are going to create a voice assistant reminiscent of Jarvis or Friday from the iconic Iron Man movies, which can operate offline on your computer.
New Features with ChatterBox:
First, you should set up a virtual Python environment. You have several options for this, including pyenv, virtualenv, poetry, and others that serve a similar purpose. Personally, I'll use Poetry for this tutorial due to my personal preferences. Here are several crucial libraries you'll need to install:
For a detailed list of dependencies, refer to the link here.
The most critical component here is the Large Language Model (LLM) backend. By default, we use Ollama for running LLMs locally. Alternatively, you can use MiniMax as a cloud LLM provider for higher-quality responses without local GPU requirements. If Ollama is new to you, I recommend checking out my previous article on offline RAG: "Build Your Own RAG and Run It Locally: Langchain + Ollama + Streamlit". Basically, you just need to download the Ollama application, pull your preferred model, and run it.
Okay, if everything has been set up, let's proceed to the next step. Below is the overall architecture of our application, which fundamentally comprises 3 main components:
The workflow is straightforward: record speech, transcribe to text, generate a response using an LLM, and vocalize the response using ChatterBox.
flowchart TD
A[🎤 User Speech Input] --> B[Speech Recognition<br/>OpenAI Whisper]
B --> C[📝 Text Transcription]
C --> D[Conversational Chain<br/>Langchain + Ollama / MiniMax<br/>Gemma3 / Llama-4 / MiniMax-M2.7]
D --> E[🤖 Generated Response]
E --> F[Speech Synthesizer<br/>Chatterbox TTS]
F --> G[🔊 Audio Output]
G --> H[👤 User Hears Response]
style A fill:#e1f5fe
style B fill:#f3e5f5
style C fill:#e8f5e8
style D fill:#fff3e0
style E fill:#e8f5e8
style F fill:#f3e5f5
style G fill:#e1f5fe
style H fill:#fce4ec
⚠️ Important: We strongly recommend using uv for dependency management instead of pip with requirements.txt. The requirements.txt file was generated by uv pip freeze and contains pinned versions that may not install correctly across different systems.
# Install uv if you haven't already
curl -LsSf https://astral.sh/uv/install.sh | sh
# or on macOS: brew install uv
# or on Windows: powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"
# Clone the repository
git clone https://github.com/vndee/local-talking-llm.git
cd local-talking-llm
# Install dependencies using uv (recommended)
uv sync
# Activate the virtual environment
source .venv/bin/activate # On Windows: .venv\Scripts\activate
# Download NLTK data (for sentence tokenization)
python -c "import nltk; nltk.download('punkt_tab')"
If you prefer to use pip, install directly from pyproject.toml:
# Clone the repository
git clone https://github.com/vndee/local-talking-llm.git
cd local-talking-llm
# Create virtual environment
python -m venv venv
source venv/bin/activate # On Windows: venv\Scripts\activate
# Install from pyproject.toml (NOT requirements.txt)
pip install -e .
# Download NLTK data
python -c "import nltk; nltk.download('punkt')"
# Install and start Ollama
# Follow instructions at https://ollama.ai
ollama pull gemma3 # or any other model you prefer
If you don't have a local GPU or prefer higher-quality cloud models, you can use MiniMax as the LLM backend:
export MINIMAX_API_KEY="your-api-key-here"
No Ollama installation is needed when using MiniMax — the LLM runs in the cloud while TTS and STT still run locally.
python app.py
Record a 10-30 second audio sample of the voice you want to clone, then:
python app.py --voice path/to/voice_sample.wav
# Adjust emotion and pacing
python app.py --exaggeration 0.7 --cfg-weight 0.3
# Use a different LLM model
python app.py --model codellama
# Save generated voice samples
python app.py --save-voice
# Use MiniMax as the LLM provider (requires MINIMAX_API_KEY env var)
python app.py --provider minimax
# Use a specific MiniMax model with custom temperature
python app.py --provider minimax --model MiniMax-M2.7 --temperature 0.8
# Pass API key directly
python app.py --provider minimax --api-key your-api-key-here
# Combine with voice cloning and emotion control
python app.py --provider minimax --voice path/to/voice.wav --exaggeration 0.7
--provider: LLM provider (ollama or minimax, default: ollama)--api-key: API key for cloud LLM providers (or use MINIMAX_API_KEY env var)--temperature: LLM temperature (0.0-1.0, default: 0.7)--voice: Path to audio file for voice cloning--exaggeration: Emotion intensity (0.0-1.0, default: 0.5)--cfg-weight: Controls pacing and delivery style (0.0-1.0, default: 0.5)--model: Ollama model to use (default: llama2)--save-voice: Save generated audio responses to voices/ directoryThe new TextToSpeechService leverages ChatterBox's advanced features:
from chatterbox.tts import ChatterboxTTS
class TextToSpeechService:
def __init__(self, device: str = "cuda" if torch.cuda.is_available() else "cpu"):
self.device = device
self.model = ChatterboxTTS.from_pretrained(device=device)
self.sample_rate = self.model.sr
def synthesize(self, text: str, audio_prompt_path: str = None,
exaggeration: float = 0.5, cfg_weight: float = 0.5):
wav = self.model.generate(
text,
audio_prompt_path=audio_prompt_path,
exaggeration=exaggeration,
cfg_weight=cfg_weight
)
return self.sample_rate, wav.squeeze().cpu().numpy()
Key improvements over the previous Bark implementation:
exaggeration parameterThe app now includes automatic emotion detection to make responses more expressive:
def analyze_emotion(text: str) -> float:
emotional_keywords = ['amazing', 'terrible', 'love', 'hate', 'excited',
'sad', 'happy', 'angry', '!', '?!']
emotion_score = 0.5
for keyword in emotional_keywords:
if keyword in text.lower():
emotion_score += 0.1
return min(0.9, max(0.3, emotion_score))
Voice Cloning:
Emotion Control:
exaggeration=0.5, cfg_weight=0.5exaggeration=0.7+, cfg_weight=0.3exaggeration=0.3, cfg_weight=0.7Performance:
For those aiming to elevate this application to a production-ready status, consider:
Performance Optimization:
Enhanced Features:
Voice Database:
API Service:
Problem: requirements.txt installation fails with errors like:
ERROR: Could not find a version that satisfies the requirement cPython==0.0.6ModuleNotFoundError: No module named 'distutils'Solution: The requirements.txt file was generated by uv pip freeze and contains exact versions that may not work across different systems. Use one of these alternatives:
Use uv (Recommended):
uv sync
Use pip with pyproject.toml:
pip install -e .
Manual installation of core packages:
pip install chatterbox-tts langchain-ollama openai-whisper sounddevice rich nltk
With the integration of ChatterBox, we've significantly enhanced our local voice assistant. The addition of voice cloning and emotion control opens up new possibilities for creating personalized and expressive AI assistants. Whether you're building a personal Jarvis, creating content, or developing voice-enabled applications, this updated stack provides a powerful foundation.
The combination of Whisper's robust speech recognition, Ollama's flexible LLM serving, and ChatterBox's advanced TTS capabilities creates a fully-featured voice assistant that runs entirely offline. No cloud services, no API keys, just pure local AI power!