Case study In production

A voice AI that answers the phone for Austrian service businesses

The problem it solves

Every missed call is a lost lead. In a service business the phone rings while the owner is on a job or already home for the day, and a voicemail box books nothing.

What it does

DienstBot answers incoming calls in German. It qualifies the caller: what they need, whether the job fits the business and whether it lies in the service area. It books the appointment in the calendar, sends the caller an SMS confirmation and sends the business a summary email. When it cannot complete a call safely, it escalates to the owner with everything captured so far instead of improvising.

It is a multi-tenant SaaS with self-service onboarding. A business connects its Google account, sets up call forwarding, and the agent is live. Each business gets its own voice, prompt and vocabulary. Language, voice, scenario and previous call history are assembled from the caller ID, with no setup per call. Besides reception, the platform runs a personal-assistant mode and an automated onboarding mode.

How it is built

I built and operate DienstBot end to end: telephony, real-time speech dialogue, phase-based conversation control, billing and monitoring.

A call arrives over an Austrian SIP trunk, passes a SIP proxy (Asterisk) and is bridged into WebRTC on a self-hosted LiveKit server. A voice-agent worker runs the pipeline: voice activity detection, streaming speech-to-text, a tool-calling language model and neural text-to-speech. The stages overlap, transcription while the caller still speaks and synthesis from the first clause, for a response budget of one second over the public phone network. End of turn is predicted by the model, not by a fixed silence timer.

The conversation runs in phases with explicit transitions and fallback logic. The AI has no direct access to the calendar, the database or even the current time. Every concrete fact it states, a free slot or a confirmation, comes from an explicit function call, so inventing data is structurally impossible. The behaviour is pinned by an adversarial end-to-end test suite covering conversation and onboarding scenarios, including prompt-injection and jailbreak attempts.

Each stage runs with an EU provider and fails over to an independent EU backup during a live call. Deploys let running calls finish before a worker exits. Provider balance and live errors raise alerts by messenger and email, and every change passes a CI gate from lint and tests to a secret and CVE scan. Data sits in PostgreSQL with row-level tenant isolation; the calendar is Google Calendar, billing runs through Stripe.

Privacy by design

Audio is never written to disk. It is processed in memory, and there is no storage path for it by design. The system is self-hosted in EU data centres, with EU speech and language providers. OAuth tokens are encrypted with AES-256, webhooks authenticated with HMAC, and database roles have the least privilege they need. The business receives the appointment and a summary, not a recording.

What it shows about how I build

DienstBot is where my rules for client work come from. The AI decides only what it must decide, and every fact comes from a system, not from the model. When it is unsure, it hands over to a person with the context. Privacy is a property of the architecture, not a paragraph in the terms. And a system is finished only when it runs in production, with monitoring and failover.

See it live: dienstbot.at

Architecture at a glance

1

Call connects

PSTN caller
SIP
EU SIP trunk
INVITE
SIP proxy
WebRTC
Self-hosted SFU
dispatch
2

Voice Agent · real-time worker

audio
VAD
STT
LLM (tool-calling)
TTS
audio

Speech-to-text

Streaming STT, EU region

Reasoning

Tool-calling LLM, EU region

Text-to-speech

Neural TTS, EU region

Each leg: automatic in-call EU failover

3

Connected systems

CalendarRead / book live
Database · cacheMulti-tenant, encrypted
Payments · messagingBilling, onboarding, alerts

Technology stack

TelephonyEU SIP trunk → SIP-REGISTER proxy → SIP↔WebRTC bridge
Real-time voiceSelf-hosted SFU + real-time agent framework (async Python worker)
Speech-to-textEU-based streaming STT provider — streaming + acoustic/semantic end-of-turn
ReasoningEU-based tool-calling LLM
Text-to-speechEU-based neural TTS provider
Turn-takingVAD on-server + model-native end-of-turn
Backend / APIAsync Python API framework
DataRelational database with row-level security · in-memory cache
IntegrationsCalendar · payments · messaging channels
ObservabilityLLM tracing · metrics & dashboards · error tracking
Infra / deliveryContainerised deploys · EU cloud hosting · CI/CD

Engineering highlights

Real-time pipeline

  • Overlapped STT ▸ LLM ▸ TTS — transcription during speech, synthesis on the first clause
  • Sub-second time-to-first-byte, streaming end to end
  • VAD + model-native end-of-turn predicts "finished speaking" instead of relying on a fixed silence timer

Grounded, not hallucinating

  • The LLM is structurally forced to call a tool for any concrete fact — time, confirmation, availability
  • Behaviour pinned by an adversarial end-to-end suite — 20 conversation + 6 onboarding scenarios, including prompt-injection / jailbreak guards

Resilience

  • Per-leg automatic in-call provider failover — STT, LLM and TTS each fail over to an independent EU backup
  • Zero-downtime worker deploys — drain in flight, finish live calls, exit
  • Self-monitoring — provider balance and live error alerts over Telegram and email

Privacy by design

  • Audio is never written to disk — processed in RAM, no storage path by design
  • Fully self-hosted — EU data centers, EU data residency

Security

  • Row-level tenant isolation in the database · OAuth tokens AES-256-encrypted at rest
  • Webhook HMAC auth · rate limiting · least-privilege DB roles

Delivery

  • Single-command containerised deploys
  • CI gate on every change: lint → type-check → tests → secret & CVE scan → build → migrate → deploy
  • Self-hosted, EU-region observability stack

Describe what slows you down.

You get back what to automate first — and what not to.

Have a defined project already? Send the brief.