WebRTC vs. WebSocket for Real-Time Voice Apps on iOS: What Builders Need to Know

If you're building a real-time voice app on iOS in 2026, the single most consequential infrastructure decision you'll make is your audio transport layer. WebRTC and WebSocket are not interchangeable: on a clean wired connection the gap can feel academic, but the moment a user steps onto LTE in a busy airport or walks into an elevator, the difference between the two protocols becomes the difference between a voice assistant that feels alive and one that stutters, freezes, and loses turns ^[1]. The short answer: for conversational voice AI, choose WebRTC — and set up your iOS AVAudioSession correctly, or you'll lose most of the gain.

• Transport philosophy: WebRTC runs over UDP and is purpose-built for real-time media; WebSocket runs over TCP and inherits ordered-delivery guarantees that actively fight low-latency audio ^[2].
• Packet loss behavior: A 20 ms audio dropout from UDP is nearly imperceptible; a TCP retransmission stall that takes 100–200 ms is heard as dead silence that breaks conversational rhythm ^[3].
• Jitter resilience: WebRTC ships with NetEQ — an adaptive jitter buffer that dynamically stretches or compresses audio playback to absorb network variability without halting the stream ^[4].
• Opus codec integration: WebRTC's default codec, Opus, supports bitrates from 6 kbit/s to 510 kbit/s, frame sizes from 2.5 ms to 60 ms, and built-in packet loss concealment (PLC) that reconstructs lost frames under 120 ms without audible artifacts ^[5].
• iOS audio session: Apple's AVAudioSession must be configured to AVAudioSessionModeVoiceChat with the .playAndRecord category; using WebRTC's built-in audio configuration is the safest path to enabling system echo cancellation, noise reduction, and automatic gain control ^[6].
• OpenAI Realtime API: OpenAI's Realtime API now supports WebRTC as a first-class transport, which is why it's recommended for sub-second latency voice agents over the WebSocket fallback ^[7].

TL;DR: On clean networks the protocols feel similar; on the lossy mobile connections your users actually live on, WebRTC's UDP transport with NetEQ jitter buffering and Opus PLC makes it the only realistic choice for conversational voice AI on iOS.

---

Why TCP Is the Wrong Foundation for Voice

Head-of-Line Blocking: The Latency Killer You Don't See Coming

WebSocket is a thin bidirectional layer over TCP, and TCP guarantees ordered, reliable delivery ^[2]. For file transfers, this is exactly what you want. For real-time voice, it creates a failure mode called Head-of-Line (HoL) blocking: if packet #100 is lost, the TCP stack holds packets #101, #102, and #103 in its receive buffer until #100 is retransmitted before releasing any of them to the application ^[2]. With a 100 ms round-trip time, a single packet loss introduces at least 100 ms of delay. Under congestion, TCP's retransmission backoff can extend this to seconds ^[2].

In a chat UI, that's a brief freeze. In a voice agent, it produces audible clipping or dead-silence gaps that sound like a dropped call — destroying the conversational illusion your entire product depends on ^[8].

TCP also uses a sliding window for congestion control. When packets are lost, the window shrinks, throttling throughput right when consistent delivery is most needed. After the loss clears, the window doesn't snap back — it grows conservatively through slow start and congestion avoidance, taking multiple round trips to recover ^[3]. On high-latency mobile paths, each round trip takes longer, creating bursts of underdelivery followed by slow recovery ^[3].

Why WebSocket Works Fine for Signaling — Just Not Audio

None of this is a knock on WebSocket as a technology. It remains the right choice for:

• Signaling payloads (session description, ICE candidates, turn-taking messages)
• Structured data streaming (tool call results, transcripts, telemetry)
• Chat, notifications, and event delivery

The problem is precisely when developers reach for WebSocket to carry the audio stream itself. Every property of TCP that makes it great for structured data actively works against the real-time timing requirements of voice ^[3].

The latent.space teardown of the OpenAI Realtime API states it plainly: "WebSocket reconnection logic is very hard to implement robustly… TCP timeouts and connection events behave differently on different platforms" — and when you're building voice, you'd also need to implement echo cancellation, noise reduction, and automatic gain control yourself ^[7].

---

How WebRTC Solves Each of These Problems

UDP + DTLS: Accepting Loss for Consistent Timing

WebRTC's foundational design choice is to use UDP instead of TCP, accepting occasional packet loss in exchange for consistent delivery timing ^[3]. The insight is precisely calibrated to human auditory perception: "A missing 20 ms audio frame is nearly imperceptible to a listener; a 200 ms stall while TCP retransmits is not" ^[3]. WebRTC trades perfect reliability for consistent timing — and for voice, that is exactly the right tradeoff ^[3].

Security is not sacrificed: WebRTC wraps UDP in DTLS (Datagram Transport Layer Security) for encryption and SRTP (Secure Real-time Transport Protocol) for authenticated media delivery ^[8].

NetEQ: The Adaptive Jitter Buffer Inside WebRTC

WebRTC ships with NetEQ, a dynamic jitter buffer and error concealment algorithm built specifically for voice ^[4]. Unlike a static buffer, NetEQ dynamically adjusts its size to the observed network jitter — it can slightly accelerate playback during unvoiced speech segments to reduce accumulated delay when buffers grow, and decelerate or insert comfort noise when buffers run low ^[1].

In practice on a lossy mobile connection, NetEQ means the listener hears subtly adjusted pacing rather than a frozen stream. Over-the-top voice services that use WebRTC and Opus all depend on this framework to adapt to changing real-world network conditions ^[4].

Opus: The Codec That Was Built for Exactly This

WebRTC's default audio codec is Opus (IETF RFC 6176), and it's a remarkable engineering achievement for mobile voice:

• Variable bitrate from 6 kbit/s to 510 kbit/s, adapting to available bandwidth ^[5]
• Frame sizes from 2.5 ms to 60 ms — short frames for low latency, longer frames for efficiency in poor signal conditions ^[5]
• Sampling rates from 8 kHz to 48 kHz, covering everything from narrowband telephony to full hearing range ^[5]
• Packet Loss Concealment (PLC): Opus uses algorithmic reconstruction of plausible replacements for lost audio packets, combined with comfort noise, at losses under 120 ms — without audible artifacts [9]
• Forward Error Correction (FEC): redundant data embedded in the stream allows partial reconstruction of lost packets on the receiver side before PLC is needed [9]

Critically, Opus in WebRTC is "tightly coupled to WebRTC's bandwidth estimation and packet pacing (congestion control) logic," making the audio stream resilient to the wide range of real-world network behaviors that would cause a WebSocket connection to accumulate latency ^[7].

"The Opus audio codec used for WebRTC is tightly coupled to WebRTC's bandwidth estimation and packet pacing logic, making a WebRTC audio stream resilient to a wide range of real-world network behaviors that would cause a WebSocket connection to accumulate latency." — Latent Space, OpenAI Realtime API: The Missing Manual ^[7]

---

iOS-Specific Constraints: AVAudioSession and WebRTC

Building on iOS adds a layer of platform-specific audio session management that most web-centric WebRTC guides skip entirely. Getting this wrong silently destroys your latency gains.

Configuring AVAudioSession for Voice

Apple's AVAudioSession controls how your app interacts with the audio hardware, other apps, and system audio processing. For VoIP and voice AI apps, the correct configuration is ^[6]:

• Category: .playAndRecord — enables simultaneous microphone input and speaker output (full duplex)
• Mode: AVAudioSessionModeVoiceChat — activates system-supplied signal processing optimized for voice, including echo cancellation and noise reduction ^[6]
• Options: .allowBluetooth — enables AirPods and Bluetooth headsets

Apple's documentation states that AVAudioSessionModeVoiceChat "ensures that signals are optimized for voice through system-supplied signal processing" ^[6]. Failing to set this mode means you lose the hardware echo cancellation that makes handsfree usage possible.

The WebRTC iOS library (libwebrtc / GoogleWebRTC) ships its own RTCAudioSessionConfiguration that sets .playAndRecord + AVAudioSessionModeVoiceChat automatically when the peer connection is activated [10]. The practical advice: defer audio session configuration to WebRTC's own configuration rather than fighting it with manual setup — iOS developer Kostya Tsyvilko documented exactly this pitfall, finding that conflicting AVAudioSession configurations caused speaker-mode switching bugs that only WebRTC's internal configuration resolved [10].

Sample Rate and I/O Buffer Duration

WebRTC's iOS configuration sets the sample rate to match the audio stream format exactly. This matters because if the I/O unit's sample rate differs from the stream format, Core Audio performs on-device sample rate conversion — which adds latency and CPU cost [10]. On multi-core devices (every iPhone since the 4S), WebRTC selects the high-performance sample rate and I/O buffer duration path [10].

For wake-word detection apps that need the microphone active before a WebRTC session is established, the audio session must be initialized before the WebRTC peer connection — and you must be careful not to let WebRTC's session activation deactivate the wake-word engine's audio capture.

Barge-In on iOS: The Full-Duplex Requirement

Voice AI assistants need barge-in: the user can start speaking while the assistant is mid-sentence, and the assistant stops talking immediately. This requires true full-duplex audio — simultaneous capture and playback ^[8].

WebSocket-based implementations frequently discover a subtle problem here: TCP's stream ordering means that when the user barges in, the inflight audio data already queued for TCP delivery can't be cancelled — it must complete delivery before the interrupt signal arrives ^[8]. WebRTC, because it operates over UDP with no delivery queue, can simply stop sending audio frames. The transport layer doesn't fight the application's intent.

---

What Real-World Benchmarks Show on Mobile Networks

Performance on Clean vs. Lossy Links

Adamo's published comparison of WebRTC stacks on degraded networks (measured against LiveKit and Transitive) gives useful intuition for what to expect. On a clean link, glass-to-glass latency across WebRTC stacks clustered in the 83–100 ms range for the transport layer alone [11]. The stacks are close when the network cooperates.

Lossy mobile links reveal the structural differences: at 10% packet loss, WebRTC stacks (LiveKit) held median latency at ~183 ms — elevated but still in the intelligible-conversation band. A TCP-based transport in the same test climbed to 617 ms at 10% loss, and at 15% dropped the stream entirely [11].

For the builder of a voice AI assistant on iOS, 10% packet loss is not an edge case — it's a subway ride, a conference WiFi network, or an LTE handoff between towers. Your transport choice determines whether your app works in the real world your users actually inhabit.

"The gap between 'audio is flowing' and 'this feels like a real conversation' is enormous, and it's almost entirely a transport problem. WebSockets weren't designed for realtime media. WebRTC was." — LiveKit Engineering Blog ^[3]

The OpenAI Realtime API's Own Recommendation

OpenAI's Realtime API (the engine inside apps like Aurex) explicitly supports both WebRTC and WebSocket transports — and recommends WebRTC for lowest-latency voice applications ^[7]. The Realtime API's WebRTC path targets glass-to-glass latency of 500–1,200 ms on first turn, 300–600 ms on subsequent turns ^[7] — numbers that include model inference, not just transport. The WebSocket path carries additional latency risk on any mobile network with meaningful packet loss.

For a voice command layer where the stated latency target is sub-1s perceived voice-to-voice, WebRTC is not an optional nice-to-have. It's the baseline on which that target was set.

Practical Checklist for iOS WebRTC Voice Apps

Here's the implementation checklist that separates shipped apps from perpetual prototypes:

1. Use AVAudioSessionModeVoiceChat — don't fight WebRTC's built-in audio configuration.
2. Minimize playback buffers — smaller buffers reduce buffering latency; tune to the minimum that avoids underruns on your target device set.
3. Tune VAD (voice-activity detection) aggressiveness — false speech-end detections are the single biggest experience killer; they cause the model to start responding before the user has finished ^[7].
4. Co-locate your token broker with the OpenAI Realtime API region — each millisecond of broker RTT adds directly to first-turn latency.
5. Test under real mobile conditions — use a network link conditioner at 1–5% packet loss and 80 ms RTT before calling your latency numbers real.
6. Handle ICE/STUN/TURN fallback — WebRTC's connection negotiation handles firewall traversal automatically; make sure you have TURN server coverage for restrictive networks.
7. Don't use WebSocket for audio — use it only for signaling (session setup, turn-taking events, tool-call results) where ordered delivery is actually required.

---

Choosing Your Transport in the Context of a Real Product

The OpenAI Realtime API vs. STT→LLM→TTS pipeline latency comparison breaks down why the native speech-to-speech approach already cuts one major latency source. Transport is the remaining variable you control directly. On iOS, the combination of WebRTC + properly configured AVAudioSession + Opus PLC/FEC is the only path to consistent sub-second feel on the networks your users actually move through.

The hidden latency killers in AI voice assistants post covers what happens after the transport handshake — VAD tuning, tool call narration, and caching — but none of those optimizations matter if your transport choice is causing sawtooth 200–600 ms stalls on any lossy hop.

If you want to experience the full stack — WebRTC transport, Opus audio, AVAudioSession VoiceChat mode, and OpenAI Realtime API inference — all assembled into a working voice command layer for iOS, that's exactly what Aurex is. It's built for the power user who needs to fire off commands to Gmail, Slack, Linear, and Notion without ever unlocking their phone — and the transport architecture is why it actually feels fast in the real world.