How I built real-time AI streaming in Flutter with Socket.IO

The Problem

ORTH is an AI-powered agronomy assistant. Farmers ask crop questions and expect advice that appears token by token — the same progressive feel as a ChatGPT reply — not a spinner that resolves into a finished block of text.

HTTP polling cannot deliver that experience well. You either wait for the full completion before painting anything, or you poll on a timer and accept lag, wasted requests, and awkward chunk boundaries. In the field, where connectivity is uneven, that lag feels like the app is broken.

We needed a push-based channel: the NestJS backend emits tokens as the model produces them, and the Flutter client paints each chunk as it arrives.

Why Socket.IO over raw WebSockets

Raw WebSockets give you a socket and little else. Socket.IO sits on top with reconnect, event names, and transport fallbacks that matter on mobile networks. ORTH already needed a persistent connection for live AI output, so Socket.IO was the pragmatic choice: named events for stream chunks, automatic reconnection when the radio drops, and less custom glue than a bare WebSocket client.

The backend still owns the model call. The mobile app only listens for chunk events and renders them. That split keeps the Flutter layer focused on UI and lifecycle instead of reimplementing streaming protocol details.

Wiring the socket to a StreamController

The client pattern is deliberately thin. On connect, subscribe to the stream event. Each payload is a string chunk. Push those chunks into a StreamController<String>. A StreamBuilder (or an equivalent listener in the Cubit) appends to the visible reply so the UI types forward in real time.

final controller = StreamController<String>();

socket.on('ai:chunk', (data) {
  controller.add(data as String);
});

// StreamBuilder listens to controller.stream and appends text.

Closing the controller when the stream ends or the user leaves the screen is part of the contract — not an afterthought. Leaving it open leaks subscriptions across navigations and can keep the socket handler alive after the widget tree is gone.

State management with Cubit + Freezed

ORTH isolates each feature into its own Cubit with Freezed union states (initial, loading, success, error). Streaming fits that model cleanly: enter loading when the user sends a question, emit intermediate success-style updates as the accumulated text grows, then settle on a final success (or error) when the server signals completion.

Keeping stream chunks out of ad-hoc setState calls matters. The Cubit owns the accumulating string and the socket subscription. The UI stays thin: it maps states to widgets and lets Freezed exhaustiveness catch missing cases when new AI features land.

Clean Architecture layering stays intact. The data layer talks to Socket.IO. The domain exposes a stream or use case. The presentation Cubit consumes that stream and never imports socket types directly.

Handling interruptions and lifecycle

Real devices interrupt streams. The socket can drop mid-reply. The user can pop the route while tokens are still arriving. Another feature — audio playback of AI responses — already taught us that lifecycle bugs show up as corrupted state and leaked players; text streaming has the same shape of risk.

Practical rules we followed:

• Cancel the socket listener and close the StreamController in the Cubit close path and on route dispose.
• On reconnect, do not assume the previous stream is still valid — treat a dropped connection as an error or a resumable session only if the backend explicitly supports it.
• Disable send while a stream is active so overlapping replies cannot race the same controller.
• Surface offline-aware errors with retry, matching the rest of the app’s error UX.

Results

The progressive StreamBuilder path is what users feel: advice appears as it is generated, which makes long agronomic answers usable in the field instead of a blank wait. Combined with Cubit + Freezed and strict cleanup, the streaming path stayed testable and stable enough to ship — ORTH is live on the App Store and Google Play with real-time AI responses as a core feature.