By VidaVolta in fastapi — Jun 16, 2023

Streaming with FastAPI

Streaming JSON with FastAPI and Python in HTTP using Server Sent Events (SSE)

Use Server Side Events to stream JSON in Python over HTTP

Modern users have modern sensibilities - and with that comes the desire for live results when an API request is made. Sometimes though, the request is too expensive for a complete result to be returned instantaneously. In this case,we can return chunks of the response as it becomes available. Take a look at ChatGPT in action for an example:

What Will We Build?

We will use Server Side Events to stream JSON data in a FastAPI application.

Server-Sent Events (SSE) Architecture

Server-Sent Events (SSE) is a technology that enables a server to push real-time updates to the client over a single, long-lived HTTP connection. It is a part of the HTML5 specification and is commonly used for server-to-client communication in web applications.

The obvious advantage of this approach is that multiple events are being sent over a single connection, which reduces overhead. We will also show that, with some nice syntactic sugar, it can be intuitive for the client to handle the event data.

Basic Text Streaming

We can return data in chunks using a StreamingResponse, which fortunately is a built in class in the FastAPI framework.

This will send data within a single HTTP connection from server to client. We can read the data quite simply using python requests:

Since we are reading 16 bytes at a time from the stream, we truncate the data mid stream. For some data types - this would not be problematic. However, in most cases we will be sending structured data that we will need to deserialize - and for those cases, we need to have some idea of the "beginning" and "end" of the data chunks that we are reading.

Streaming JSON Events

Imagine we wanted to send a stream of JSON events, something like this:

{"event_id": 0, "data": "some data", "is_last_event": false}
{"event_id": 1, "data": "some more data", "is_last_event": false}
{"event_id": 2, "data": "and more", "is_last_event": false}
{"event_id": 3, "data": "finally the end", "is_last_event": true}

Preferably, the client would be able to read the stream like a collection, with confidence that for each iteration, they will have a fully formed JSON object. This would give an intuitive abstraction for the client. Something like this:

for valid_json_object in api.get():
    # do something with the json object

Let's see if we can get there - first we need to adjust our server a little bit.

Each event will be a serialized JSON object terminated by a newline. We also change the media type to "application/x-ndjson", which stands for newline delimited JSON. I did some digging and this is the appropriate media type - so hopefully I saved you some trouble.

With these adjustments, the client can read the line separated stream with the "readline()" function, which will return raw bytes that can then be decoded and deserialized into a JSON object. With some other syntactic sugar we can get the clean interface we were looking for. Let's first look at an async example:

The synchronous example is even simpler:

When we use this code, we see a nice stream of JSON events being generated:

And just like that, we've successfully set up JSON streaming with SSE using FastAPI - happy streaming!

Conclusion

To see the code used in this article, visit the Github repo.