DaFoster (Django)

Reliable rendering of web pages that view concurrently modified data

2021-03-09T00:00:00+00:00

Any time a backend Django or Rails function calculates something complex from the database to send to the frontend as part of a view, there is a chance the database will be modified concurrently before the view is displayed to the visitor, causing the site visitor to see outdated information.

In many cases displaying stale information is fine. After all refreshing the page will bring the latest information to a visitor who is temporarily seeing stale information. But in other cases showing stale information can be a major problem:

For example if the stale data is some editable text that is shared with other site visitors and the new visitor tries to edit it, they’ll clobber the last set of changes made by the previous visitor! Data loss is not OK.
Or perhaps the stale information controls whether the visitor is allowed to access a piece of content that they are actively interested in. For example a user may be requesting to join a chat room and is waiting to be let in. Presenting stale access information saying that the visitor is locked out when they really aren’t will cause them to wait forever and eventually give up! Abandonment is not OK.

How can we avoid serving stale data to the frontend to avoid such problems? This article presents one technique to both avoid serving stale data from the backend in the first place and repair the stale data quickly after page load if necessary.

Although this article is written from the perspective of a Django developer, the concepts apply to any web page that is rendered dynamically based on data from a database or other data source that is subject to concurrent modification.

The Problem

Let’s start by reviewing the problem: Consider a web page that is templated by a backend server such as Django or Rails which uses information from a database. If visitor A reads from the database while templating such a page, but visitor B then alters the part of the database that A read, A will see outdated information when her page finishes loading:

We want to ensure that A sees the freshest data either immediately or as quickly as possible after initial page load.

Designing a Solution

How can visitor A detect the change made by visitor B? One idea is to double-check the freshness of information from the database just before returning information to the frontend:

Although such a strategy does narrow the time window in which a race condition could occur, it is still possible that a concurrent modification is made while the templated page is in transit to the frontend:

Another idea is for the frontend to explicitly double-check the freshness of the information it receives from the backend shortly after it initially renders. If the frontend happens to be displaying outdated information it can repair its state immediately with the new information from the backend:

This strategy is almost perfect (for getting the most up-to-date state reliably to the frontend). However there’s still a narrow time window in which visitor B can make a modification after visitor A’s final request.

What now? To detect these types of concurrent changes it is necessary for the backend to push any new changes to the frontend in real-time (via WebSocket) and the frontend needs to be prepared to receive these kinds of updates to its displayed state continuously:

Note: My last article explains in depth how to setup such real-time updates over WebSocket in Django using Channels. In Rails you’d use ActionCable.

Additionally the backend must be prepared to push even those changes that occur between when the backend templates a view and before the frontend has connected a WebSocket:

Note: Neither Django’s Channels nor Rails' ActionCable provide out-of-the box support for observing events that occur during this critical time period. In the Implementation section below I outline a technique of tracking the “timepoint” an event was generated at so that it can be buffered and delivered later reliably.

Great! We’ve got a bulletproof design to quickly observe an accurate up-to-date state on the frontend. But it does seem to be rather complex…

Alternative Design: No backend templating

It is possible to remove the initial logic that templates information initially on the backend and rely entirely on the WebSocket established after frontend page load to receive both the initial page state and any updates to that state in real-time. Indeed this is the approach taken by many Single Page Applications (SPAs):

However removing backend templating entirely will cause the initial page load to contain no content (beyond an annoying spinner) and raise your time-to-first-render. Content won’t be delivered until after JavaScript is loaded - which can take a while on mobile devices - and after a socket connection is established, disproportionately slowing the browsing experience of any site visitor who isn’t on a high-end device with a fast low-latency internet connection.

I’d like to reach users who are on mobile devices, in rural areas, and from faraway countries where minimizing latency and bandwidth is important to avoid an unacceptable user experience. So we’re back to the more complex design with both backend templating and real-time updates…

Implementation (using Django)

Let’s actually sketch our design for reliable rendering in code!

I’ll use the example of a User Home Page that lists a set of associated Chat Rooms from my last article.

Please review the User, ChatRoom, and ChatRoomJoinRequest models.

Let’s define a cacheable calculation for the set of available rooms given a particular user:

from typing import TypedDict

IntStr = str  # parseable as an int

class RoomInfo(TypedDict):
    id: IntStr
    title: str
    pending: bool

def calculate_chat_room_list(user: User) -> List[RoomInfo]:
    joined_rooms = [
        dict(
            id=str(room.id),
            title=room.title,
            pending=False,
        )
        for room in user.joined_room_set.all()
    ]
    pending_rooms = [
        dict(
            id=str(join_request.room.id),
            title=join_request.room.title,
            pending=True,
        )
        for join_request in ChatRoomJoinRequest.filter(user=user, status='pending')
    ]
    return list(sorted(
        joined_rooms + pending_rooms,
        key=lambda room_info: (room_info['title'], int(room_info['id']))
    ))

Then in the backend view function that templates the page’s HTML, we’ll prepopulate the latest data in that HTML:

# chat_project/chat/views.py

@login_required
def user_home_page(request: HttpRequest) -> HttpResponse:
    ...

    last_updated = UserConsumer.create_timepoint()  # capture
    chat_room_list = calculate_chat_room_list(request.user)  # capture

    return render(request, 'chat/user_home_page.html', dict(
        user_id=str(request.user.id),
        last_updated=last_updated,
        chat_room_list=chat_room_list,
    ))

Then on the frontend JavaScript will wake up and establish a WebSocket connection to the backend, to see if any concurrent changes were made to the room list since the version in the HTML was calculated:

# chat_project/chat/static/chat/user_home_page.js

/*public*/ function setupUserHomePage() {
    const userId = JSON.parse(
        document.querySelector('#user-id').innerText);
    const lastUpdated = JSON.parse(
        document.querySelector('#last-updated').innerText);
    setupUserSocket(userId, lastUpdated);
}

Whenever a user’s room join request is updated, the backend must notify the frontend appropriately through the socket by creating, buffering, and forwarding an event stamped with a new timepoint¹:

# chat_project/chat/models.py

class ChatRoomJoinRequest(models.Model):
    ...  # fields

    def save(self, *args, **kwargs) -> None: ...

    def notify_did_alter(self,
            action: Literal['create', 'admit', 'deny']) -> None:
        ...

        from chat.consumers import UserConsumer  # avoid circular import
        UserConsumer.notify_did_alter_join_request(self, action)

# chat_project/chat/consumers.py

class UserConsumer(WebsocketConsumer):
    ...

    # Send message to group
    @classmethod
    def notify_did_alter_join_request(cls,
            join_request: ChatRoomJoinRequest,
            action: Literal['create', 'admit', 'deny']) -> None:
        update_timepoint = cls.create_timepoint()  # capture
        async_to_sync(get_channel_layer().group_send)(
            cls.user_group_for(join_request.user_id),
            {
                'timepoint': update_timepoint,
                'type': 'did_alter_join_request',
                'kwargs': { ... }
            }
        )

When the frontend first connects a socket to the backend, the backend will attempt to immediately forward any events that were buffered since the timepoint passed in the connect call:

# chat_project/chat/static/chat/sockets.js

/*public*/ function setupUserSocket(userId, lastUpdated) {
    new WebSocketClient(
        '/ws/user/' + userId + '/' + lastUpdated, {
            onmessage: function(e) {
                didReceiveUserSocketMessage(JSON.parse(e.data));
            },
        }
    );
}

function didReceiveUserSocketMessage(data) {
    var type = data['type'];
    var kwargs = data['kwargs'];
    if (type === 'did_alter_join_request') {
        console.log(
            'User socket: Did alter join request: ' + 
            kwargs['action'] + ' ' + kwargs['id']);
        if (kwargs['action'] === 'create') {
            // TODO: Create a new row on User Home Page in the Rooms section
        } else if (kwargs['action'] === 'admit') {
            // TODO: Promote the row in the Rooms section to be a full room,
            //       removing the "Waiting to be admitted" banner
        } else if (kwargs['action'] === 'deny') {
            // TODO: Remove the row in the Rooms section
        }
    } else {
        console.warn(
            'User socket: Did receive unexpected message type: ' + type);
    }
}

That’s it! Hopefully you found this technique useful for reliably rendering the freshest information to visitors, even in the presence of concurrent modifications.

In Django⁶ :

Real-time updates in Django with WebSockets, Channels, and pub-sub
Building web apps with Vue and Django - The Ultimate Guide - Planning how to integrate Vue with a new or existing Django app

A timepoint represents a point in time relative to a shared clock. If you have a single shared Redis instance which you’re already using to forward socket messages, as is the case with Django’s default Channels configuration, it is convenient to use Redis’s TIME command to generate a timepoint, perhaps as part of a stored procedure.↩

Real-time updates in Django with WebSockets, Channels, and pub-sub

2021-03-02T00:00:00+00:00

It’s easy to build a simple chat server in Channels with real-time updates¹ but it’s a bit more complicated to design a system for a more realistic (and complex) data model that has real-time updates. Here, I will show a publish-subscribe (or “pub-sub”) pattern using WebSockets and Channels that can be used by your frontend to watch elements of your backend data model for updates in real-time.

Topics and Messages
Simple Example: Public Chat Rooms
Extended Example: Private Chat Rooms
Implementation in Django
Challenge: Lost events

Topics and Messages

In a publish-subscribe design, frontend pages subscribe to topics that are managed by the backend, by connecting a WebSocket to a wss:// backend endpoint corresponding to the topic.

The backend sends messages to a topic when taking certain actions and those messages are then forwarded on by the topic to all of its subscribers.

Simple Example: Public Chat Rooms

In the context of a simple chat app, there exist Chat Rooms and Chat Messages as models in the database. There also exists a frontend Chat Room Page which is populated with the initial set of Chat Messages on page load. This page wants to observe any new Chat Messages created related to the Chat Room it’s displaying.

To observe newly created Chat Messages, the Chat Room Page subscribes to a Chat Room Topic by immediately opening a WebSocket to the topic endpoint as soon as its JavaScript starts running. That topic receives messages like

{
    'type': 'chat_message_created',
    'kwargs': {
        'id': '42',
        'message': 'Hello world!'
    }
}

which the Chat Room Page then also receives and uses to update its local copy of Chat Messages on the page.

Extended Example: Private Chat Rooms

Let’s say you wanted to extend your chat app such that each chat room is owned by a particular user and that other users must get permission from that owning user before they are allowed to join the room:

Now there are Users, each of which has their own User Home Page which lists the set of Chat Rooms they are already members of and all Chat Rooms that they have submitted a request to join. From this page a user can navigate to a chat room or submit a request to join a new room, perhaps by providing some kind of join code associated with the room.

Additionally the Chat Room Page is extended so that if the person viewing the Chat Room is also the owner of the Chat Room, the page will display not just the list of users already in the room but also a list of users that have created a join request and are waiting to join the room. The owner can then choose to admit a waiting user to the room or to deny a waiting user from entering.

Now things are a bit more complicated:

There is now an additional User Topic that the User Home Page must subscribe to.
There is a new Chat Room Join Request model to track which rooms a user has requested to join. Creating a new request needs to be observed by the Chat Room Page in real-time so that it can display the new waiting user to the room owner. When the owner chooses to admit or deny the join request, that action needs to be observed in real-time on the User Home Page of the requesting user.
A single model (Chat Room Join Request) can have various actions applied to it (create, admit, deny) that are observed by multiple types of pages.
Topics need to authenticate its subscribers:
- In particular the Chat Room Topic will only allow members of the Chat Room to subscribe to it.
- And the User Topic will only allow its related User to subscribe to it.
Some Topics choose to forward certain types of messages to only a subset of their subscribers. In particular the Chat Room Topic should forward Chat Room Join Request messages to the room owner only and not to the other room members.

Implementation in Django

Models, Consumers, and Routes

Here’s a sketch of what an implementation of the above extended chat app might look like in Django with the Channels library.

First we need some models:

# django/contrib/auth/models.py

class User(models.Model): ...  # built-in to Django

# chat_project/chat/models.py
from django.db import models

class ChatRoom(models.Model):
    owner = models.ForeignKey(
        'User', related_name='owned_room_set', on_delete=models.PROTECT)
    title = models.CharField(max_length=50)
    member_set = models.ManyToManyField(
        'User', related_name='joined_room_set', blank=True)

class ChatMessage(models.Model):
    room = models.ForeignKey(
        'ChatRoom', related_name='message_set', on_delete=models.CASCADE)
    sender = models.ForeignKey(
        'User', related_name='message_set' on_delete=models.PROTECT)
    posted = models.DateTimeField(auto_now_add=True)
    message = models.CharField(max_length=200)

class ChatRoomJoinRequest(models.Model):
    user = models.ForeignKey(
        'User', related_name='join_request_set', on_delete=models.CASCADE)
    room = models.ForeignKey(
        'ChatRoom', related_name='join_request_set', on_delete=models.CASCADE)

Then we need some topics. In Django Channels the backend object that manages a topic is called a “consumer”, so let’s create some consumers:

# chat_project/main/routing.py
import chat.routing
from channels.auth import AuthMiddlewareStack
from channels.routing import ProtocolTypeRouter, URLRouter
from django.conf.urls import url

application = ProtocolTypeRouter({
    # (http->django views is added by default)
    'websocket': AuthMiddlewareStack(
        URLRouter(
            chat.routing.websocket_urlpatterns +
            # (add websocket_urlpatterns from more apps here)
            []
        )
    ),
})

# chat_project/chat/routing.py
from django.conf.urls import url

from . import consumers

websocket_urlpatterns = [
    url(r'^ws/user/(?P<user_id>\d+)$', consumers.UserConsumer),
    url(r'^ws/room/(?P<chat_room_id>\d+)$', consumers.ChatRoomConsumer),
]

# chat_project/chat/consumers.py
from channels.generic.websocket import WebsocketConsumer

class UserConsumer(WebsocketConsumer): ...

class ChatRoomConsumer(WebsocketConsumer): ...

Connect Frontend to Backend

Let’s subscribe to a topic from the frontend:

// chat_project/chat/static/chat/room.js

/*public*/ function setupChatRoomPage() {
    const roomId = ...;

    setupChatRoomSocket(roomId);  // create early to avoid missing events

    ...
}

// chat_project/chat/static/chat/sockets.js

/*public*/ function setupChatRoomSocket(roomId) {
    new WebSocketClient(
        '/ws/room/' + roomId, {
            onmessage: function(e) {
                didReceiveChatRoomSocketMessage(JSON.parse(e.data));
            },
        }
    );
}

function didReceiveChatRoomSocketMessage(data) {
    ...
}

WebSocketClient opens and manages a WebSocket, transparently handling connect failures and temporary disconnections, retrying with exponential backoff and jitter. Please substitute your favorite WebSocket management library here.

Now let’s fill out the consumer on the backend so that it can accept a connection from the frontend:

# chat_project/chat/consumers.py
from asgiref.sync import async_to_sync
from channels.generic.websocket import WebsocketConsumer
from channels.layers import get_channel_layer
from chat.models import ChatRoom
from django.contrib.auth.models import User
import json

class ChatRoomConsumer(WebsocketConsumer):
    def __init__(self, *args, **kwargs) -> None:
        super().__init__(*args, **kwargs)
        self.user_group = None

    def connect(self) -> None:
        # Authenticate
        if self.user.is_anonymous:
            self.close_during_connect(code=4401)  # Unauthorized
            return

        room_id = self.scope['url_route']['kwargs']['room_id']

        # Identify requested objects
        try:
            self.room = ChatRoom.objects.get(id=room_id)
        except ChatRoom.DoesNotExist:
            self.close_during_connect(code=4404)  # Not Found
            return

        # Authorize
        if (self.user.id != self.room.owner_id and 
                self.user not in self.room.member_set.all()):
            self.close_during_connect(code=4403)  # Forbidden
            return

        # Join group
        self.room_group = self.room_group_for(room_id)
        async_to_sync(self.channel_layer.group_add)(
            self.room_group,
            self.channel_name
        )

        self.accept()

    def close_during_connect(*, code: int) -> None:
        self.accept()  # must accept before closing with custom code
        self.close(code=code)

    def disconnect(self, close_code) -> None: ...

    # === Group ===

    @staticmethod
    def room_group_for(room_id: int) -> str:
        return 'room_%s' % room_id

    # === Utility ===

    @property
    def user(self) -> User:
        return self.scope['user']

In a regular Django view:

We’d normally use the @login_required decorator to authenticate a user but here we need to do that manually by checking user.is_anonymous and returning WS 4401 Unauthorized manually if the user hasn’t logged in.
We’d normally receive parameters from the URL as arguments to the view function but here we have to look up the arguments manually from self.scope['url_route']['kwargs']['PARAMETER_NAME'] instead.
When authorizing we’d normally check properties of request.user against the room ownership and membership, but here we must check self.user (which is a shortcut for self.scope['user']) instead. A failure would normally be reported using an HttpResponseForbidden but here we must manually return a WS 4403 Forbidden code.

At the end of the connection sequence the consumer adds itself to the Channels “group” for all consumers related to the same room topic.

Upon disconnection the consumer also need to unregister from the group:

class ChatRoomConsumer(WebsocketConsumer):
    def __init__(self, *args, **kwargs) -> None: ...

    def connect(self) -> None: ...

    def disconnect(self, close_code) -> None:
        # Leave group
        if self.room_group is not None:
            async_to_sync(self.channel_layer.group_discard)(
                self.room_group,
                self.channel_name
            )

    ...

This consumer does not expect to receive any incoming messages from the frontend; it only expects to send messages to the frontend later:

class ChatRoomConsumer(WebsocketConsumer):
    def __init__(self, *args, **kwargs) -> None: ...

    def connect(self) -> None: ...

    def disconnect(self, close_code) -> None: ...

    # Receive message from page via WebSocket
    def receive(self, text_data) -> None:
        pass

    ...

Backend Can Send Messages to Frontend

Okay now we have the frontend connecting to the backend. How about we trigger an event on the backend such that it gets observed by the frontend in real-time?

Let’s create a helper method on the consumer that sends an event to all other consumers in the same topic, and forwards it on to the frontend through the consumer’s connected socket:

# chat_project/chat/consumers.py
...
from chat.models import ChatRoomJoinRequest
from typing import Literal

class ChatRoomConsumer(WebsocketConsumer):
    def __init__(self, *args, **kwargs) -> None: ...

    def connect(self) -> None: ...

    def disconnect(self, close_code) -> None: ...

    def receive(self, text_data) -> None: ...

    # Send message to group
    @classmethod
    def notify_did_alter_join_request(cls,
            join_request: ChatRoomJoinRequest,
            action: Literal['create', 'admit', 'deny']) -> None:
        async_to_sync(get_channel_layer().group_send)(
            cls.room_group_for(join_request.room_id),
            {
                'type': 'did_alter_join_request',
                'kwargs': {
                    'id': join_request.id,
                    'action': action,
                    'requesting_user': {
                        'id': join_request.user_id,
                        'first_name': join_request.user.first_name,
                        'last_name': join_request.user.last_name,
                    } if action == 'create' else None
                }
            }
        )

    # Receive message from group
    def did_alter_join_request(self, event) -> None:
        kwargs = event['kwargs']

        # Only the room owner may observe join request events so suppress for other users
        if self.user.id == self.room.owner_id:
            # Forward message to page via WebSocket
            self.send(text_data=json.dumps({
                'type': 'did_alter_join_request',
                'kwargs': {
                    'id': str(kwargs['id']),
                    'action': kwargs['action'],
                    'requesting_user': kwargs['requesting_user'],
                }
            }))

Notice that only the room owner (and not its other members) are notified of events related to join-requests, since the room owner is the only one with the ability to admit or deny folks trying to join the room.

Now we need to receive that forwarded event on the frontend:

// chat_project/chat/static/chat/sockets.js

/*public*/ function setupChatRoomSocket(roomId) { ... }

function didReceiveChatRoomSocketMessage(data) {
    var type = data['type'];
    var kwargs = data['kwargs'];
    if (type === 'did_alter_join_request') {
        console.log(
            'Chat room socket: Did alter join request: ' + 
            kwargs['action'] + ' ' + kwargs['id']);

        if (kwargs['action'] === 'create') {
            // TODO: Create a new row on Chat Room Page in the Members section
        } else if (kwargs['action'] === 'admit') {
            // TODO: Promote the row in the Members section to a full member of
            //       the room, removing the "Deny" and "Allow" buttons
        } else if (kwargs['action'] === 'deny') {
            // TODO: Remove the row in the Members section
        }
    } else {
        console.warn(
            'Chat room socket: Did receive unexpected message type: ' + type);
    }
}

We’ve now carved out a path for events related to join-requests to be forwarded all the way to the frontend in real-time.

Backend Triggers Event to Send to Frontend

We still need to actually trigger the event in the backend by calling ChatRoomConsumer.notify_did_alter_join_request() somewhere.

Let’s make a ChatRoomJoinRequest model fire a create event automatically when it is created by an initial save:

# chat_project/chat/models.py

class ChatRoomJoinRequest(models.Model):
    ...  # fields

    def save(self, *args, **kwargs) -> None:
        change = self.id is not None  # capture
        super().save(*args, **kwargs)
        if not change:  # add
            self.notify_did_alter('create')

    def notify_did_alter(self,
            action: Literal['create', 'admit', 'deny']) -> None:
        from chat.consumers import ChatRoomConsumer  # avoid circular import
        ChatRoomConsumer.notify_did_alter_join_request(self, action)

        # NOTE: In the future we'll want to notify the User Home Page here too
        #from chat.consumers import UserConsumer  # avoid circular import
        #UserConsumer.notify_did_alter_join_request(self, action)

Now if a ChatRoomJoinRequest is created in the Django admin site, its creation should be observed in real-time by the Chat Room Page. Great!

Now let’s alter ChatRoomJoinRequest such that an admit or deny action fires an admit or deny event appropriately. We can temporary add a status field to the model, with a “pending”, “admitted”, or “denied” state, and listen for state changes to determine whether an admit or deny action has been taken:

class ChatRoomJoinRequest(models.Model):
    ...  # original fields
    status = models.CharField(
        max_length=10,
        choices=[
            ('pending', '⏳ Pending'),
            ('admitted', '✅ Admitted'),
            ('denied', '✖️ Denied'),
        ],
        default='pending',
    )

    def __init__(self, *args, **kwargs) -> None:
        super().__init__(*args, **kwargs)
        self._initial_status = self.status

    def save(self, *args, **kwargs) -> None:
        change = self.id is not None  # capture
        super().save(*args, **kwargs)
        if not change:  # add
            self.notify_did_alter('create')
        else:  # change
            if self.status != self._initial_status:
                if self.status == 'admitted':
                    self.notify_did_alter('admit')
                elif self.status == 'denied':
                    self.notify_did_alter('deny')

    def notify_did_alter(self, ...): ...

Now if the status field of a ChatRoomJoinRequest is changed in the Django admin site, the related admit or deny event should be observed in real-time by the Chat Room Page. Fantastic!

Now all actions that can be taken on a ChatRoomJoinRequest can be observed by the Chat Room Page in real-time. ✅ But what the User Home Page? It also wants to be notified of events related to join-requests.

Wiring up the User Home Page

Recall that the User Home Page also wants to observe events related to ChatRoomJoinRequests so that it can notify a user whether their request to join a room was approved or denied by the room owner, in real-time.

Similar to how the Chat Room Page observes events in real-time, we’ll need to create a User Topic that is backed by a frontend socket and a backend consumer that manages that socket (UserConsumer).

The code for setupUserSocket is in the same pattern as that for setupChatRoomSocket so I won’t bother listing it.

Ditto for UserConsumer, whose code is similar to that for ChatRoomConsumer.

Now we need to link ChatRoomJoinRequest.notify_did_alter to notify not just the Chat Room Topic of changes, but also the User Topic via UserConsumer.notify_did_alter_join_request:

# chat_project/chat/models.py

class ChatRoomJoinRequest(models.Model):
    ...  # fields

    def save(self, *args, **kwargs) -> None:
        change = self.id is not None  # capture
        super().save(*args, **kwargs)
        if not change:  # add
            self.notify_did_alter('create')

    def notify_did_alter(self,
            action: Literal['create', 'admit', 'deny']) -> None:
        from chat.consumers import ChatRoomConsumer  # avoid circular import
        ChatRoomConsumer.notify_did_alter_join_request(self, action)

        from chat.consumers import UserConsumer  # avoid circular import
        UserConsumer.notify_did_alter_join_request(self, action)

Review of Implementation

Congratulations! You’ve wired up a non-trivial chat server that can admit and deny requests to join private chat rooms! 🎉

Let’s review how a particular sequence of actions might be taken on the backend and observed on the frontend in real-time:

A staff member logs into the Django admin site and creates a ChatRoomJoinRequest object:
- ChatRoomJoinRequest.save() triggers and detects a create action.
- ChatRoomJoinRequest.notify_did_alter() does forward the message to all interested consumers.
- *Consumer.notify_did_alter_join_request() does forward the message to the Channels group corresponding to the * topic (either a Chat Room Topic or a User Topic).
- *Consumer.did_alter_join_request() for all related consumer instances (on all backend servers) does receive the message, and forwards it on to its connected socket.
- didReceive*SocketMessage on either the Chat Room Page or User Home Page does receive the message, and takes action to update the page immediately. ✅
The staff member changes the status field from its default ⏳ Pending value to be ✅ Admitted, and presses the “Save and continue” button on the admin page:
- ChatRoomJoinRequest.save() triggers and detects an admit action.
- (… same intermediate calls as above …)
- didReceive*SocketMessage on either the Chat Room Page or User Home Page does receive the message, and takes action to update the page immediately. ✅
The staff member does press the “Delete” button on the admin page.
- The ChatRoomJoinRequest model is deleted.

Challenge: Lost events

The above implementation does not handle a particular race condition that can occur if an event of interest (such as the creation of a Chat Message) happens while a page (like the Chat Room Page) is loading. Imagine:

User Alice is on the Chat Room Page and has already posted a first message to the room.
User Bob requests the Chat Room Page from Django, which renders to HTML the current set of Chat Messages in the Chat Room. Bob’s Chat Room Page hasn’t yet loaded its JavaScript yet…
Alice posts a second message to the room, which fires a create action to every loaded page that is listening to the room. However because Bob’s page hasn’t finished loading its JavaScript yet, it has not established a socket connection to the backend yet and misses the event.
Bob’s JavaScript finally loads and establishes a socket connection to the backend.
Alice posts a third message to the room, which get observed by Bob, now that his socket is connected.

By the end of this story Alice has posted three messages to the chat room but Bob only sees the first and third message that Alice posted. Bob missed the second message because it was fired in between when Bob started and finished loading the Chat Room Page.

How can we avoid losing events like this? Join me next week as I sketch some solutions to this tricky scenario.

Happy coding!

In Django⁶ :

Building web apps with Vue and Django - The Ultimate Guide - Planning how to integrate Vue with a new or existing Django app
Database clamps - Writing deterministic performance tests for database-dependent code in Django
Tests as Policy Automation - Has ideas for creatively using automated tests to enforce various (non-functional) properties in your Django web app.

The official Channels tutorial has a nice example of building a chat server.↩

Building web apps with Vue and Django (2024) - The Ultimate Guide

2021-02-16T00:00:00+00:00

1 server or 2 servers?
1-server approach
2-server approach
Conclusion

Vue and Django are both fantastic for building modern web apps - bringing declarative functional reactive programming to the frontend, and an integrated web app platform, ecosystem, and battle-hardened ORM to the backend. However they can be somewhat tricky to use together.

Here I’d like to show some approaches for setting Vue and Django up in combination for both new web apps and existing Django-based web apps. I’ve been building web apps with Django for ~9 years and with Vue for ~6 years, and in particular I’ve extensively tested the 1‑server concatenated bundling approach described below in production.

1 server or 2 servers?

The first question to consider when planning to use Django and Vue together is whether to use a single server that serves both backend endpoints and frontend assets, or to use two different servers, one to host the frontend and a separate “API server” to host the backend.

Factors in favor of a 1-server setup:

Operational maintenance costs are significantly simplified with only a single server to deploy, monitor, and manage.
Relegating Django’s role to only that of an API server - in the 2-server setup - will complicate or even prevent you from using any Django features or third-party apps that rely on server-side rendering by Django, such as its famous built-in administration interface. Instead you’ll be doing most of your server-side rendering with some JavaScript framework that is Node-compatible. And Node-based server-side rendering is generally rather complex to setup.

Factors in favor of a 2-server setup:

If you want to use the latest and greatest version of JavaScript, with transpilation and module bundling, the 2-server setup has great community documentation for setting up a frontend server and build pipeline that supports all of these new JavaScript features.
- However most of those features can still be obtained in a single-server setup with some effort, as described later in this article.
If your organization already has separate frontend and backend teams, it may be easier to have 2 servers - one managed by the frontend team, and one managed by the backend team - to allow each team to focus on each server separately (and satisfy Conway’s Law ¹ 🙂).
If your organization already has a separate operations team that has the capacity to manage the additional server implied by a 2-server setup, then the additional operational overhead may be acceptable.

At my company we went with the single server option because our engineering team is small (< 5 engineers) - so we care a lot about minimizing operational overhead - and is composed of full-stack generalists who are familar with both backend and frontend technologies - so we have engineers that can work anywhere in the stack but aggressively prefer simple architechures that don’t require extreme specialization in multiple domains.

In the next section I’ll drill down into the 1-server approach, but you can also skip to the 2-server approach if that’s what you’re leaning toward.

1-server approach

When using a single server, you’ll need to:

pick an asset bundling strategy,
render SEO-friendly HTML server-side with critical navigation and content available immediately on page load, and
enhance that HTML client-side with Vue after page load with additional dynamic content.

Bundling strategies

When using a single integrated Django server to not only host your backend but also serve your frontend, you’ll need to decide how you want to bundle your JavaScript assets on the frontend together. Bundling of some kind is necessary for a production deployment that is fast enough to be mobile-friendly and usable by distant international customers who may not have fast network connectivity to your server.

There are multiple bundling techniques that can be used with Django, with various pros and cons:

Concatenated Bundling,
Import-Traced Bundling, and
Transpiled Bundling.

Concatenated Bundling

This is by far the easiest bundling technique to use with Django and is a good technique to start with.

With concatenated bundling, you write JavaScript files that consist entirely of top-level function definitions and other non-executable code. Your HTML template served by Django contains <script>-includes of every JavaScript file that the current page requires, directly or transitively. And after all JavaScript files are included the HTML page uses an inline <script> to call the root JavaScript function which sets up the rest of the page:

<!-- todo/templates/todo/todo.html -->
<!DOCTYPE html>
<html>
<head>...</head>
<body>...</body>
{% compress js %}
    <script src="{% static 'todo/todo.js' %}"></script>
    <script src="{% static 'todo/todo/list.js' %}"></script>
    <script src="{% static 'todo/todo/item.js' %}"></script>
    <script>
        setupTodoPage();
    </script>
{% endcompress %}
</html>

Note: The {% compress js %} tag above assumes that you are using the excellent Django Compressor library which integrates well with Django as your concatenating bundler.

Pros of concatenated bundling:

Very easy to setup. Leverages excellent documentation from the Django Compressor library.
Zero bundle build times during development.
Fastest bundle build times during deployment, compared with other approaches.²

Cons of concatenated bundling:

You must manage your JS dependencies in HTML manually:
- Whenever adding a new JS module you must remember to add it to the HTML for the appropriate page(s).
- If you alter an existing JS module to depend on a new JS module, you must find all page HTMLs that include the first module and update them to include the second module if needed.
- If you want to avoid managing JS dependencies manually, consider import-traced bundling instead.
JavaScript files are not transformed or transpiled in any way, so if you want to use newer JavaScript features that aren’t supported by your customers' browsers then you’re out of luck. If you need transpilation consider the transpiled or the 2-server approaches instead.
It is awkward to define a class in a JavaScript module that inherits from a base class in a different module, because that requires <script>-including the base class’s module first, breaking the usual rule that “the order that JS files are imported should not matter”. (However if you just limit inheritance to be within a single module or avoid it entirely, then this restriction doesn’t matter in practice.)

Note that choosing concatenated bundling does not prevent you from using TypeScript-based type checking in your JS files, which I do recommend for long-lived projects. In short, you can put a special // @ts-check comment at the top of a JS file to enable type-checking of JS with the TypeScript compiler (tsc).

Import-Traced Bundling

With the advent of the Vite bundler we can get all the benefits of the concatenated bundling approach while eliminating the need to manage JS dependencies manually, at the cost of a slightly-more-complex deployment process.

With import-traced bundling, you write a root JavaScript file for each page that uses regular JavaScript import statements to bring in related modules. Your HTML template then only needs to include the root JavaScript file:

<!-- todo/templates/todo/todo.html -->
<!DOCTYPE html>
<html>
<head>...</head>
<body>...</body>
<!-- js -->
    <script type="module">
        import { setupTodoPage } from "@app/todo/todo.js";
        setupTodoPage();
    </script>
<!-- endjs -->
</html>

The root JavaScript file might look like:

// static/todo/todo.js
import { defineTodoList } from "@app/todo/list.js";

export function setupTodoPage() {
    const app = Vue.createApp(...);
    
    defineTodoList(app);
    
    app.mount(...);
}

And that root JS file can include other modules like:

// static/todo/list.js
import { defineTodoItem } from "@app/todo/item.js";

export function defineTodoList(app) {
    app.component('todo-list', {
        props: {
            ...
        },
        template: `
            ...
        `,
        methods: {
            ...
        }
    });
    
    defineTodoItem(app);
}

Note: The import statements above assume that Vite has been configured to resolve @app to Django’s root static directory in vite.config.js.

Some key differences between import-traced bundling and concatenated bundling:

The HTML page only needs to include the root JS file for the page and not any of its indirect JS dependencies.
JS modules must use import to declare other JS modules that they depend on, and export any functions that they want to be importable by other modules.
JS files for all Django apps in the Django project should be put in a common static directory rather than using per-app static directories. Having a common directory for all JS files will make it easier to configure Vite to build combined JS bundles for production deployments.
- It should still be possible to configure Vite to use app-specific static directories, with some custom build system modifications, but I haven’t yet tried to do so myself.
Django Compressor is no longer necessary.

For a production deployment, you’ll need to alter your deployment script to run Vite on the root static directory for the Django project, enumerating the set of root JS files, and generating same-named files for deployment to your static asset server.

Pros of import-traced bundling:

Easy to setup for development. (Trickier to setup for deployment.)
Zero bundle build times during development.
Fast bundle build times during deployment.
JS dependencies are automatically managed through regular import statements in JS.

Cons of import-traced bundling:

JavaScript files are not transpiled at development time (although they are at deployment time), so if you want to use the most bleeding-edge JavaScript features that aren’t even supported by your development browser then you’re out of luck. If you need transpilation during development consider the transpiled or the 2-server approaches instead.

To see a full example of combining Django and Vite together using the above strategy, take a look at the hello-django-vite prototype I put together.

Transpiled Bundling

2024 Update: The import-traced bundling approach used with newer bundlers like Vite gives all the advantages of transpiled bundling without any of the downsides. Therefore I would not recommend a traditional transpiler approach in 2024.

If you want the latest and greatest JavaScript features that haven’t made it even to your latest development browser (ex: the latest Chrome or Firefox) then you’ll need to pay the cost of needing to transpile during development time:

The transpiled bundling approach generally uses the same kind of HTML, JS, and filesystem structure as the import-traced bundling approach but you have additional flexibility depending on the particular set of bundler and transpiler tools you select.

Common choices for transpiler tools as of mid 2024 are Vite/Rollup, Webpack, Babel, and TypeScript.

Pros of transpiled bundling:

Latest bleeding-edge JavaScript features are available.
JS dependencies are automatically managed.
- You can also manage dependencies automatically with the simpler import-traced bundling approach.

Cons of transpiled bundling:

Moderate-to-slow bundle build times during development.
- You can eliminate build times during development entirely using either the import-traced bundling or concatenated bundling approaches.
Slow bundle build times during deployment, assuming you enable aggressive optimizations.

For a sketch of how you might wire these tools together, take a look at Jacob Kaplan-Moss’s thoughts on the transpiled bundling approach at PyCon 2019.

Render baseline HTML with Django

Now that you’ve picked a bundling approach, we can move on to rendering our first page with Django and Vue.

When a browser (or search engine crawler) first requests a page from your site, it will only be able to immediately render the initial HTML served by Django. In particular, browsers will take some time to start running any JavaScript on your HTML page, so it’s important that the initial HTML served by Django contains your most important page content.

For example if we were building the product page of an online store, we’d want to ensure the initial HTML rendered by Django immediately contained things like:

the site logo and top navigation links,
the product image,
the product description,
placeholders for other less-important panels that are okay to load later in JavaScript, with appropriate animated spinner icons or other loading indicators.

So the HTML rendered by Django might look something like:

<!-- store/templates/store/product.html -->
<!DOCTYPE html>
<html>
<head>...</head>
<body>
    <!-- Top navigation -->
    <nav class="navbar navbar-inverse">
        <div class="container-fluid">
            <div class="navbar-header">
                <!-- Site logo -->
                <a class="navbar-brand" href="/">Acme Store</a>
            </div>
            <div class="navbar-collapse collapse">
                <!-- Top navigation links -->
                <ul class="nav navbar-nav">
                    <li><a href="/computer-systems/">Computer Systems</a></li>
                    <li><a href="/components/">Components</a></li>
                    <li><a href="/electronics/">Electronics</a></li>
                    ...
                </ul>
            </div>
        </div>
    </nav>
    
    <!-- Main content -->
    <div class="container">
        <div class="content-container">
            <!-- Primary panel -->
            <img alt="Product image" href="{{ product.image_url }}" style="float: left;" />
            <h1>{{ product.title }}</h1>
            <div>
                {{ product.description }}
            </div>
            
            <!-- Secondary panel; a placeholder filled out by JS later -->
            <div id="recommendations-panel">
                <h2>Recommended for You</h2>
                <img
                    v-if="loading" 
                    alt="Loading spinner"
                    src="{% static 'store/loading-spinner.gif' %}" />
                <template
                    v-else
                    v-cloak>
                    ...
                </template>
            </div>
        </div>
    </div>
</body>

{{ recommendations_panel_data|json_script:"recommendations-panel-data" }}

{% compress js %}
    <script src="{% static 'store/product.js' %}"></script>
    <script src="{% static 'store/product/recommendations.js' %}"></script>
    <script>
        setupProductPage();
    </script>
{% endcompress %}
</html>

Notice that most of the page is initially rendered server-side with Django’s built-in templating system and not with Vue. This server-side rendered content loads fast and can be appropriately indexed by search engines for better SEO.

However certain less-important panels like the Recommendations panel don’t need to be loaded immediately and will be given life by Vue later after the page’s JavaScript starts running. Let’s consider how that works:

Enhance baseline HTML with Vue

When the browser (or search engine) first loads the Recommendations panel it will initially see just a loading spinner:

<div id="recommendations-panel">
    <h2>Recommended for You</h2>
    <img
        v-if="loading" 
        alt="Loading spinner"
        src="{% static 'store/loading-spinner.gif' %}" />
    <template
        v-else
        v-cloak>
        ...
    </template>
</div>

Note: The v-cloak directive is associated with the CSS rule [v-cloak] { display: none; } so nothing marked by that directive will be displayed initially.

Later when the page’s JavaScript starts running, it will call setupProductPage():

// store/product.js
/*public*/ function setupProductPage() {
    setupRecommendationsPanel();
}

which will call setupRecommendationsPanel():

// store/product/recommendations.js
/*public*/ function setupRecommendationsPanel() {
    Vue.createApp({
        data() {
            return JSON.parse(document.querySelector('#recommendations-panel-data').innerText);
        },
        computed: {
            loading() {
                ...
            },
            ...
        },
        methods: {
            ...
        }
    }).mount('#recommendations-panel');
}

which will use Vue to render the interior of the panel, perhaps after performing an Ajax request back to a Django endpoint to fetch more data.

Notice that some data for the panel can be prepopulated in HTML via the {{ ...|json_script:"..." }} template tag by Django and then fetched later via document.querySelector(...).innerText in JavaScript.

Done! Skip to the conclusion.

2-server approach

When using a separate frontend server for Vue and a separate backend server for Django, you’ll need to:

setup a standalone Vue server and a standalone Django server using official documentation from each project,
alter the frontend Vue server to support server-side rendering or prerendering using official documentation,
create some pages in Vue, some endpoints in Django (perhaps using the Django REST Framework), and have those pages access those routes using plain old window.fetch() (or something shinier like Axios).

Conclusion

Hopefully this guide has been useful in helping you setup Vue inside your new or existing Django web app. Happy coding!

Database clamps - Writing deterministic performance tests for database-dependent code in Django
Tests as Policy Automation - Has ideas for creatively using automated tests to enforce various (non-functional) properties in your Django web app.
Other Django⁶ articles

Related Projects

TechSmart Platform - Large web app that I work on that uses Django and Vue. (Sorry it’s closed-source!)

Conway’s Law: The technical architecture of a system tends to mirror the organizational and communication structure of the people that build it. So if you have 2 teams building a compiler, they’re likely to build a 2-pass compiler. Similarly if you have a frontend and a backend team, they’re likely to build separate frontend and backend servers if left to themselves.↩
At the time of writing it takes 2.1 seconds for me to bundle 100,650 lines (5,304 KiB) of JS and 17,617 lines (699 KiB) of CSS using Django Compressor’s default settings which uses rJSMin to minify JS (via regex no less!).↩

Database clamps: Deterministic performance tests for database-dependent code

2021-02-09T00:00:00+00:00

If you’ve got a moderate-sized Django web application then you’re probably already writing automated tests to make sure none of its pages break unexpectedly when you’re making changes to them. That is, you’re testing page functionality.

However another way that pages can break is that they take too long to display, or otherwise don’t have enough performance. The very first Django application I deployed to customers got crushed with only 12 concurrent users! How embarassing! 🤭 I hadn’t even bothered to do basic performance testing before that initial deployment because I didn’t think it was possible for such a small number of expected users to bring down my site. I now know better and require that any new web page have automated performance tests before being deployed to customers.

There are many kinds of performance tests, but right now I’d like to focus on automated database performance tests, or what I like to call database clamps:

What are they?

A database clamp measures the number of database queries issued when a web page is being rendered server-side. For example:

from django.test import TestCase

class TodoListPageTests(TestCase):
    ...

    def test_todo_list_mdp(self):  # mdp = maintains database performance
        self.client.login(username='user', password='password')
        with self.assertNumQueries(3):  # <-- DATABASE CLAMP
            response = self.client.get(reverse('todo:list'))
            self.assertEqual(200, response.status_code)

Here, assertNumQueries is used to clamp the number of database queries issued when the todo:list page is rendered server-side. If any changes are made to the the page that increases (or otherwise changes) the number of database queries issued, then the test will detect the change and fail.

Why are they useful?

Database clamps are particularly useful for web applications because server-side rendering time is typically dominated by database query time. If you get your database access patterns under control then it’s likely the remaining server-side rendering time will be negligible.

Also, unlike most other kinds of performance tests, database clamps are fully deterministic and always give consistent results no matter how fast the machine the test is being run on. Very useful!

Conclusion

Avoid the embarassment of your site falling over when only a handful of customers try to use it. Use database clamps!

Appendix: Better database clamps

A database clamp which uses Django’s assertNumQueries function will fail not just when the number of database queries increases (which is usually a problem) but will also fail when the number of queries decreases (which is usually okay, and even desirable).

In my own Django web application I use a custom version of assertNumQueries that still fails if the number of queries increases but only issues a warning (via warnings.warn(...)) if the number of queries decreases:

$ python3 manage.py test gradebook
System check identified no issues (0 silenced).
.
----------------------------------------------------------------------
Ran 1 test in 0.758s

OK

Warnings:
gradebook/tests/test_gradebook.py:425: UserWarning: 14 database queries executed, no more than 15 expected. Consider reducing the expected query count to match.

In addition, my version of assertNumQueries expects to be called from an automated test method whose name contains the word mdp (“maintains database performance”) and warns if it is being called from a test lacking that acronym. This restriction allows my engineering team to easily search for and run exactly those tests which use database clamps when making large scale changes that may break many database clamps at once:

$ python3 manage.py test $(python3 manage.py list_tests mdp -s)
System check identified no issues (0 silenced).
..............s..s..................................................
----------------------------------------------------------------------
Ran X tests in Ys

OK (skipped=Z)

Performance Testing - Details a few of the big guns of performance testing.
Tests as Policy Automation - Has more ideas for creatively using automated tests to enforce additional (non-functional) properties in your backend web application.

Tests as Policy Automation

2021-02-02T00:00:00+00:00

Automated tests are usually used for testing functional requirements of your product code. But they can also be used to enforce other policies and coding practices as well.

If writing a web application it’s likely you already have a rule like “all automated tests must pass before any new version of the web application can be deployed to customers on the production environment”. In that case any new policies you want to enforce regularly can be added to your standard automated test suite and they will necessarily have to be satisfied on every deployment!

Here are some example of special policies I’ve enforced from the automated test suite of a large web application I work on (which uses Django, Python+mypy, and JavaScript+TypeScript as core technologies):

Ensure the typechecker reports no errors
- test_type_checker_reports_no_errors_in_python ¹
- test_type_checker_reports_no_errors_in_typed_javascript ²
Ban unsafe coding patterns by inspecting source code
- test_no_new_fragile_test_suites³
- test_ensure_all_directories_containing_py_files_have_init_file⁴
Ensure hard-coded debug modes are turned off
- test_the_debug_toolbar_is_disabled ⁵
- test_that_compress_is_enabled ⁶
Ensure test-only environmental settings are correctly configured
- test_that_tests_do_not_send_real_emails
Ensure invariants that should apply to all types of a large/unbounded number of domain objects are satisfied:
- test_all_django_add_and_edit_admin_pages_render ⁷ 👈 especially useful and powerful
- test_every_block_type_satisfies_all_block_standards ⁸
  - test_c_blocks_for_python_blocks_must_use_consistent_indent_width
  - test_every_field_id_must_use_underscore_case
  - test_every_block_type_whose_codegen_always_references_x_library_must_import_x_library
  - … (9 more)
Prevent certain configuration settings from changing without triggering a discussion with Product Management, the Business, or your Dev Lead
- test_max_redirect_count_is_5 ⁹

Hopefully these examples give you some ideas of some special policies you might enforce in your own automated test suite. Happy coding!

Shitlist Driven Development - Gives techniques for how to effectively apply automated policy changes of the type discussed in this article at large scale.

Database clamps: Deterministic performance tests for database-dependent code - Shows how to clamp the database performance of server-side rendering using automated tests.

TechSmart’s backend Python and Django code is typechecked using the mypy type checker.↩
TechSmart’s frontend JavaScript code is typechecked using the TypeScript compiler, tsc.↩
In this context a “fragile test suite” corresponds to a subclass of Django’s StaticLiveServerTestCase or TestCase whose setUpClass method fails to use a try-finally to invoke super().tearDownClass() explicitly if something goes wrong partway through the test suite setup. This fragile-detection metatest walks through all test suite classes and uses Python’s inspect.getsourcelines to read the source code of all test classes to look for the absense of the proper kind of try-finally.↩
It is important for any directory containing Python source files (*.py) to contain an __init__.py file so that the directory is marked properly as a Python package and is recognized correctly by Python typecheckers like mypy.↩
The Django Debug Toolbar is an amazingly useful tool to profile the database queries that your Django-rendered page is making.↩
TechSmart’s frontend JavaScript code is concatentated and minified using the excellent Django Compressor app. At some point we may migrate instead to using Snowpack, a lightweight module bundler.↩
The “all Django Add and Edit pages must render” metatest automatically discovers all Django models and related pages that exist in Django’s admin site. Then it tries to navigate to each such admin page and ensures that it renders completely. This metatest has caught cases where some of our more complex custom admin pages broke due to a code change elsewhere.↩
The “blocks” referred to here are the lego-like blocks which snap together to form programs in the visual Skylark language.↩
The Python IDE in the TechSmart Platform contains an API-compatible reimplementation of the popular Requests HTTP client library so that students can write programs that access the internet, in a controlled fashion.↩

Performance Testing

2018-06-02T00:00:00+00:00

In this article I will describe the theory of performance testing¹ and how we conduct such testing in practice at my company TechSmart, in the context of rich web applications and web services.

By the end of this article you should understand how we define performance testing and perhaps get some ideas for implementing or customizing your own tools for performance-testing your own web service.

Theory
Gatling Concepts
- Scenarios and simulations
- Simulation parameters
Load generation
Stress testing
End

Theory

Performance testing of a web service typically involves verifying non-functional requirements such as whether it is:

Responsive - responds quickly to requests
Resilient - handles a large number of requests; avoids crashing outright; rejects or queues requests as needed if too many in progress at once
Fault-Tolerant - whether the service self-heals if it crashes

Performance testing aims to answer the following kinds of questions:

How responsive is the service under a particular amount of load?
(See § “perftest run” below)
What is the maximum load that the service can handle?
(See § “perftest maxload” below)
How responsive is the service under its maximum load?
What is the bottleneck that is saturated when the service is at its maximum load?
(See § “Bottleneck isolation” below)
How does the service behave when stressed beyond its maximum load?
(See § “Overload behavior” below)

The core of performance testing is load generation. You write a program that generates a specified amount of load on your target web service and measure what happens when various amounts of load are applied.

There are many load generation tools that exist:

One of the oldest load generation tools is JMeter, which uses a GUI to define simulation programs in an XML-based file format. JMeter uses a separate OS thread for each concurrent HTTP connection which severely limits the number of concurrent HTTP connections it can host on a single box.
At my company TechSmart we use Gatling as our load generation tool. Compared with JMeter, Gatling is far more efficient at generating load because it uses multiplexed async I/O on a single OS thread to handle all HTTP connections. Also, Gatling simulations are written in an actual programming language (Scala) rather than XML, so you can write your simulations with rich abstractions and do more-advanced customizations.

Gatling Concepts

Scenarios and simulations

A scenario describes a pattern of HTTP requests that a single user makes against a web service. For example in the ViewCodePage scenario a user performs the {LoginPage.loginWithoutRedirect, CodePage.view} subscenarios which consist of individual HTTP requests.

A simulation describes an aggregate pattern of HTTP requests that multiple users make against a web service. For example the ViewCodePage(X, Y) simulation simulates X users arriving over Y seconds that each perform the actions described in the ViewCodePage scenario.

At TechSmart we have written several simulations that exercise each of the major pages on our platform website.

Simulation parameters

Most simulations written at TechSmart vary their behavior based on parameters that are passed in as environment variables. Simulations read these environment variables upon initialization using code like:

class LoginAsStudent extends Simulation {
    val X = sys.env.getOrElse("X", "__missing__").toInt
    val Y = sys.env.getOrElse("Y", "__missing__").toInt

Thus the set of parameters that a particular simulation expects can be deduced by reading the top of the simulation’s source code.

Most simulations at TechSmart support X and Y parameters to inject X users over Y seconds during the simulation.

Load generation

The “gatling” management command

The “gatling” management command is a low-level command we’ve implemented at TechSmart that invokes the Gatling tool, sets up various required paths automatically, and runs a Gatling simulation script.

A typical invocation of the “gatling” management command looks like:

$ X=4 Y=1 pm gatling --simulation tsplatform.LoginAsStudent

Note: The pm command above is an alias for python3 manage.py which is the Django task runner.

This invocation is equivalent to the more-verbose:

$ X=4 Y=1 $GATLING_HOME/bin/gatling.sh --simulation tsplatform.LoginAsStudent --simulations-folder $PERFORMANCE_HOME/simulations --data-folder $PERFORMANCE_HOME/data --bodies-folder $PERFORMANCE_HOME/bodies

The Gatling tool emits lots of output in the console while it is running and eventually generates an HTML report with detailed statistics about what HTTP requests were made during the simulation, response times for individual and aggregated requests, and other information.

The “perftest run” management command

Typically the most important information in the Gatling HTML report generated by running a simulation is the maximum response time² for a particular type of HTTP request that was made during the simulation.

Consequently we’ve created a “perftest run” management command that behaves similarly to the “gatling” command but automatically presents the maximum response time for the most important request type³ after running the simulation.

With the “perftest run” management command, we can easily answer the question:

How responsive is the service under a particular amount of load?
- When R requests/second are made continuously, for some R = X/Y:
  - What is the maximum response time?
  - What does the response time distribution look like?

Here is an example of running a simple simulation, using “perftest setup” to create test data and then using “perftest run” to get the maximum response time for a particular amount of load:

$ pm perftest setup students 8
Deleting test students...
Creating 8 test student(s)...

$ pm perftest setup calendars 1
Deleting test calendars...
Creating 1 test calendar(s)...
Associating 8 user(s) with 1 calendar(s)...

$ pm perftest run LoginAsStudent 4 1
Running simulation with 4 user(s) over 1 second(s)...
When 4 user(s) over 1 second(s), max response time for request 'submit_login' is 393 ms.
  Report: /Users/me/pkgs/gatling-charts-highcharts-bundle-2.2.2/results/loginasstudent-1497049484195/index.html

(We also have a “perftest teardown” command that deletes all test data created by “perftest setup”.)

Targeting remote environments

Our simulations are written by default to target the website running on the developer’s local machine (127.0.0.1). For real testing you’ll want to run tests on a remote version of the website such as the one on a dedicated perf environment (perf.example.com).

For example, to run a command on our performance environment we would first setup the environment with:

$ pm on perf perftest setup students 8
$ pm on perf perftest setup calendars 1

Note: The “on” management command runs some other command in the context of a particular remote environment.

And then we’d run the performance test by typing:

$ pm on perf perftest run LoginAsStudent 4 1

The “on” command sets the GATLING_BASE_URL environment variable (among other things) and the “perftest run” subcommand passes that environment variable to the underlying Gatling simulation. The GATLING_BASE_URL variable specifies the base URL that all HTTP requests are prefixed with. For example the above command is equivalent to:

$ # (Change environment to "perf", defaulting to its database tier, cache tier, etc)
$ X=4 Y=1 GATLING_BASE_URL=http://perf.example.com pm gatling --simulation tsplatform.LoginAsStudent

All simulations support the GATLING_BASE_URL parameter to change the base URL because they all use a common Gatling HTTP Protocol object that defines its base URL from GATLING_BASE_URL:

class LoginAsStudent extends Simulation {
    ...
    val httpProtocol = Common.httpProtocol
    ...
}

object Common {
    private val baseUrl = sys.env.getOrElse(
        "GATLING_BASE_URL", "http://127.0.0.1:8000")

    val httpProtocol = http
        .baseURL(baseUrl)
        ...
}

Stress testing

The “perftest maxload” management command

The “perftest maxload” management command can be used to automatically perform “perftest run” several times to determine the maximum number of users (X) over a given period of time (Y) such that the maximum response time for the HTTP request of interest is less than a particular threshold (1,200 ms by default).

With the “perftest maxload” management command, we can answer the questions:

What is the maximum load that the service can handle?
- What is the maximum requests/second that can be made before the requests start backing up and response times go hockeystick?
How responsive is the service under its maximum load?

An example invocation of “perftest maxload” looks like:

$ pm perftest maxload LoginAsStudent --preserve-calendars
Determining baseline response time...

Deleting test students...
Creating 1 test student(s)...
Associating 1 user(s) with 1 calendar(s)...
Running simulation with 1 user(s) over 5 second(s)...
When 1 user(s) over 5 second(s), max response time for request 'submit_login' is 138 ms.
  Report: /Users/davidf/pkgs/gatling-charts-highcharts-bundle-2.2.2/results/loginasstudent-1497567171908/index.html

Seeking shaft of hockeystick...

Deleting test students...
Creating 2 test student(s)...
Associating 2 user(s) with 1 calendar(s)...
Running simulation with 2 user(s) over 5 second(s)...
When 2 user(s) over 5 second(s), max response time for request 'submit_login' is 124 ms.
  Report: /Users/davidf/pkgs/gatling-charts-highcharts-bundle-2.2.2/results/loginasstudent-1497567182571/index.html

(... ditto for 4 users over 5 seconds ... OK ...)
(... ditto for 8 users over 5 seconds ... OK ...)
(... ditto for 16 users over 5 seconds ... OK ...)
(... ditto for 32 users over 5 seconds ... OK ...)
(... ditto for 64 users over 5 seconds ... OK ...)
(... ditto for 128 users over 5 seconds ... FAIL ...)

Seeking knee of hockeystick... About 6 step(s) or 1:41.

Deleting test students...
Creating 96 test student(s)...
Associating 96 user(s) with 1 calendar(s)...
Running simulation with 96 user(s) over 5 second(s)...
When 96 user(s) over 5 second(s), max response time for request 'submit_login' is 3754 ms.
  Report: /Users/davidf/pkgs/gatling-charts-highcharts-bundle-2.2.2/results/loginasstudent-1497567312703/index.html

(... ditto several times, performing a binary search ...)

num_users,response_time,num_seconds
1,138,5
2,124,5
4,120,5
8,157,5
16,123,5
32,182,5
64,920,5
128,6844,5
96,3754,5
80,2407,5
72,1449,5
68,1227,5
66,1232,5
65,939,5

Maximum load is 65 user(s) over 5 second(s) (13.0 users/second) with a response time of 939 ms.

Notice how “maxload” used a binary search to automatically find the maximum load that the service could handle before the maximum response times went out of bounds. Neat.

Also notice that “maxload” outputs a CSV of every sample taken. This CSV is useful to graph as a scatterplot to see graphically how the response time varies depending on the number of users. We generate these scatterplot graphs often enough that we’ll probably extend “maxload” in the future to just generate a scatterplot image automatically.

Bottleneck isolation

Once you’ve determined the maximum load that your service can support, you can ask yourself whether that load is good enough, based on the level of traffic that you forecast your site will receive in the near future.

Should the maximum load not be good enough, you’ll want to isolate the bottleneck in the system that is constraining the maximum load. Then you can focus on optimizing that bottleneck so that the maximum load increases to be good enough. Note that if you optimize a bottleneck in one area sufficiently, you may find that the bottleneck moves to a different location.

For a web service, the most common bottlenecks in our experience are:

CPU
- Is the CPU usage 100% on some box?
Network
- Are the number of synchronous database requests made by the application tier to the database tier very high (i.e. more than a dozen or so)?
Memory
- Is there no available memory remaining on some box?
- Is there paging activity on some box?
IOPS
- Are the IOPS the maximum for the storage type on some box?
Concurrency Configuration
- Is the maximum number of worker processes active? (Gunicorn)
- Is the maximum number of connections per worker process being reached? (Nginx)
Disk Usage
- Has the maximum quota on the /tmp filesystem been reached?

To isolate the bottleneck, use “perftest run” to start generating a load on the service equivalent to the maximum load it can handle (as measured previously by “perftest maxload”) for a long time interval, say 10 minutes. While that load is being generated, use monitoring tools to look at the usage of CPU, Memory, IOPS, and other resources on each box in the system to look for saturation. When you find the saturated resource, that’s the bottleneck.

Once you’ve located the bottleneck, there are usually a few options to optimize it away. To give you some ideas, here are some bottlenecks that the TechSmart website has hit in its performance testing (from least to most recently):

Not enough frontend workers
- Saturated resource: All frontend Gunicon workers busy 100% of the time, yet not 100% of the CPU utilized
- Fix: Reconfigure Gunicorn to increase the worker count to (2*N + 1), where N is the number of CPU cores.
Too many database requests
- Saturated resource: Frontends busy waiting on the network for dozens of synchronous database queries to complete for a single page load
- Fixes:
  - Optimize frontend logic to reduce the number of database queries per page load to less than a dozen.⁴
  - Use automated tests to clamp the database query count so that it doesn’t increase.
Not enough frontend CPU
- Saturated resource: 100% CPU on each box in the frontend cluster
- Fix: Increase frontend cluster size from 2 up to 5 boxes. At this point the bottleneck moves elsewhere.
Not enough database IOPS
- Saturated resource: IOPS is maximum for the database storage type (i.e. Magnetic or SSD).
- Fixes:
  - Shrink database contents with compression to reduce persisted data volume.
  - Switch database storage type from Magnetic to SSD to increase the maximum IOPS.
- Other potential fixes:
  - Increase RAM on database servers so that the database contents fit into memory, making IOPS irrelevent.
  - Scale reads. Create read replicas of the database.
  - Scale writes. Shard the database to multiple database masters.
Not enough remaining concurrent requests quota
- Saturated resource: Maximum number of concurrent requests to Nginx
- Fix: Increase configured maximum number of concurrent requests.

And here are some bottlenecks that our performance testing has identified we will hit under much higher loads than what we currently experience:

Not enough database CPU
- Saturated resource: 100% CPU on the single master database server
- Potential fixes:
  - Cache more aggressively, so that requests are diverted from the database tier to the cache tier.
  - Scale reads. Create read replicas of the database.
  - Scale writes. Shard the database to multiple database masters.

Overload behavior

It is useful to determine what happens when your service is subjected to greater than its maximum load. In particular:

Does it reject requests loudly?
Does it drop requests silently?
Does it queue requests? Up to a particular soft or hard limit?
Does it crash?
If it crashes, does it automatically restart?

You should decide what desired behavior you want your system to exhibit when receiving a temporary over-maximum load (i.e. a spike) or a sustained over-maximum load (i.e. a flood). Then verify whether the actual behavior matches the desired behavior.

If your service is generally written with infinitely-flexible buffers, it’s likely that receiving sustained over-maximum load will causes requests to queue up until memory is exhausted, and the out-of-memory condition will cause the service to crash.

On the other hand if your service is generally written with fixed-size buffers, it’s likely that receiving any over-maximum load will cause requests to be rejected or dropped.

End

Hopefully this article has provided some insight into concepts around performance testing and given you some ideas about how to implement or improve tooling to perform performance testing.

If you have any improvements or other comments on the contents of this article I’d love to hear from you.

Updates:

2022-02-06:
- Mention more types of bottlenecks I’ve encountered at TechSmart since 2018.
- Bold common patterns of overload behavior.
- Add table of contents for easier navigation.

For the purposes of this article I define performance testing to include load testing and stress testing.↩
Many other performance testing tools focus not on the maximum response time but rather on other measures such as the 99th percentile response time or the mean response time, which are inappropriate to use.↩
The most important request type for a particular simulation is determined by inspecting the source code for a simulation file and looking for a line like val PROFILED_REQUEST_NAME = "view_code".↩
In Django, an easy way to eliminate a bunch of database queries is to make appropriate use of select_related() and prefetch_related() to pre-fetch a group of relationships all at once (with a constant number of queries) rather than one at a time (with a variable number of queries, depending on how many relationships there are).↩

DaFoster (Django)

Reliable rendering of web pages that view concurrently modified data

The Problem

Designing a Solution

Alternative Design: No backend templating

Implementation (using Django)

Related Articles

Real-time updates in Django with WebSockets, Channels, and pub-sub

Topics and Messages

Simple Example: Public Chat Rooms

Extended Example: Private Chat Rooms

Implementation in Django

Models, Consumers, and Routes

Connect Frontend to Backend

Backend Can Send Messages to Frontend

Backend Triggers Event to Send to Frontend

Wiring up the User Home Page

Review of Implementation

Challenge: Lost events

Related Articles

Building web apps with Vue and Django (2024) - The Ultimate Guide

1 server or 2 servers?

1-server approach

Bundling strategies

Concatenated Bundling

Import-Traced Bundling

Transpiled Bundling

Render baseline HTML with Django

Enhance baseline HTML with Vue

2-server approach

Conclusion

Related Articles

Related Projects

Database clamps: Deterministic performance tests for database-dependent code

What are they?

Why are they useful?

Conclusion

Appendix: Better database clamps

Related Articles

Tests as Policy Automation

Related Articles

Performance Testing

Theory

Gatling Concepts

Scenarios and simulations

Simulation parameters

Load generation

The “gatling” management command

The “perftest run” management command

Targeting remote environments

Stress testing

The “perftest maxload” management command

Bottleneck isolation

Overload behavior

End