Designing Faster Django APIs: From Database Queries to Response-Time Optimization

When an endpoint feels slow, the first instinct is often to blame the framework. In practice, Django API performance optimization rarely comes down to how fast Django itself runs. Response time is the sum of many smaller costs: the number and shape of database queries, how the ORM loads related objects, how much work serialization does, what the application logic computes per request, which external APIs it waits on, the network between every hop, and whether the infrastructure has enough capacity for the traffic it receives.

Those costs interact. A harmless-looking serializer can trigger hundreds of queries, and a cache can hide a slow query until it expires under load.

This article walks through the layers of a typical Django REST Framework service, from the ORM up to load testing, with before-and-after code. The guiding principle: start with measurement, not assumptions, and measure again after every change.

1. Understanding API response time#

Where the time goes#

A single API request passes through several stages, and each one contributes latency:

• Network: DNS, TLS handshakes, round trips, proxies and load balancers.
• Middleware: every middleware class runs on the way in and on the way out.
• Authentication and permissions: token lookups and permission checks often hit the database or a cache.
• Database queries: usually the largest and most variable component.
• Business logic: calculations, validation and Python loops over data.
• Serialization: converting model instances, nested objects and computed fields into primitives.
• Response generation: rendering JSON, compression and writing bytes to the client.

Profiling tells you which of these dominates for a given endpoint. Without it, you can spend days tuning serialization on an endpoint that mostly waits on a third-party API.

Latency versus throughput#

Latency is how long one request takes. Throughput is how many requests the system completes per second. Adding Gunicorn workers can raise throughput without changing per-request latency, so when someone says an API is "slow", clarify which they mean.

Percentiles: p50, p95 and p99#

Averages hide the experience of your unluckiest users. Percentiles describe the distribution: at p50 (the median) half of requests are faster, at p95 one request in twenty is slower, and at p99 one in a hundred is slower.

The gap between p50 and p99 is where lock contention, cache misses, cold connections and unusually large customers show up. A client that makes several calls to render one screen hits tail latency far more often than any single percentile suggests, so track p95 and p99, not only the median.

2. Identifying database bottlenecks#

Example models#

The examples in this article use a hypothetical order management API with four models.

from django.db import models
 
 
class Customer(models.Model):
    name = models.CharField(max_length=200)
    email = models.EmailField(unique=True)
    created_at = models.DateTimeField(auto_now_add=True)
 
 
class Product(models.Model):
    name = models.CharField(max_length=200)
    sku = models.CharField(max_length=64, unique=True)
    price = models.DecimalField(max_digits=12, decimal_places=2)
    description = models.TextField(blank=True)
 
 
class Order(models.Model):
    class Status(models.TextChoices):
        PENDING = "pending"
        PAID = "paid"
        SHIPPED = "shipped"
 
    customer = models.ForeignKey(
        Customer, on_delete=models.PROTECT, related_name="orders"
    )
    status = models.CharField(
        max_length=20, choices=Status.choices, default=Status.PENDING
    )
    created_at = models.DateTimeField(auto_now_add=True)
 
    class Meta:
        indexes = [
            models.Index(fields=["customer", "-created_at"], name="order_customer_created_idx"),
            models.Index(fields=["status", "-created_at"], name="order_status_created_idx"),
        ]
 
 
class OrderItem(models.Model):
    order = models.ForeignKey(Order, on_delete=models.CASCADE, related_name="items")
    product = models.ForeignKey(Product, on_delete=models.PROTECT)
    quantity = models.PositiveIntegerField()
    unit_price = models.DecimalField(max_digits=12, decimal_places=2)

Lazy QuerySets and when they evaluate#

QuerySets are lazy. Building Order.objects.filter(status="paid") does not touch the database. The query runs when the QuerySet is evaluated: when you iterate over it, slice it with a step, or call len(), list() or bool() on it. Results are then cached on that instance. .count() and .exists() run separate, cheaper queries, while len(qs) loads every row. Laziness also makes it easy to trigger queries in unexpected places, such as a serializer field or a model property.

The N+1 query problem#

The classic Django N+1 queries problem looks like this:

from rest_framework import serializers
 
from .models import Order, OrderItem
 
 
class OrderItemSerializer(serializers.ModelSerializer):
    product_name = serializers.CharField(source="product.name")
 
    class Meta:
        model = OrderItem
        fields = ["id", "product_name", "quantity", "unit_price"]
 
 
class OrderSerializer(serializers.ModelSerializer):
    customer_name = serializers.CharField(source="customer.name")
    items = OrderItemSerializer(many=True)
 
    class Meta:
        model = Order
        fields = ["id", "status", "created_at", "customer_name", "items"]

from rest_framework import viewsets
 
from .models import Order
from .serializers import OrderSerializer
 
 
class OrderViewSet(viewsets.ReadOnlyModelViewSet):
    serializer_class = OrderSerializer
    queryset = Order.objects.all()

For a page of 50 orders, this runs one query for the orders, one per order for the customer, one per order for the items, and one per item for the product: easily hundreds of queries, each cheap but expensive in aggregate.

select_related and prefetch_related#

select_related follows foreign keys and one-to-one relations with a SQL join, loading the related object in the same query. prefetch_related handles reverse foreign keys and many-to-many relations with one extra query per relation, then joins the results in Python.

A Prefetch object lets you customise the prefetch query itself, which is how you avoid a second N+1 on the product inside each item:

from django.db.models import Prefetch
from rest_framework import viewsets
 
from .models import Order, OrderItem
from .serializers import OrderSerializer
 
 
class OrderViewSet(viewsets.ReadOnlyModelViewSet):
    serializer_class = OrderSerializer
 
    def get_queryset(self):
        items = OrderItem.objects.select_related("product").only(
            "id", "order_id", "quantity", "unit_price", "product__name"
        )
        return (
            Order.objects.select_related("customer")
            .prefetch_related(Prefetch("items", queryset=items))
            .order_by("-created_at", "-id")
        )

The request now runs two queries regardless of page size: orders joined with customers, then items joined with products.

only() and defer(), with care#

only() and defer() restrict which columns are loaded, which helps when a model has large columns (such as Product.description) the endpoint never returns. The pitfall: accessing a deferred field later is not an error. Django silently issues another query per instance, so a serializer field added later can reintroduce an N+1 pattern with no warning. Above, order_id is included deliberately because Django needs it to attach prefetched items to their orders. Apply only() once an endpoint's fields have stabilised, and cover it with a query-count test (section 3).

Database indexes#

An index lets PostgreSQL find rows without reading the whole table. Indexes belong on columns used in WHERE, JOIN and ORDER BY clauses of frequent queries. The Meta.indexes on Order support "this customer's recent orders" and "recent orders with a given status". Column order matters: an index on (customer, created_at) serves filters on customer alone, but not on created_at alone. Each index costs storage and slows writes, so add them in response to observed query plans rather than speculatively.

3. Measuring SQL performance#

Query logging in development#

With DEBUG = True, Django records every query on django.db.connection.queries, and the django.db.backends logger emits each SQL statement at the DEBUG level:

LOGGING = {
    "version": 1,
    "disable_existing_loggers": False,
    "handlers": {
        "console": {"class": "logging.StreamHandler"},
    },
    "loggers": {
        "django.db.backends": {
            "handlers": ["console"],
            "level": "DEBUG",
        },
    },
}

Watching the console while calling an endpoint is one of the fastest ways to spot repeated queries. Never enable DEBUG in production.

Django Debug Toolbar#

Django Debug Toolbar shows each request's queries, timings and duplicates in the browser, and works with DRF's browsable API.

Locking in query counts with assertNumQueries#

Once an endpoint is optimised, protect it with a test. assertNumQueries fails if the code inside the block runs a different number of queries:

from django.urls import reverse
from rest_framework.test import APITestCase
 
from orders.models import Customer, Order, OrderItem, Product
 
 
class OrderListQueryCountTests(APITestCase):
    @classmethod
    def setUpTestData(cls):
        product = Product.objects.create(name="Widget", sku="W-1", price="10.00")
        for i in range(20):
            customer = Customer.objects.create(name=f"C{i}", email=f"c{i}@example.com")
            order = Order.objects.create(customer=customer)
            OrderItem.objects.create(
                order=order, product=product, quantity=2, unit_price="10.00"
            )
 
    def test_order_list_query_count_is_constant(self):
        url = reverse("order-list")
        with self.assertNumQueries(2):
            response = self.client.get(url)
        self.assertEqual(response.status_code, 200)

The expected number depends on your authentication, permissions and pagination (page-number pagination adds a COUNT query). What matters is that it does not grow with the data.

EXPLAIN and EXPLAIN ANALYZE in PostgreSQL#

EXPLAIN shows the plan PostgreSQL intends to use, with estimated costs and rows. EXPLAIN ANALYZE runs the query and adds actual timings and row counts.

EXPLAIN ANALYZE
SELECT id, status, created_at
FROM orders_order
WHERE customer_id = 42
ORDER BY created_at DESC
LIMIT 20;

From Django, QuerySet.explain() returns the plan for the SQL the ORM generates. On PostgreSQL you can pass options such as analyze=True:

from orders.models import Order
 
qs = Order.objects.filter(customer_id=42).order_by("-created_at")[:20]
print(qs.explain(analyze=True))

Reading a plan#

The following plan is illustrative, written to show what to look for rather than taken from a real system:

Limit  (cost=0.42..8.95 rows=20 width=32) (actual time=0.031..0.064 rows=20 loops=1)
  ->  Index Scan using order_customer_created_idx on orders_order
        (cost=0.42..512.30 rows=1200 width=32) (actual time=0.030..0.060 rows=20 loops=1)
        Index Cond: (customer_id = 42)
Planning Time: 0.150 ms
Execution Time: 0.085 ms

Points to check in any plan:

• Seq Scan versus Index Scan: a sequential scan reads the whole table. That is often optimal for small tables or queries returning many rows, but on a large table filtered to a few rows it usually signals a missing or unusable index.
• Execution Time: actual time spent, as distinct from the planner's abstract cost units.
• Estimated versus actual rows: large mismatches mean the planner is working from stale statistics and may choose a poor plan. Running ANALYZE on the table can help.
• Sort nodes: a large explicit sort, especially one spilling to disk, often means no index matches the ORDER BY.

4. Optimizing DRF serialization#

Where serializer overhead comes from#

Django REST Framework optimization is mostly about two things: how much Python work each object requires, and whether serializers trigger queries. Each field runs conversion logic, and nested serializers multiply that cost by the number of related objects. Consider lightweight serializers for list endpoints and richer ones for detail views.

The SerializerMethodField trap#

SerializerMethodField is the most common place for hidden queries:

from rest_framework import serializers
 
from .models import Order
 
 
class OrderSummarySerializer(serializers.ModelSerializer):
    item_count = serializers.SerializerMethodField()
    total = serializers.SerializerMethodField()
 
    class Meta:
        model = Order
        fields = ["id", "status", "created_at", "item_count", "total"]
 
    def get_item_count(self, obj):
        return obj.items.count()
 
    def get_total(self, obj):
        return sum(item.quantity * item.unit_price for item in obj.items.all())

get_item_count runs a COUNT query per order, and get_total loads every item per order unless items were prefetched. Note that .count() on a prefetched relation reuses the prefetch cache, but .filter() or .aggregate() on it always hits the database.

Preparing data in get_queryset#

Move the computation into the database, where it runs once for the whole page:

from django.db.models import Count, DecimalField, ExpressionWrapper, F, Sum
from rest_framework import serializers, viewsets
 
from .models import Order
 
 
class OrderSummarySerializer(serializers.ModelSerializer):
    item_count = serializers.IntegerField(read_only=True)
    total = serializers.DecimalField(max_digits=14, decimal_places=2, read_only=True)
 
    class Meta:
        model = Order
        fields = ["id", "status", "created_at", "item_count", "total"]
 
 
class OrderSummaryViewSet(viewsets.ReadOnlyModelViewSet):
    serializer_class = OrderSummarySerializer
 
    def get_queryset(self):
        line_total = ExpressionWrapper(
            F("items__quantity") * F("items__unit_price"),
            output_field=DecimalField(max_digits=14, decimal_places=2),
        )
        return (
            Order.objects.annotate(
                item_count=Count("items"),
                total=Sum(line_total),
            )
            .order_by("-created_at", "-id")
        )

The serializer now reads plain attributes, so it cannot trigger queries. One caution: combining several Count or Sum annotations across different multi-valued relations in the same query produces join duplication and wrong results. In that case, use Subquery expressions or Count(..., distinct=True) where appropriate.

The general rule is to make serializers pure functions of data that get_queryset has already loaded. When a serializer needs something, the fix belongs in the view's queryset, not in the serializer method.

5. Caching strategies#

Caching trades freshness and complexity for speed. Cache results that are expensive to compute, read often and tolerant of brief staleness.

Configuring Redis#

Since Django 4.0, Django ships a built-in Redis backend, so Redis caching in Django needs no third-party cache package:

import os
 
CACHES = {
    "default": {
        "BACKEND": "django.core.cache.backends.redis.RedisCache",
        "LOCATION": os.environ.get("REDIS_URL", "redis://127.0.0.1:6379/1"),
        "KEY_PREFIX": "orders-api",
        "TIMEOUT": 300,
    }
}

Cache-aside with careful key design#

In the cache-aside pattern, the application checks the cache, falls back to the database on a miss and stores the result. Key design determines correctness:

• Include everything that changes the output: query parameters, page or cursor, language, API version.
• Scope per user or tenant when the response depends on who asks. A missing user ID in a key is a data leak.
• Version keys so a new response shape never receives old payloads.

import hashlib
import json
import random
 
from django.core.cache import cache
 
from .models import Product
 
PRODUCT_LIST_VERSION = 2
BASE_TTL = 300
 
 
def product_list_key(params: dict) -> str:
    raw = json.dumps(params, sort_keys=True)
    digest = hashlib.sha256(raw.encode()).hexdigest()[:16]
    return f"products:list:v{PRODUCT_LIST_VERSION}:{digest}"
 
 
def get_product_list(params: dict) -> list:
    key = product_list_key(params)
    data = cache.get(key)
    if data is not None:
        return data
 
    qs = Product.objects.order_by("name").values("id", "name", "sku", "price")
    if search := params.get("search"):
        qs = qs.filter(name__icontains=search)
    data = list(qs[:100])
 
    ttl = BASE_TTL + random.randint(0, 60)
    cache.set(key, data, timeout=ttl)
    return data

The random jitter on the TTL stops keys created together from expiring together.

Expiration and invalidation#

A TTL bounds staleness. Explicit invalidation reduces it further when data changes. Deleting every parameter variant is impractical, so a common approach is a generation counter embedded in the key:

from django.core.cache import cache
from django.db import transaction
from django.db.models.signals import post_delete, post_save
from django.dispatch import receiver
 
from .models import Product
 
GENERATION_KEY = "products:generation"
 
 
def bump_product_generation():
    try:
        cache.incr(GENERATION_KEY)
    except ValueError:
        cache.set(GENERATION_KEY, 1, timeout=None)
 
 
@receiver([post_save, post_delete], sender=Product)
def invalidate_product_cache(sender, **kwargs):
    transaction.on_commit(bump_product_generation)

The list key then includes cache.get(GENERATION_KEY). Bumping in on_commit stops readers repopulating the cache with old data before the write is visible. Signals do not fire for QuerySet.update() or bulk operations, so explicit invalidation in a service function is often more reliable.

Per-view caching and its caveats#

cache_page caches an entire response, keyed by URL by default, so on authenticated endpoints one user's response could be served to another. Varying on the relevant headers makes the key per-credential:

from django.utils.decorators import method_decorator
from django.views.decorators.cache import cache_page
from django.views.decorators.vary import vary_on_headers
from rest_framework import viewsets
 
from orders.models import Product
 
from .serializers import ProductSerializer
 
 
class PublicProductViewSet(viewsets.ReadOnlyModelViewSet):
    queryset = Product.objects.order_by("name")
    serializer_class = ProductSerializer
 
    @method_decorator(cache_page(60 * 5))
    @method_decorator(vary_on_headers("Authorization", "Accept-Language"))
    def list(self, request, *args, **kwargs):
        return super().list(request, *args, **kwargs)

Even so, reserve per-view caching for public data. For per-user data, the low-level API (cache.get, cache.set, cache.delete) with explicit keys gives more control.

Cache stampedes#

When a popular key expires, every concurrent request misses at once and recomputes the same result, potentially overwhelming the database. Mitigations: jittered TTLs; locking (single-flight) so only one request recomputes, using cache.add, which sets a key only if absent; and early recomputation shortly before expiry, from requests or a scheduled task.

import random
import time
 
from django.core.cache import cache
 
 
def get_or_rebuild(key, builder, ttl=300, lock_ttl=30):
    value = cache.get(key)
    if value is not None:
        return value
 
    lock_key = f"{key}:lock"
    if cache.add(lock_key, "1", timeout=lock_ttl):
        try:
            value = builder()
            cache.set(key, value, timeout=ttl + random.randint(0, ttl // 10))
            return value
        finally:
            cache.delete(lock_key)
 
    for _ in range(10):
        time.sleep(0.05)
        value = cache.get(key)
        if value is not None:
            return value
    return builder()

The lock's own timeout means a crashed worker cannot block rebuilding forever. The final fallback recomputes rather than failing; serving a stale copy is a valid alternative.

Stale data and financial information#

6. Pagination and large datasets#

Unbounded lists make endpoints slow and memory-hungry. DRF offers three built-in pagination styles.

OFFSET makes the database produce and discard every skipped row, so deep pages cost far more than early ones, and the COUNT(*) for the total can be expensive on large filtered tables. For feeds, logs and transaction histories, cursor pagination is usually the better fit:

from rest_framework.pagination import CursorPagination
 
 
class OrderCursorPagination(CursorPagination):
    page_size = 50
    max_page_size = 200
    page_size_query_param = "page_size"
    ordering = ("-created_at", "-id")

from rest_framework import viewsets
 
from .models import Order
from .pagination import OrderCursorPagination
from .serializers import OrderSerializer
 
 
class OrderViewSet(viewsets.ReadOnlyModelViewSet):
    serializer_class = OrderSerializer
    pagination_class = OrderCursorPagination
 
    def get_queryset(self):
        return Order.objects.filter(
            customer__email=self.request.user.email
        ).select_related("customer")

The ordering must be deterministic. Two orders can share a created_at value, so -id acts as a unique tiebreaker. DRF builds the cursor position from the first ordering field and handles ties on it with an offset, so that first field should be nearly unique and unchanging after creation, which a creation timestamp satisfies. Back the ordering with a matching index, for example models.Index(fields=["-created_at", "-id"], name="order_created_id_idx"), plus a composite index that leads with any filter column you apply, as with the (customer, -created_at) index from section 2.

7. Asynchronous processing#

Emails, reports, slow external integrations and other long tasks do not need to finish before the response is sent. Celery moves them onto background workers so the request returns quickly.

Celery removes work from the request path; it does not make CPU-bound work faster. A thirty-second report still takes thirty seconds on a worker, and saturated workers mean a growing queue. You also gain new failure modes: lost tasks, duplicate execution and out-of-order processing.

Retries and idempotency#

Tasks calling external services should retry with backoff. Because retries and redeliveries mean a task can run more than once, tasks must also be idempotent.

import requests
from celery import shared_task
from django.db import transaction
 
from .models import Order
 
 
@shared_task(
    bind=True,
    autoretry_for=(requests.RequestException,),
    retry_backoff=True,
    retry_jitter=True,
    retry_kwargs={"max_retries": 5},
    acks_late=True,
)
def notify_fulfilment(self, order_id):
    with transaction.atomic():
        order = Order.objects.select_for_update().get(pk=order_id)
        if order.status == Order.Status.SHIPPED:
            return "already processed"
 
    response = requests.post(
        "https://fulfilment.example.com/api/orders/",
        json={"order_id": order_id, "idempotency_key": f"order-{order_id}"},
        timeout=(3.05, 10),
    )
    response.raise_for_status()
 
    Order.objects.filter(pk=order_id).update(status=Order.Status.SHIPPED)

The status check makes a repeated run a no-op, and the idempotency key lets the remote service deduplicate retried requests. Without the timeout, a hung connection would occupy the worker indefinitely.

Transaction boundaries#

If you enqueue a task inside a transaction, the worker can run before the commit and find missing or stale data. Enqueue after commit:

from django.db import transaction
 
from .models import Order
from .tasks import notify_fulfilment
 
 
def mark_order_paid(order_id):
    with transaction.atomic():
        order = Order.objects.select_for_update().get(pk=order_id)
        order.status = Order.Status.PAID
        order.save(update_fields=["status"])
        transaction.on_commit(lambda: notify_fulfilment.delay(order.pk))

If the transaction rolls back, the task is never sent. Celery 5.4 added a delay_on_commit() shortcut for Django projects that does the same thing; transaction.on_commit works with any version.

8. Benchmarking and load testing#

Optimizations are hypotheses until measured:

1. Establish a baseline on an environment resembling production in data volume, database configuration and worker count. A query that is instant against 100 rows can behave very differently against ten million.
2. Change one thing at a time, so you can attribute any difference.
3. Compare percentiles, throughput and error rates at the same concurrency, over repeated runs.

Record concurrency, throughput, p50/p95/p99 latency and error rate for each run. A change that improves p50 while raising the error rate under load is not an improvement.

Tools#

• Locust describes user behaviour in Python, which suits Django teams.
• k6 uses JavaScript scripts and supports thresholds that fail a run when, for example, p95 latency exceeds a target.
• Django Silk profiles requests and queries inside the application. It helps locate the slow part of a request, but its overhead makes it unsuitable for load testing.

A minimal Locust file for the order API:

from locust import HttpUser, between, task
 
 
class OrderApiUser(HttpUser):
    wait_time = between(1, 3)
 
    def on_start(self):
        self.client.headers.update({"Authorization": "Token test-token-for-load-env"})
 
    @task(3)
    def list_orders(self):
        self.client.get("/api/orders/", name="/api/orders/")
 
    @task(1)
    def order_detail(self):
        self.client.get("/api/orders/1/", name="/api/orders/[id]/")

The name argument groups URLs with different IDs into one statistic. Run load tests against a dedicated environment, and watch that the load generator itself does not become the bottleneck.

9. Production optimization checklist#

A review aid for endpoints that matter, not a set of universal rules:

• Query counts: list endpoints run a constant number of queries, enforced by assertNumQueries tests.
• Indexing: frequent filters and orderings are backed by indexes verified with EXPLAIN ANALYZE on realistic data; unused indexes are removed.
• Caching: clear keys, TTLs and invalidation; per-user scoping; an explicit consistency decision for financial data.
• Pagination: no unbounded lists; large or append-heavy collections use cursor pagination with an indexed, deterministic ordering.
• Serialization: lean list serializers; computed values come from annotations.
• External service timeouts: every outbound HTTP call sets a timeout. requests has no default timeout, so a call without one can hang a worker indefinitely.
• Monitoring: per-endpoint latency percentiles, error rates and throughput, alongside slow-query and connection metrics.
• Database connection management: connections are reused rather than opened per request.

import os
 
DATABASES = {
    "default": {
        "ENGINE": "django.db.backends.postgresql",
        "NAME": os.environ["POSTGRES_DB"],
        "USER": os.environ["POSTGRES_USER"],
        "PASSWORD": os.environ["POSTGRES_PASSWORD"],
        "HOST": os.environ.get("POSTGRES_HOST", "db"),
        "PORT": os.environ.get("POSTGRES_PORT", "5432"),
        "CONN_MAX_AGE": 60,
        "CONN_HEALTH_CHECKS": True,
    }
}

Pick either persistent connections or the native pool for a deployment rather than layering them. Under ASGI, the Django documentation recommends disabling persistent connections in favour of pooling.

Conclusion#

Faster Django APIs come from finding where time is spent and addressing the largest costs first. In most DRF services, that means controlling query counts with select_related, prefetch_related and annotations, backing frequent access patterns with indexes confirmed by EXPLAIN ANALYZE, keeping serializers free of hidden queries, and bounding responses with the right pagination. Caching and background processing help, but each brings consistency and failure-mode decisions that must be made deliberately, especially around financial data.

None of this replaces measurement. Establish a baseline, change one thing at a time, compare percentiles and keep watching production after you ship. Fast APIs are the result of measurement, efficient data access, thoughtful architecture, and continuous observation.

Further reading#

• Django: Database access optimization
• Django: QuerySet API reference
• Django: Model index reference
• Django: Testing tools
• Django: Cache framework
• Django: Databases
• Django REST framework: Serializers
• Django REST framework: Pagination
• PostgreSQL: Using EXPLAIN
• Celery: Tasks
• Locust documentation
• Grafana k6 documentation