Table of contents

• Middleware
  • Test environment
  • Baseline measurement
  • FastAPI timing middleware
    • Observations
  • With two middlewares
    • Observations
  • Starlette timing middleware
    • Observations
  • With two Starlette middlewares
    • Observations
• Verdict

Middleware

FastAPI is based on Starlette which supports Middleware, a codebase which wraps your application and runs before / after the request processing. With this you can resolve various functions ( authentication, session, logging, metric collection, etc) without taking care of these functions in your application. Unfortunately the most straightforward implementation has a drawback, it has major impact on the application latency and throughput.

Test environment

• The usual test set was used

Baseline measurement

Sample application without any middleware.

Note: The test application is available here which is from the FastAPI docs

Test attribute	Test run 1	Test run 2	Test run 3	Average
Requests per second	1885.57	1924.34	1953.53	1921.15
Time per request [ms]	53.034	51.966	51.189	52.063

FastAPI timing middleware

Source of middleware is the official FastAPI docs

You can add middleware as decorator:

    @app.middleware("http")
    async def add_process_time_header(request: Request, call_next):
        start_time = time.time()
        response = await call_next(request)
        process_time = time.time() - start_time
        response.headers["X-Process-Time"] = str(process_time)
        return response

Test attribute	Test run 1	Test run 2	Test run 3	Average	Difference to baseline
Requests per second	1346.22	1448.14	1423.86	1406.07	-26.81 %
Time per request [ms]	74.282	69.054	70.232	71.1893	-19.13 ms

Observations

• Significant drop in the througtput of the container while the average latency raised by ~19ms

With two middlewares

Middleware can be defined as child class of BaseHTTPMiddleware:

    class CustomHeaderMiddleware(BaseHTTPMiddleware):
        async def dispatch(self, request, call_next):
            response = await call_next(request)
            response.headers["Custom"] = "Example"
            return response

Alternative way of adding the middleware to the aplication is:

app.add_middleware(CustomHeaderMiddleware)

Test attribute	Test run 1	Test run 2	Test run 3	Average	Difference to baseline
Requests per second	1128.21	1121.99	1113.53	1121.24	-41.64 %
Time per request [ms]	88.636	89.127	89.804	89.189	-37.13 ms

Observations

• By adding another middleware there is significant drop again, the container throughput is around half than before
• Average response time is raised

Starlette timing middleware

Fortunately there is a better way of extending the application with middleware capabilities however this is slightly less convenient: Motivation is from Starlette Session Middleware

class STARLETTEProcessTimeMiddleware:

    app: ASGIApp

    def __init__(
            self,
            app: ASGIApp,
    ) -> None:
        self.app = app

    async def __call__(self, scope: Scope, receive: Receive, send: Send) -> None:
        if scope["type"] != "http":
            await self.app(scope, receive, send)
            return
        start_time = time.time()

        async def send_wrapper(message: Message) -> None:
            if message["type"] == "http.response.start":
                headers = MutableHeaders(scope=message)
                headers.append("X-Process-Time", str(time.time() - start_time))
            await send(message)

        await self.app(scope, receive, send_wrapper)

Test attribute	Test run 1	Test run 2	Test run 3	Average	Difference to baseline
Requests per second	1869.11	1891.52	1948.64	1903.09	-0.94 %
Time per request [ms]	53.501	52.868	51.318	52.5623	-0.5 ms

Observations

• Negligible change on performance
• Same function as FastAPI timing middleware, but better performance

With two Starlette middlewares

In order to see the performance difference if multiple middlewares are added, another one has been implemented and measured

Test attribute	Test run 1	Test run 2	Test run 3	Average	Difference to baseline
Requests per second	1832.51	1871.4	1916.97	1873.63	-2.47 %
Time per request [ms]	54.57	53.436	52.166	53.3907	-1.33 ms

Observations

• Still no significant difference, much better than BaseHTTPMiddleware

Verdict

Numbers clearly indicate the significant performance improvement between BaseHTTPMiddleware and Starlette middleware. Avoid using BaseHTTPMiddleware if you can