← back to home

Kraken

2023•19 views

A multi-threaded HTTP/1.1 server framework written in C with Express-like routing

Kraken Logo

I wanted to understand what happens between a browser sending a request and a server sending back a response, not at the framework level, but at the socket and thread level. So I built an HTTP server in C, on top of Owl, a small utility library I wrote alongside it.

There's a live demo served by Kraken itself. It shows your request exactly as the server read it off the socket, how it was parsed, and live stats from the server process. It runs on AWS Lambda through the Lambda Web Adapter, so the server scales to zero and costs nothing while idle.

How it handles requests

The main thread does nothing but accept connections. It sits in an accept() loop, and for each new socket it wraps the connection in a task and hands it to a thread pool of 20 workers, then goes straight back to accepting. No parsing or responding happens on the main thread, so a slow request can't stall the listener, and there's no thread-per-connection blowup: at most 20 requests are handled at once, bounded by the pool rather than by load.

A worker then takes the connection through the full request lifecycle:

  1. it reads off the socket until the headers and the full Content-Length body have arrived, up to 64 KB, and gives up on a connection that sits idle for 10 seconds so a slow client can't pin a worker;
  2. it parses the request line into method, URI, and version, URL-decodes the path and splits the query string into a hashmap of parameters, and parses the header block into a hashmap with case-insensitive lookup (SipHash, a keyed hash that stays fast and resists hash-flooding from attacker-chosen header names);
  3. it looks up the URI in the route hashmap, an O(1) hit that returns the handler;
  4. it runs the handler, builds the status line and headers, and writes the response back.

Responses go out in 4KB chunks rather than a single send(). For static files that's what keeps memory flat: the worker streams the file off disk a chunk at a time instead of reading the whole thing in, so the size it can serve is bounded by disk, not RAM. Each connection is closed once its response is sent.

Routing and templating

I modeled the API after Express: you register a route as a path plus a handler, and the framework owns parsing, matching, and response building. It felt like the right level of abstraction for a C server: familiar enough to be ergonomic, thin enough to leave the request lifecycle fully visible.

http_server_t *server = http_server_init(PORT, BACKLOG);
register_route(server, "/", index_handler);
register_static(server, "public");
http_server_listen(server);

A handler can return a string directly, build one printf-style with res_sendf, set its own headers with res_header, or render an HTML file through a small templating pass that swaps {{variable}} placeholders for values, enough for dynamic pages without pulling in a real template engine. For anything static, the server resolves the path under a registered directory and sets the Content-Type from the file extension, with the common web types (HTML, CSS, JS, images, fonts, media) mapped out.

Working in C

Writing the server in C means owning the details a higher-level runtime hides. Every request allocates its request and response objects and frees them when the worker is done, the socket is closed once the response is out, and a Ctrl-C runs through a signal handler that tears down the thread pool and frees the route table and status-code map instead of leaking them on exit.

The server speaks HTTP/1.1 and closes the connection after each response, so there's no keep-alive, and there's no TLS. Routing matches exact paths only, with no route parameters like /users/:id and no method-specific routes. These are all natural next steps, but request parsing, the thread-pool concurrency, and the connection lifecycle were the parts I wanted to get right.

kudos