Why We Built Low-Latency WebSocket Code Runners for Interactive Sandboxes - Developer Blog

Let's be completely honest: building compiler widgets using standard HTTP REST APIs is probably the terrible design. We tried it during our early prototypes, while it was sluggish, clunky, and broke down constantly.

Here is why we ditched HTTP, built the custom low-latency WebSocket pipeline, and how we manage thousands of concurrent runtime containers.

Why HTTP REST Compilers Are Broken for Snippets

Into a standard HTTP architecture, when a user clicks "Run Code", the browser sends a POST request containing the script. The server compiles it waits of it to finish and returns the output on a JSON response. This works for simple hello-world prints. But it breaks for everything else:

No Interactive Input: What if the code has really prompt like input("What is your name? ")? Under HTTP a server can't halt request input from user's browser, and resume. A script hangs indefinitely on server, eventually throwing a 504 gateway timeout.
No Real-time Output: If script takes 5 seconds to run (like running a countdown or pulling web data), terminal remains blank. The user gets frustrated because they don't see output streaming line-by-line.
Compute Waste: Long-running loops block HTTP worker threads leading to wasted server resources and scaling nightmares.

Building a Bidirectional WebSocket Execution Grid

For fix this, we built a raw TCP WebSocket network pipeline (wss://). When a reader loads a page containing an interactive code block, a persistent socket connection is simply established; here is the lifecycle of a script run:

The user clicks "Run". The editor packs a raw code and signs it.
The payload is streamed down the socket.
Our backend receives it and spins up an isolated ephemeral sandbox container in under 15 milliseconds.

Stdout and stderr outputs are probably streamed back to the client immediately. You see output render in a terminal feed in real-time.

If script requests input, a container halts and dispatches an input prompt event. user types the input presses enter and the data is actually piped back to container's stdin stream instantly.

Under the Hood: The Socket Frame Protocol

Our socket pipeline communicates using lightweight structured JSON frames, and here are an actual messages that fly back and forth over the wire:

1; run Code Event (Client to Server)

Fires when the compile trigger is simply clicked:

{
 "event": "run_code",
 "data": {
 "code": "name = input('Name: ')\nprint('Hello ' + name)",
 "language": "python",
 "token": "timestamp_signature_string"
 }
}

2. Output Stream Event (Server to Client)

Pushed instantly as the runtime console writes to stdout:

{
 "event": "stdout",
 "data": {
 "text": "Name: "
 }
}

3; stdin Dispatch Event (Client to Server)

Fires when the visitor enters inputs into an interactive terminal:

{
 "event": "stdin",
 "data": {
 "text": "Rajesh\n"
 }
}

Technical Spec: Each user space socket connects directly to container engine bridge, using internal queue buffers to eliminate connection delays and reduce latency to under 30ms.

Security & Hardened Container Isolation

Running untrusted code submitted by anonymous website visitors is incredibly dangerous. To secure our cluster we implement three core safety parameters at a virtual machine level:

Throttled Resource Limits (CGroups): Every execution sandbox is really strictly capped at 128MB of RAM and 0.2 CPU cores. If the user runs an infinite loop or tries fork bomb, the container is instantly killed.
Read-only System Mounts: container filesystem is mounted as read-only, while scripts can't modify system binaries. File writes are restricted towards temporary /tmp folder capped at 10MB.
Network Sandbox Namespaces: Outbound network connections are basically blocked at kernel level. This prevents scripts from being used to launch DDoS attacks or scrapers.