Founder. Engineer. AI researcher.

An LLM gateway that streams, fails over and keeps the receipts

One OpenAI-compatible endpoint in front of OpenAI, Anthropic, Gemini, Mistral and MiniMax, covering chat, voice, image and video.

How it works

A Rust proxy (Axum on Tokio) forwards provider bytes as they arrive, so streaming stays streaming. Each key has a provider chain with circuit breakers, a spend ceiling and a record of every attempt. Failovers and recoveries go out as signed webhooks and log drains to Datadog, Splunk or OpenTelemetry.

The hard part

The worst failures return HTTP 200. So the gateway checks the answer itself, flagging a conversation that no longer matches what was stored, or tool-call arguments that are not valid JSON.

Rust · Axum · Tokio · Streaming proxies · Circuit breakers · Webhooks · OpenTelemetry

Stateful failover at 100 concurrent sessions

Carrying the whole conversation to the fallback provider, so the user never has to start over.

How it works

On a timeout, 429 or 5xx, the request is replayed on the next provider with its full history. To measure it, a fault injector fires failures on the turn that depends on earlier context: 750 failover events per system, scored by a calibrated model judge.

The hard part

At 100 concurrent sessions, parallel tasks mutated a shared history cache between async yields and conversations bled into each other. Copying each message array before yielding fixed it. Then 100 sessions hit a fallback allowing 15 requests a minute, retried in lockstep and ran out of sockets, until backoff with jitter broke the sync.

Python · asyncio · Fault injection · Backoff with jitter · LLM-as-judge evaluation

A creative pipeline that checks its own claims

Understand, create, check, test, learn: a loop that turns product context into video and demos.

How it works

It reads a website, release or brief and works out the audience, what is true and what must not be said, before generating anything. Output stays as editable scenes and layers, and it can watch GitHub, Linear or a changelog to draft or update videos when something ships.

The hard part

Trusting the output. Every on-screen claim is traced to a source. Deterministic checks for readability, composition, brand and playback run first, and an AI review steps in only when judgment is needed. Variants change one thing at a time against a goal you set.

Context extraction · Claim verification · Layered video composition · A/B testing · PostHog, Segment, Amplitude

Instruments for looking inside transformers

Measurement pipelines and kernels behind the interpretability and attention papers.

How it works

For trajectory geometry, hidden states are mean-pooled per layer and treated as a path, with length, curvature and convergence computed in the full ambient space across GPT-2, TinyLlama and Qwen2.5. For VLA, the memory update is an online regularised least-squares step, kept current with Sherman-Morrison rank-1 updates.

The hard part

Trusting the numbers. Every geometric effect is checked against shuffled-layer and random-embedding controls. For VLA, a Triton-fused kernel runs 14× faster than the sequential loop and drops below softmax attention latency at around 43,000 tokens.

Python · Transformer hidden states · Triton · Representation geometry · Statistical controls

The failures that answer 200 OK.

Why the most severe gateway failures look healthy on every dashboard, and the two we found ourselves.

An outage your dashboard never sees.

How we measured stateful failover, and the two concurrency failures that broke the first prototype.

A forward pass has a shape.

Measuring the path a prompt takes through a model instead of inspecting one layer at a time.

Long-context memory is a geometry problem.

Why a scalar decay cannot fix linear attention, and what replacing it with a matrix does.

Notes archive

Short notes on television and books from an earlier version of this site, alongside my reading and watching lists.