The open-source stack I run an AI company on — for $0

We're an AI-native revenue-cycle company. Small team, pre-revenue in places, and no budget to spend on software. So the rule is simple: if there's a free, open-source tool that does the job, we self-host it. No card on file anywhere. No monthly SaaS creeping up on us.

People assume that means we run a toy setup. It doesn't. This week I added observability — real logs and metrics dashboards — and it cost nothing, because I ran it myself. That got me writing down the whole stack. Here's what's actually running, port numbers and all, and I'll be honest about the parts that don't work yet.

The layer most people pay for: observability

When something breaks at 2am, you want to see logs and metrics in one place, not SSH into five boxes. The usual answer is Datadog or New Relic, and the bill grows with your traffic. The open-source answer I landed on is OpenObserve — a single Rust binary that does logs, metrics, and traces with a clean UI.

I run it on demand rather than around the clock, because this laptop is tight on RAM and I'd rather not cook it. When I need it, one command brings it up:

# one binary, no Docker, no cluster
./openobserve.exe        # → http://localhost:5080

That's the whole install. It ingests logs over a plain HTTP API, so I can pipe our process-manager and bot logs straight into it. When we free up the always-on server, this moves there and runs full-time. Cost so far: zero.

The AI layer: one gateway, many models

We don't marry a single model provider. Prices change, rate limits bite, and a model that's great today gets beaten next month. So everything AI in the company talks to one gateway — OmniRoute, running on localhost:20128 — and the gateway decides where the request actually goes: Groq for speed, OpenRouter for range, Gemini for long context and its free tier.

The point is that no product hard-codes a provider. If one goes down or gets expensive, I change one config, not ten codebases. Under that sits Ollama with llama.cpp for models that run fully local — no API, no per-token cost, no data leaving the machine. That last part matters in healthcare.

Voice: text to speech, speech to text, and cloning

Voice is where "free open-source" surprises people the most.

• edge-tts gives us a natural voice out of the box — it's what our assistant "Maya" speaks with.
• faster-whisper turns speech into text — meeting audio, call recordings — accurately and offline.
• pocket-tts (from Kyutai) is the one that raises eyebrows: a 100M-parameter model that clones a voice and runs on a plain CPU. On this laptop, no GPU, it generates at 1.48× real-time. So a script can be read out in a cloned voice with no cloud, no subscription.

Put those together and you have a voice pipeline — listen, think, speak — assembled entirely from free parts.

Video: a live screen becomes an ad for $0

We make short product videos without a designer or a paid tool. The pipeline captures the real product on screen, and ffmpeg — the Swiss-army knife that half the internet's video quietly runs on — cuts it into a clip. For heavier generative video, the job runs on free GitHub Actions minutes against open models (LTX, Wan) on Hugging Face, and the finished mp4 comes back. No render farm, no rupee spent.

Data and the rest of the company

The lead engine, Storm, holds around 229,000 leads in MongoDB. The internal command center runs on Supabase's free tier. Meeting notes come from a native transcript path, with a self-hosted bot (vexa) as the backup. None of these has a paid plan behind it.

The honest part: what doesn't work yet

I'd be lying if I said it's all wired and humming. Two real gaps:

Talking-head video. A believable video of a person speaking — the "UGC" everyone wants — is where open-source is still weak. The free options (SadTalker and its cousins) look rough. The clean version needs a paid tool. So for now, a cloned voice over real screen recordings is our honest ceiling, and I'll say so rather than pass off something fake.

Social publishing. There's no magic free tool that auto-posts you everywhere. Self-hosted schedulers exist, but every platform still gates its posting API — LinkedIn needs an approved partner app, Meta needs a reviewed business account. The tool is the easy half; getting each platform's blessing is the real work, and I'm in the middle of it.

That's the whole picture. A serious AI company can run on free, open-source software today — not a stripped-down version of one, the real thing. What it costs you instead of money is the wiring: cloning, self-hosting, and connecting the parts yourself. For a founder with more time than cash, that's a trade worth making.

References

The projects, if you want to run them yourself:
· OpenObserve — logs, metrics, traces in one binary
· Ollama + llama.cpp — local model serving
· edge-tts · faster-whisper · pocket-tts
· FFmpeg — the video workhorse
· The full map of what we clone and why lives on my stack page.