How we built MailBench in Go

In the last post I said we’d write up the stack. This is that post. It covers how the SMTP server works, where the email goes once it arrives, and how we stop one noisy customer (or one bored attacker) from ruining it for everyone else.

Short version: MailBench is one Go binary, one Postgres database and a folder of files. That’s it. No queue, no Redis, no Kafka. Most of this post is about why that turned out to be enough, and the few places where it almost wasn’t.

The shape of the thing

MailBench does four jobs:

1. Accept email over SMTP (ports 587, 465 and 2525).
2. Store it, per team and per inbox.
3. Let people read it: a dashboard, a REST API, and an MCP server for AI editors.
4. Tell other systems when something arrives: webhooks, live updates in the browser, and a “wait for this email” endpoint for CI.

All four run in the same process. Each role can be switched off, but in practice we run two identical containers behind a load balancer and let both do everything.

The React dashboard is embedded in the binary with embed, so deploying MailBench means copying one file. That is very nice when you’re a small team and it’s 11pm.

The SMTP server

We didn’t write an SMTP parser. emersion/go-smtp handles the protocol: the command loop, pipelining, STARTTLS, size limits, dot-stuffing. You give it a Backend that creates a Session per connection, and the session gets callbacks: Auth, Mail, Rcpt, Data, Reset.

That interface turned out to be a perfect place to hang every rule we care about, because each SMTP command is a checkpoint:

Rejecting early matters. If a client is over its rate limit, telling it at MAIL FROM means we never read 25 MB of body we’re going to throw away.

No relay code. At all.

This was the most important design decision and it’s not even code. It’s the absence of code.

An email sandbox accepts mail for any address. That’s the point: your app sends a password reset to user-42@example.com and we catch it. But a server that accepts mail for any address and then delivers it is an open relay, and open relays get found by spammers within hours.

So the ingest package has no outbound SMTP client in it. The comment at the top of the package says so, and there’s a test called TestNotAnOpenRelay that connects without auth and tries to send to a Gmail address. It has to be refused before DATA. Then it logs in properly, sends to an outside address, and checks the message landed in our database instead of going anywhere.

func (s *session) Rcpt(to string, _ *smtp.RcptOptions) error {
	if s.inbox == nil {
		return smtp.ErrAuthRequired
	}
	// Any recipient is accepted: nothing is ever delivered onward.
	s.to = append(s.to, to)
	return nil
}

Authentication

Every inbox gets its own SMTP username and a random secret. We store the SHA-256 of the secret, not the secret, and compare with subtle.ConstantTimeCompare. A random 32-byte secret doesn’t need bcrypt: there’s nothing to brute-force in a dictionary, and SMTP logins happen on every connection, so a slow hash would be a slow server.

Unknown usernames still do a hash comparison against a dummy value, so “no such user” and “wrong password” take about the same time. And “workspace suspended” is only revealed after the password matched, so guessing usernames tells you nothing.

We support AUTH PLAIN and AUTH LOGIN. go-sasl only ships LOGIN as a client, so we wrote the server side ourselves. It’s about 25 lines, and worth it: .NET’s SmtpClient and some PHP setups won’t do PLAIN.

Reading DATA

Here’s the core of Data, trimmed a bit:

raw, err := io.ReadAll(r) // the server caps this at MaxMessageBytes

msg := s.buildMessage(raw, now) // parse MIME, pull out text, HTML, links

switch err := s.b.store.ReserveQuota(ctx, workspaceID, now, attachmentBytes); {
case errors.Is(err, store.ErrQuotaExceeded):
	return s.b.quotaError() // 552 5.2.2
case err != nil:
	return errTemporary // 451, the sender will retry
}

if err := s.b.blobs.Put(ctx, msg.BlobKey, raw); err != nil {
	return errTemporary
}
if err := s.b.store.SaveMessage(ctx, msg); err != nil {
	return errTemporary // the blob is orphaned, which is harmless
}
s.b.notify(ctx, msg)

Yes, we read the whole message into memory. go-smtp enforces the size cap while reading, so the worst case is 25 MB per connection, and connections are capped. We need the whole thing anyway to parse MIME and count attachment bytes before deciding whether to accept it.

The order matters too. Quota is reserved first, because it’s the cheapest thing to say no to. The raw file goes to disk second. The database row is written last. If the database write fails, we’re left with a file nobody points to. That’s harmless, and a daily sweep deletes it. The opposite order would leave a database row pointing at a file that doesn’t exist, which shows up as a broken email in someone’s inbox.

Reply codes matter as well. 4xx means “try again later”, and a good mail library will queue and retry. 5xx means “give up”. Rate limits and database hiccups get 4xx. An exhausted quota gets 5xx, so nobody’s app retries all month against a limit that won’t change until the 1st.

The MIME parser uses only the standard library (net/mail, mime/multipart, mime/quotedprintable), with a nesting limit of 12 and a link limit of 200 so a multipart bomb can’t run away with it. If parsing fails, we still store the email with the error attached. Broken mail is exactly what a sandbox is there to show you.

Storage

Postgres for everything that is queried, files for everything that isn’t

A captured email is two things: a raw .eml file (which can be megabytes) and a bunch of fields you want to search and list (subject, sender, recipients, received time, extracted bodies, links).

We split those. Postgres gets a row per message with the parsed fields. The raw bytes go into a blob store, keyed like this:

{workspace_id}/{YYYY}/{MM}/{DD}/{message_id}.eml

The blob store is an interface with three methods: Put, Get, Delete. Today it’s a directory on disk. The day we need S3 or R2, it’s one new type.

The filesystem version writes to a temp file and renames it into place, so a reader never sees half an email:

// Write then rename so readers never see a half-written file.
tmp, err := os.CreateTemp(filepath.Dir(path), ".tmp-*")
// ... write, close ...
return os.Rename(tmp.Name(), path)

It also refuses keys containing ... We generate the keys, so that should never happen, but “should never happen” is how path traversal bugs are born.

One transaction per email

Saving a message means inserting the row, applying tags, queueing a delivery for every matching webhook and notifying anyone waiting on the inbox. All of that is one Postgres transaction. The webhook queue is just a table (the “outbox pattern”), so a webhook fires if and only if the message was actually stored. No message, no webhook. No webhook lost because the process crashed between “saved” and “sent”.

The notification is a pg_notify inside the same transaction. Postgres only delivers a NOTIFY when the transaction commits, so nobody gets woken up for a row they can’t see yet.

The quota counter

Free workspaces get 2,500 emails a month. The counter has to be exact, and it has to hold up when 50 connections from someone’s test suite all hit DATA at the same moment.

We do it in one statement:

INSERT INTO workspace_usage (workspace_id, period, message_count)
SELECT w.id, $2, 1
FROM workspaces w
WHERE w.id = $1 AND COALESCE(w.monthly_limit, 2147483647) > 0
ON CONFLICT (workspace_id, period) DO UPDATE
SET message_count = workspace_usage.message_count + 1
WHERE workspace_usage.message_count <
      (SELECT COALESCE(monthly_limit, 2147483647) FROM workspaces WHERE id = $1)
RETURNING message_count

ON CONFLICT ... DO UPDATE takes a row lock, so concurrent sessions line up behind each other. The WHERE on the update means that once you’re at the limit, the statement updates zero rows and returns nothing. No row back means “quota exceeded”. There’s no read-then-write gap for two connections to slip through, and no application lock.

The period is the first day of the month in UTC, so a new month is just a new row. Nobody has to run a “reset counters” job at midnight.

Workers without a queue

Webhooks need retries, which usually means a job queue. We used Postgres here too:

UPDATE webhook_deliveries d
SET attempts = d.attempts + 1,
    next_attempt_at = now() + make_interval(secs => $2)
WHERE d.id IN (
	SELECT id FROM webhook_deliveries
	WHERE status = 'pending' AND next_attempt_at <= now()
	ORDER BY next_attempt_at
	LIMIT $1
	FOR UPDATE SKIP LOCKED)
RETURNING ...

SKIP LOCKED lets both containers pull jobs without fighting over the same rows. Instead of holding a transaction open during the HTTP call, claiming a job pushes next_attempt_at forward. That’s a lease. If a worker dies halfway, the lease runs out and the job comes back by itself.

The same pattern runs our other background jobs. At our volume, a separate queue would be one more thing to deploy and back up, for no real gain.

Retention

A sweeper in the same binary deletes old mail in small batches to keep transactions short. Rows go first, files second, for the same reason as before.

Once a day it also walks the blob directory looking for orphans: files over an hour old whose message ID has no row pointing at them. That catches leftovers from failed saves and failed deletes, so “deleted” really means deleted, backups included over time. It only touches files that match the ws/YYYY/MM/DD/msg_….eml pattern, so a mistyped BLOB_DIR can’t turn it into rm -rf.

“Wait for this email”: the endpoint CI actually uses

The endpoint CI cares about most is this one:

GET /v1/inboxes/{id}/messages/wait?to=user-42@example.com&timeout=20

Your test triggers a signup, then calls this, and it returns the moment the email lands. No sleep(5) and hoping.

Behind it is a tiny in-process pub/sub we call the hub. The handler:

1. Subscribes to the inbox first,
2. then queries the database,
3. and if nothing matches, waits for a wake-up, a one-second poll tick, or the timeout.

Subscribing before the first query matters. If you query first and subscribe second, an email that arrives in between is missed and the test sits there until it times out. That’s a flaky test you’d chase for weeks.

Since there are two containers, an email received on container A has to wake a waiter on container B. That’s what the pg_notify from the save transaction is for: each container LISTENs and passes notifications into its local hub. The one-second poll is a safety net in case the listener connection drops.

One detail we got right early: the notification carries the recipient list. Picture 40 parallel tests waiting on one inbox, each for its own address. If every email woke all 40, each email would cost 40 database queries. With the recipients attached, a waiter only wakes up if the email could be for it. Postgres caps NOTIFY payloads at 8,000 bytes, so if the recipient list is huge we drop it and fall back to waking everyone. Correct first, efficient when we can.

select {
case s.ch <- struct{}{}:
default: // already signalled
}

Each subscriber’s channel has a buffer of one, and publishing never blocks. A slow waiter just gets one merged signal and queries again, so it can never hold up email delivery. The dashboard’s live inbox uses the same hub.

Throttling

A public SMTP server attracts everything: credential stuffing, connection floods, a misconfigured cron job sending the same email in a loop. We throttle in layers, cheapest check first.

Layer 1: the firewall

Before Go sees anything, iptables with hashlimit caps new SMTP connections per source IP at 30 a minute, with a burst of 60. That stops dumb floods without spending a goroutine on them. HAProxy’s stick tables are a second line behind it.

Layer 2: connection caps

The listener is wrapped in a small type that counts open connections per IP and in total: 20 per IP and 1,000 overall by default. A connection over the cap gets a proper SMTP greeting, 421 4.7.0 Too many connections, try again later, and is closed. A polite reply matters, because a real mail library will back off and retry, while a hard close often makes it retry immediately.

func (l *limitListener) Accept() (net.Conn, error) {
	for {
		c, err := l.Listener.Accept()
		if err != nil {
			return nil, err
		}
		ip := remoteIP(c)
		if ok, _ := l.gate.AcquireReason(ip); ok {
			return &limitConn{Conn: c, release: func() { l.gate.Release(ip) }}, nil
		}
		c.SetWriteDeadline(time.Now().Add(time.Second))
		io.WriteString(c, l.reject)
		c.Close()
	}
}

The release runs in a sync.Once, so a connection closed twice can’t free two slots. Together with 60-second command timeouts, a slow client can’t pin down sockets and goroutines forever.

Layer 3: failed logins

Each IP gets 10 failed AUTH attempts a minute. The trick is that we only spend a token on failure, and we check before even looking at the password:

if s.b.authFails.Exhausted(s.remoteIP) {
	return errTooManyAuthFailures
}
// ... look up credentials ...
if errors.Is(err, store.ErrInvalidCredentials) {
	s.b.authFails.Allow(s.remoteIP) // spend one of this IP's failure tokens
	return smtp.ErrAuthFailed
}

A correct login costs nothing, so a busy CI server sharing an IP with lots of jobs is never locked out by its own success. Someone guessing passwords gets 10 tries, then a wall.

Layer 4: per-inbox sending rate

Each inbox can accept 10 messages a second on average, with a burst of 50. This is checked at MAIL FROM, before any body is sent, and the reply is a 450, so the sender’s library queues and retries.

The burst is generous on purpose. Test suites fire 30 emails in two seconds and then nothing for ten minutes. A big burst with a modest refill handles that and still stops runaway loops.

The limiter itself

All of these use one small token-bucket limiter we wrote ourselves, about 100 lines. A map from key (IP, inbox ID, workspace) to a bucket with a token count and a timestamp, behind one mutex:

func (l *Limiter) refill(key string, create bool, rate, burst float64) *bucket {
	now := l.now()
	b, ok := l.buckets[key]
	if !ok {
		if !create {
			return nil
		}
		b = &bucket{tokens: burst, last: now}
		l.buckets[key] = b
		return b
	}
	b.tokens = math.Min(burst, b.tokens+now.Sub(b.last).Seconds()*rate)
	b.last = now
	return b
}

No goroutine refills buckets; the refill is worked out from elapsed time whenever a key is used. A janitor drops buckets idle for 10 minutes (they’d be full anyway), so the map doesn’t grow with every IP that ever touched us.

Why not golang.org/x/time/rate? It’s one limiter per key, and we needed per-key rates (limits depend on the plan), checking without spending (the auth case above) and idle cleanup. Wrapping it would have been about as much code as writing the bucket.

“One mutex for everything” sounds like it won’t scale, but the critical section is a map lookup and some float math. If it ever shows up in a profile, sharding the map is an afternoon’s work.

What about two containers?

The limiters live in memory, so each container has its own. Behind a load balancer, an inbox could in theory get double its rate. We decided that’s fine. These limits protect the service from abuse; they’re not billing. The things that are billing, like the monthly quota and the daily API cap for free keys, are counted in Postgres and are exact.

Getting the real client IP

Every per-IP limit above is useless if every connection seems to come from the load balancer. HAProxy sits in front of the SMTP ports and passes the real client address with the PROXY protocol (v2).

There’s a catch. If you accept a PROXY header from anyone, anyone can claim to be any IP and dodge every limit. So the listener only requires the header from the load balancer’s addresses, and rejects it from everyone else. And the PROXY wrapper has to sit underneath the connection limiter, or every client would share HAProxy’s address and the 20-per-IP cap would lock everyone out at once.

Deploys and the boring bits

A few smaller things that made one binary work in production:

• TLS certificates reload without a restart. Let’s Encrypt certs last 90 days. The SMTP listener checks the cert files’ modification time every minute and swaps in the new pair. Without that, every renewal needs a restart, and a forgotten restart means an expired cert on port 465.
• Graceful draining. On shutdown, the process first starts failing /healthz, waits for HAProxy and Caddy to notice and move new connections to the other container, and only then stops listening. Rolling deploys don’t turn new connections away.
• Logs never include subjects. Test email often carries real names and one-time codes.

What we’d do differently

Not much on the core. A few notes:

• Streaming DATA to disk would cut peak memory. It hasn’t mattered yet, but it’s the first thing we’d change if attachments got bigger.
• Bodies in Postgres. We store the extracted text and HTML in the message row so search and the API are fast. It makes the table bigger than it would be with metadata only. Worth it so far, but it’s the one storage choice we argue about.
• The orphan sweep walks every file. Fine for a directory on disk at our size. Once we move to S3, listing millions of objects every day would get slow and cost money, so we’d only list recent dates.

Why Go, again

Go fit for specific reasons, not just “it’s fast”:

• A goroutine per connection is the natural way to write an SMTP server, and in Go it’s cheap.
• net.Listener is an interface, so the connection limiter and PROXY protocol handling are small wrappers that are easy to test.
• The standard library covered MIME, TLS, HTTP and embedding the UI. A short, boring dependency list is what you want on the open internet.

Try it

The free plan includes 2,500 emails a month, team support and MCP access. Point your app’s SMTP at MailBench, send something, and see it show up.

Sign up

If you have questions about any of this (or think we got something wrong), leave a comment. I’ll answer every one.