About 8 months ago, I set out to create a custom mailserver. Having bigger and bigger privacy and security concerns, open-source solutions I found felt janky, plus I am not a fan of terminals. And a paid solution — well, I can actually make it happen and learn in the process. Plus I just wanted to see how our data are actually treated.
So I picked up Zig, OpenSSL and started to see if it can become a reality or I went crazy.
Why Zig?
Even though I played with some languages, I would say I am new to systems programming. What I know is mostly from observation and reading. C scares me, C++ sounds like a mess, Rust has the reputation of being a headache — and Zig, well, it's one of the on-metal languages, praises itself on being readable and I like the branding.
Yes, Zig doesn't have a final version yet. I started on 0.14 and yes, it came to bite me. But after that time spent in the language, it's very pleasant to work with.
Mailserver Basics - Those Protocols Older Than Time Itself
SMTP, IMAP, STARTTLS, TLS — what port expects what and how — that was a steep learning curve. But once I set up first port bindings and OpenSSL's TLS, progressive implementation went quite fast.
Luckily I was hyped at that time, because I had NO IDEA how chatty those mail protocols are. And how large. I kept adding support for different commands until the very end.
Because mail clients arbitrarily create and maintain or close connections, I quickly realized that threading is not optional for performance later — it's necessary right now. And Zig had only std.Thread, but no async at the time.
I learned a huge amount about data transport. At some point, when I couldn't get messages loaded, I even got Wireshark and watched data flow between our current mail client and the mailserver to find how the flow looks and found a bug in how I create the email envelope.
It was very cool to understand how the data flows and how it is handled.
For example, I never even thought that searching in a mail client actually calls SEARCH on the server, or that emails are not really deleted until the mail client decides to EXPUNGE them. These are the reasons I have a hard time trusting software with sensitive personal and company data handling.
How To Store Data #1
Basic protocols work. Time to store it. I've heard so much good about SQLite. The idea of one file is awesome. Very refreshing when coming from web dev and MySQL.
So I tried to put in mailboxes and load emails and send emails and append emails — and it randomly started to error out and mix some data.
I was trying to resolve concurrency, but I just couldn't. No flag, no setting worked. No advice on the internet helped. I tried to live with it — I'll get back to it later. It works, somehow, sometimes. Let's not get stuck here.
Antispam, Verifications and DKIM
This is definitely the least exciting piece of the entire project. And the most absurd.
For those who don't know — DKIM is a crypto signature included in the message header and compared to a public key. If anyone changed a byte (non-whitespace character if relaxed) in the message, it doesn't match. A reasonable security measure, but a pain to rebuild the signature. There is also SPF, which lives in DNS and confirms that the sender is approved to send a message on behalf of that domain. A pain because DNS lookups often loop into the heavens.
Setting up TLS took me about 2 weeks plus some issues to resolve along the way. But resolving DKIM? That thing is a tragedy.
simple, relaxed, relaxed/simple.
DKIM is broken and I hear "oh, something along the way tried to be helpful and re-wrapped the body." What the hell? How is that any metric then? Yet if you mess up DKIM, you have a good chance your mail will go to spam or just bounce.
I still don't understand the cryptography of all this — that is way beyond me.
But as a personal part of evaluation, there are machine learning layers. Far from what an LLM is, but still — do you think we can call it "AI Enhanced" in marketing?
DNS Fetching
But this — this was interesting.
I had no idea how DNS is actually seen, what form it has. I don't know what I was expecting. An HTTP request?
It is a simple specific protocol — just open a connection to a DNS resolver, like 1.1.1.1, send a very specific binary format as a request and get a very specific binary format as a response. With limits, so you may need to ask several times.
This was a fantastic lesson. Though not much to say — you call that thing, you get back a thing, you parse a thing.
How To Store Data #2
OK, I had enough of the errors. I need to start being more serious.
I know MySQL. I got MySQL Community Server, installed, connected as usual with HeidiSQL. Great.
Now what? My mind is severely skewed by using MySQL in PHP. It was so simple — a request, a response. Takes a long time there, but simple.
Oh, download the C lib and make a wrapper. Let's learn.
That wrapper actually worked — pretty cool. And now I know how much PHP makes a developer's life simpler.
Great choice, let's continue.
First VPS Test
It was time to put the whole thing onto a VPS and plug it into the internet.
Logging everything, printing everything. In literally 20 minutes, probes — I learned later it's what these bots are called — started to check those connections: 25, 465, 993, all of them.
And it started to break. I wasn't expecting, nor testing for, closures that don't look back and say "bye."
But it didn't take long — a couple of hours. It's just simple logic, some conditions, some slice read boundaries.
I tried to send messages from different providers — Outlook, Gmail, Proton. Every single one broke my program on some level.
This was a good test. I learned that the internet is extremely messy. Real emails are extremely messy. Not all providers play by the SMTP spec, and I had to go further into legacy than I expected.
Though after a few days, it was just running and doing its job on test data and bot traffic.
Do I even need to say that here had to come the first major refactor? The code was messy, all the patches, and of course I had several memory leaks. Zig is safe — not low-level-beginner-proof.
How To Store Data #3
MySQL did not work. I spent a month trying to fix it. It kept complaining about types, non-UTF characters — and even after all this time, after bringing in iconv (I was still on Windows at that time, so win-iconv it was), it kept doing the same thing.
"Fuck it! I'll do it myself."
Deleted MySQL, deleted the wrapper, and started thinking about how to approach it.
I certainly was not skilled at working with bytes. But by using Zig, it slowly crept into my mind that it won't be that crazy. It's just a byte, all the same. Interpretation is what matters, right?
It started with just a text file to which I put a number. Then a text. And it worked — I wrote and read bytes properly from positions.
That was the most excited I've been during the whole project so far. That was completely new to me.
So... I went to the gym to think about the format in which all of it must go into files.
Meta file and Data file were the first implemented. At the time, I thought it's enough. It was not enough.
The whole thing grew slowly. Before I finished one step, it was pretty clear I needed more — and that was either for data safety, security or performance.
These days GraceDb has segmentation, indexing, WAL, restoration, journaling and much more. It passed many many tests and integrates directly into the project with comptime schemas and checks.
By chasing bugs, I learned to read binary files. Whenever I saw a binary format before, I just closed it — it's nonsense, I can't see anything in it. Yeah, you can't see anything, unless you are looking for something specific — a pattern or a specific byte.
Recently, I was chasing why segmentation rolled up and then reverted. Which should never happen. My full screen was filled with GHex and different files. That is something I never expected to be doing.
Now I Am Scared
When working on my database, I realized something.
Everything is just files. All our precious data — emails, money transfers, all that hides behind walls like MySQL, PostgreSQL, MongoDB — no matter how locked the files are, one bad move and it might be gone. The world is so fragile in my eyes now.
Of course, everything has protections and replays and logs and backups. And my database has that too. And although not all databases are the same, now I feel like I can see them for what they are — wrappers around fragile files.
Besides that, so far I don't have threading problems, type problems — I set out to just put bytes into files, retrieve them, and do it as safely and as fast as possible. And I succeeded.
Did I Say I Don't Like Terminals?
Yeah, so the next step was logical — GUI with Vulkan.
I didn't need Vulkan (probably). I wanted Vulkan. Isn't that the cool toy? I knew it would require a large set up — and it did — but it gave me freedom.
For our projects, we use custom graphics design that I wanted to replicate. And it has some design complexity (blur, coverage, etc.). I learned from my database #1 - #3 experience and went straight for the more-setup-more-certainty approach right away.
At that time, I was on Linux dealing with Kubuntu being so slow on strong hardware and implemented X11 windowing and inputs. Later I learned Wayland is the thing to go, even with Nvidia these days, so I implemented Wayland windowing and inputs.
What I did not realize before I started was all the things I have to implement beside the graphics pipeline — inputs, redraws, clipboard, special windowing, even the icon. All that.
Windows implementation and full Metal coming soon. I can't wait, man, can't wait... (pain or annoyance, reader's pick)
But Where to Put It?
This was completely new to me.
First, I implemented it in a single program. Starting the mailserver opens the GUI.
Well, that was stupid.
I learned about Unix sockets on Linux and Named Pipes on Windows. OK, two programs now. I installed Xfce and VNC so both can run on the same server.
Much better. Both run independently. And the tiny VPS has enough resources for all of that. I was quite happy with how I optimized those things.
But there was one more iteration — the final one.
I learned about SSH tunneling over local sockets. This was the easiest thing in the entire project until now and counting. It was actually done in 30 minutes.
Right Now - Where I Am
Real testing is running right now and has been for some time.
At the start of this second test, it was the first time I saw a "thread leak". Everyone keeps talking about memory leaks, but this? I like the new Zig's std.Io. I might have been a little too eager and not careful enough with properly closing threads.
After the second major refactor, closed leaks of any sort (don't ask), measurements and testing — it is a stable, long-running program that is performant and efficient with resources.
In the end, this is a commercial product. Privacy, security and even price will win customers. It was never meant to be a toy project. Quality was the metric, a solid replacement — and I believe I succeeded.
All This Project Gave Me
It taught me that anything looking like a black box is just a few atomic elements cleverly handled — like files and databases.
Most things don't have to be complicated. In this project, the only real exception is cryptography.
I also learned how to implement AI coding into my workflow. It's not "do it." Any time I tried that out of exhaustion, it fucked me over. And even with specific instructions, it's just terrible sometimes.
I spent 7 hours chasing "Why the fuck did all the mails disappear — the files are full! Corrupted? Overwritten?"
When I asked an AI to add a specific tracer log line to wherever we send data to a TLS connection, it just decided to remove the actual sending to TLS and replaced it with tracing — and totally confused me, the mail clients, and itself.
There is no piece of code that I have not seen, and that's how it'll stay for now. It's not enough for me that it "looks like it works." I am the one responsible for data safety with clients who purchase, and it proved it's a tool that can fuck up at any time and gaslight you into looking elsewhere.
Finally, the project brought a new, modern, faster tech stack to the company. Event bus, transport layer, database, UI module, diagnostics, network connections — that's a lot of useful pieces to build on.
And it helped me finally get out of a burnout from dev work.