VPN Musings
What's Up?
I have been self-hosting stuff for years now. It started with Nextcloud once upon a time but by now there are a whole bunch of really useful services that run on my personal server. Said server runs in my house behind a completely normal, run-of-the-mill consumer-grade router with NAT. With this comes the need, or shall we say desire, to access my self-hosted things when I'm not at home.
One possible solution is to set up port forwarding on my router and route all incoming requests to my server. I've done this before but, on one hand, this is inflexible as I can only forward traffic to a single device and, on the other hand, it exposes my services with login page and all to the whole internet. As time passed I wasn't comfortable with this anymore so I needed a different solution which is, you'll have guessed by now, a VPN.
There's a whole slew of VPN providers of various sorts nowadays and I myself have been a user of Tailscale, a popular mesh VPN provider, for a number of years now. It works very well, is simple to use and frequently adds useful new features but there's this little voice at the back of my head nagging me about it. You see, the Tailscale client is open source but the server software isn't. And while they introduced a feature called Tailnet lock to reduce the amount of trust you have to place in the provider, the fact remains that I'm putting power over my networking in someone else's hands. Ultimately, this might be a tradeoff worth making but I decided to explore some of the other options out there. There are many of these but I landed on a few that I took a closer look at. This article serves as a writeup of my thoughts about each of these and what my actual needs are, these may be helpful to others too, who knows?
First of all, let's think about what I actually want and need. In decreasing order of importance these are what I need:
- Secure access to my server and my VPS
- Open source
- Self-hostable
- Low maintenance
- Ease of use
Tailscale (the service) ticks three of these boxes. It's secure since it's built on Wireguard, it's very little maintenance work for me and very easy to set up and use. Another half point for the client being open source, it's even on F-Droid.
Final score: 3.5/5
But what else is out there? The following are the options I selected for my shortlist, sometimes completely arbitrarily or simply because they appeared on my radar due to someone's recommendation.
Headscale
This seems like the most obvious choice. Headscale is the open source community reimplementation of the Tailscale server stack. So in principle it should be the perfect fit, right? Well, not quite. It's a community project and the main maintainer is employed by Tailscale by now (or so I hear) so I'm not sure what its future is gonna look like. Also, the project doesn't have feature parity with Tailscale proper and generally moves slower in terms of development because of more limited resources.
It also doesn't come with an official frontend of any kind so the canonical way to manage it is via command line and config files. I can absolutely manage that but I admit I like being able to configure things in a GUI and the web admin interface of Tailscale is quite nice to use. There are unofficial community projects to add a frontend but since they are separate projects, I'd always have to be afraid that something breaks or the integration doesn't work. I'd probably end up going vanilla, just to be sure.
On the other hand Headscale seems quite easy to set up, judging from the documentation, and not very resource-hungry (I mean, I have like 10ish hosts connected to Tailscale at the moment) so that shouldn't be a problem to handle. The maintenance burden will probably be similar to Tailscale proper, once everything is set up. The adding and managing of new nodes might need a little bit more work but I expect this to be smooth enough after getting to know how it all works and there's not much fluctuation in my Tailnet anyway.
There's a caveat though: the project still relies on some of the infrastructure of Tailscale itself, namely the DERP servers to facilitate connections between nodes. This may or may not be a critical problem if or when Tailscale decides to enter the enshittification game. It's also self-hostable and comes with instructions to do this. What I like about this is that a web frontend is built-in so you can manage your network just like you would with the SaaS offering which distinguishes this project from Tailscale/Headscale.
Final score: 4/5
Netbird
I was recommended this by someone on Mastodon and have read good things about it. It's a SaaS offering like Tailscale but is completely open source. The feature set is similar, it's also built on Wireguard (which I take as a good sign in terms of security), that's all fine.
Sounds like a perfect fit, apart from having to migrate, then yes? Well, not quite. It seems like the Android app is subpar, I've read of problems with
it on several places. The client is version 0.1.7, that doesn't exactly spark confidence, to be honest.
The documentation recommends 1 vCPU and 2 GB of RAM for self-hosting which puts this outside of the capability of a cheapo one-of-each VPS I could get for 1 €. This is of course not their fault but I was hoping on spending as little coin as possible on this endeavour, especially since I only recently rented another VPS and I don't want this to be getting out of hand.
The final thing that puts me off a little is that Netbird apparently outsources their identity management to third parties, which means that, in order to authenticate to the service, you need to log in with a third-party identity provider. Tailscale does the same which is one reason why I was looking elsewhere (though it's not a big deal) because the least privacy-hostile provider I could choose was GitHub. Which is owned my Microsoft. Right. Anyway, Netbird does this and apparently, the self-hosted setup is wired the same way so you'd either have to rely on a third party for this or self-host an OIDC identity provider. I mean, I could definitely do this but usually these come with a lot of complexity and are yet another piece of a chain that simply must not break, if I am to have access to my stuff.
I'll have to deduct half a point each for ease of use and self-hostable because of third-party reliance and resource requirements.
Final score: 4/5
Nebula
This is something that Jim Salter from 2.5 Admins frequently mentions so I decided to take a look. It's definitely open source and self-hosting is the only option which is nice. However, there seems to be no frontend for this at all and it also seems to be more maintenance-intensive than the other options. It entails creating your own CA as well as a bunch of certs and keys that you have to manually manage and copy to the nodes you want to connect. The certs, even the root cert, will by default expire at some point so you have to rotate them regularly and all of that by hand. Sure, it's of course possible to script this or automate this with Ansible or some such but manual key and cert management is something of a red flag for me, sorry. I'll have to deduct two full points in terms of maintenance effort and ease of use.
Final score. 3/5
Plain Wireguard
Finally, there is plain Wireguard. It's actually fairly easy to set up a point-to-point connection with this (I've tried) which is nice. It also gives you flexibility to design a network architecture that fits your needs, be it peer-to-peer or hub-and-spokes or a mixture thereof. But that also means that you have to get your hands dirty and think about what you want, how you can achieve that and then get busy implementing this. There's much more dealing with actual networking than with he other solutions which tend to do the hard work for you. There are tools to help you deal with the manual steps, like install scripts such as this (thanks Paul!) or wg-easy which is a web frontend for managing your clients.
At the end of the day these tools can help but you'll still have to admin your own network and know what you're doing. Also, you have to make sure all your clients can find and talk to each other which may mean opening ports, punching holes in firewalls or creating a hub that everyone else goes through with all the implications these things have. But you are the one who has to make the choice of what to do and can probably shoot yourself in the foot while you're at it.
In terms of simplicity of the tech stack and resource overhead this option is by far the best. In terms of ease of use it definitely isn't. But of course that also depends on what you want. Plain Wireguard won't give me fancy features like MagicDNS, ACLs, subnet routers, exit nodes, funnels, TLS certs for nodes and everything else out of the box. It may be possible to get them with reasonable effort but it means more elbow grease. Is it possible to get what I need out of this? Pretty sure it is. But it just means more work upfront and also more maintenance. I'll have to deduct points for this.
I'll say this in favor of this option though: Even though I haven't implemented any of this in production or even spent much time trying it out, I've learned a lot about networking, NAT and the challenges of a VPN service by reading up on all this.
Final score: 3/5
One might make the case for 3.5/5 because there are tools that make the ease of use aspect much better.
Final Thoughts
All of the above is quite subjective of course. Judging by the arbitrary point system I introduced here I should choose Netbird or Headscale but I knew this would be the result even before I wrote this because that was the way I was leaning anyway. I might end up just trying some options once I get around to breaking in a new VPS for it and see where things take me. Getting your feet wet with something usually is more telling than just reading up on it so the final decision will probably only be made once I have some more experience with the various options.
If you have some insight or opinion to share about this whole topic, please reach out to me! I'm always open to learning something new or having my mind changed in favor of something better than what I currently have and after all I still have so much to learn.
Cheers!