I'm rather excited about this post as I get to talk about something I rarely get to share in any depth. My home network is something I've spent the last few years adding to and recently it reached a point when I realised "Hey, I can actually do something cool here". I've wanted to have a setup that I can actually trust and depend on. Today, I'm going to walk you through it.
My desk's setup has taken me several years to get to where it is today. I'm glad I stuck with it as it is really enjoyable to use and is frankly, really cool to own. The extent of my interest in technology isn't really limited at all, I mess around with basically anything I can get my virtual hands on. 3D design, video editing, development, gaming, content creation (YouTube) and so on. As such, I wanted a computer that could handle anything I throw at it.
feat. futuristic dentist chair - a chair that I, as a tall person, can comfortably sit in all day
I care greatly about quality and try to buy the best I can justifiably afford. There's next to nothing I own that I regret buying and I have virtually no problems with any of it. It's not been cheap, but it's something I regularly spend many, many hours in a row using, so for me, it's more than worth every penny. It's both my job and my hobby, and for me, it's actually a rather cheap (and for me, the best) way to learn. Anyway, let's talk specs.
The computer itself has the following:
- CPU: Intel i7 4790k
- CPU Cooler: Corsair H100i
- Motherboard: MSI Z97S SLI Krait Edition
- RAM: 32GB Kingston Hyper-X Fury White DDR3 RAM
- GPU: EVGA GTX 1080 FTW
- SSD: 500GB Samsung 850 Evo
- HDD: 3TB Toshiba P300
- NIC: Intel dual gigabit something-or-other
- Power Supply: Corsair HX750i (750 watt)
- Case: Corsair H440 White/Black
It's pretty powerful and rather different from when I first built it. I've added 16GB more RAM, went from a 960 to a 1080, upgraded the SSD and added a HDD, swapped the case, and replaced the PSU (the old one died - RIP). Anyway, I've always viewed my computer as a machine that shouldn't be permanently assigned a task. By that I mean, I don't want to be running a web or media server, VM's, or anything else that isn't related to what I'm "currently" doing. I want to be able to restart or reinstall Windows, render a video, play a game or whatever else, without having to interrupt (or sacrifice performance to) anything.
For quite a long while I was stuck with no alternative but to run my VM's on my computer. Yes, I had a couple in the "cloud", but the magical cloud isn't cheap for anything even remotely resource demanding. So, what was the solution?
Ages ago, I found a couple of HP DL380 G5's on eBay for £110 each. They were both fitted with two Xeon quad core X5450's, 32GB of DDR2 RAM and 4x 146GB SAS drives. It was such an incredibly cheap way to get relatively large quantities of computing power. There was a bit of a problem however, the G5's use an insane amount of power, way too much for me to be happy with it running for more than a few hours at a time. (I only ever used one, but I brought two, so I'd have parts for mine and ones that I managed for others - £110 for what is effectively an instant availability warranty? Not bad, eh?).
So, because of the power consumption I was stuck with using my computer for most things, but my data set and computational requirements were growing, and it was really becoming a problem. I decided to take a step towards rectifying this by purchasing a HP Gen 8 Microserver - a fantastic little box to which I added a few bits. It came with 4GB of RAM which I upgraded to 12GB straight away (ZFS - the file system FreeNAS uses - needs loads), I also threw in 4x 3TB WD Reds and installed FreeNAS. It wasn't intended to be used for computational workloads (so I haven't bothered replacing the Celeron G1610T) but instead one I could use to store data and also local backups. It was really exciting to get this online, I finally had something on my own network that I could "talk" to that wasn't temporary or ancient.
Cost around £120 for the stock Microserver. Not bad for such an impressive little box.
I had a few FreeNAS jails running on the NAS for a while, which definitely reduced what was running on my computer. Unfortunately, it's not a particularly powerful box and would never be able to run the VM's that I wanted. So, I decided it was time to solve this problem, I went onto eBay and found a HP DL380 G7, I paid just over £200 for it, it has two hex core Intel X5650's and 48GB DDR3 RAM. It was a great purchase, it averages around 130w with 4 drives installed - an amount I could live with running 24/7. CPU performance wise, it's not much worse than my desktop (according to Passmark) - and it was an awful lot cheaper. So, that's sat with all the other gear and is running VMware 6.5 - and is a place I can finally host my VM's.
Might be about due a clean... Not sure where all that dust came from all of a sudden. It's an annoying downside to not having a nice filtered server rack.
Wait a second, am I really comfortable with running VM's on a host that isn't backed by a UPS? The FreeNAS box doesn't have one either... Ahh! It's a lot of money that doesn't immediately "give" me anything - though it will hopefully give the servers time to shutdown gracefully when the power goes out. Well, it's really stupid not to have one, so I brought a CyberPower CP900PFCLCD from Amazon.
If it weren't so shiny I'd take a picture of mine in situ.
It's only a 540W UPS and £160 is an awful lot of money but in total, the equipment only uses around 200w, so it's what I need and it does the job nicely (and lets me graph power usage... more on that in a later post). UPS's are in a way a consumable as the batteries wear out relatively quickly, so they're not the sort of thing you can really save money on by getting second hand. It's annoying though, as you can spend nearly as much, if not more than you spent on the equipment they protect.
You may be getting the feeling that I don't particularly like using consumer grade equipment and you'd be right. It's awful stuff, especially when talking networking gear. So for this, I have a Ubiquiti USG (router) and UAP-Pro (wireless access point), a DrayTek Vigor 130 (modem) and 3, 8 port switches (one is PoE for the CCTV cameras). I really want to get all my cables routed to one place and use a single switch and it will hopefully be the next thing I do.
A rather old, low quality picture... I can't actually get to them to grab a photo currently.
We're currently using BT Business broadband and frankly, I'm disappointed with it. There was a considerable speed drop when swapping to them (70~ to 50~ download and 20 to 11~ upload) which they won't do anything about. But at least it's stable (though it was before...).
So, what does this equipment give me?
This setup is what I'd consider to be the minimum to let me do anything I'd reasonably want to do. I've got a server that can run VM's, a server that can store data (it's setup in RAID 10 which gives me 5.5~ usable TB's) and my desktop PC. The server currently runs a VM that hosts a Nextcloud instance and another VM that runs a bunch of Docker containers including Plex, GitLab and my network's primary DNS server. My desktop runs a VM which runs most of my monitoring applications, it runs Docker containers including InfluxDB, Grafana and my backup DNS server.
I also have two VM's in the "cloud", one is my blog server which also runs my public NTP server and the other runs Plesk which hosts a bunch of sites for myself and others.
This setup affords me a massive amount of computational power and flexibility. I can quickly setup experimental environments, but I can also run production ones. All of the VM's and my computer are backed up to the NAS and the cloud. Just three computers allow for such a great home lab environment. A personal computer, a storage server, and a compute server. I'm really quite proud of the fact that I've reached this point. Of course, you could do it all with one computer, but you have no redundancy and it's a pain to use and manage. I've got a setup which is very versatile but it's more than possible get a similar setup using second hand enterprise gear for just a few hundred from a site like eBay.
Well that was fun
I really enjoy being able to share things like this. I love reading other people's blogs and seeing what equipment they use. It's a great way to learn and although it can make you feel incredibly jealous, it's a great way to find something to strive for and to see what's possible. I've got a setup that I'm really happy with and it's taken me a LONG time (which I've spent every minute of being jealous of others), but it was definitely worth the wait. There is a bit of a problem though, it's an addictive hobby, and with new hardware being released all the time... I don't think I'll ever not long for something new.
My goal with this post was to not be so much "hey look what I've got" but more "hey, I had this problem/goal, and these are the reasons why I chose this solution". There's a real problem with a lot of what people "in the know" share and that is how easy it is to fall into the trap of showing what you did/to do but missing the why. I would love to spend more time going over each item individually and why I chose them - I spend a lot of time researching everything - but we'd be here all day. I will definitely be writing more in-depth stuff soon though. If you want to know more or would like to have a chat feel free to email me or leave a comment, I'd love to hear from you.
Short link: on-te.ch/mhn