Is it worth it to build a home cluster?
By the middle of the 1990’s there was a move away from shared memory computing towards distributed cluster computing, and now the internet is full of people creating their own custom cluster computers at home as a personal project or with a specific objective like: testing parallel programs, having your own web server or file storage system, mining bitcoins… But nowadays there are available powerful web services like Azure, AWS or Google Cloud on which we can run parallel programs, train CNNs, host web applications or anything we want. So the question is, is it worth it to create a home cluster from scratch instead of using these alternatives?
Well, in short, it depends; but the most likely option is that a cloud computing option is going to be the most feasible choice. Nowadays it’s no longer a question of whether you should move to the cloud or not, it’s a matter of which platform to use since there are many options available that also provide free trials for testing. Nevertheless it’s a very good project to create your own cluster as a research in the parallel computing field or just to have some fun, we’ll get to that later.
Comparing alternatives
The three big in the public cloud computing industry are: AWS, Azure and Google Cloud. With cloud computing, you are basically establishing a distributed architecture at a remote or virtual facility. You don’t need to buy any hardware or networking, instead, you can rent the processing power you need.
This article posted on 12/08/2019 shows an interesting comparison between these alternatives considering the resources that beginners or startups would need (the most basic pack). I’ve seen comments from people that use these alternatives to host personal web applications saying they were paying more that 150$ a month on average (including DB, virtual CPUs) and some of them were considering to build their own cluster to switch all their applications.
An easy option for AI developers
For AI developers there’s also Google Colab, Google’s free cloud service for deep learning applications. Google Colab works as a Jupyter environment and is already equipped with Python and popular libraries such as Keras, TensorFlow, PyTorch, and OpenCV; moreover it has the free option to use a GPU for your training experiments. This is a very good option for small deep learning experiments or research, but keep in mind that it has an idle cut-off so you need to be present for long tasks. Hardware specifications of the Google Colab:
- n1-highmem-2 instance
- Intel(R) Xeon(R) CPUs 1 core hyper threaded @ 2.3GHz, 45MB Cache
- ~13GB GDDR5 VRAM
- ~320GB Disk space
- idle cut-off 90 minutes (maximum 12 hours)
- GPU Tesla K80
For many people seeking better hardware specially in the field of deep learning is worth to consider investing in a good GPU like a GTX 1080 and some extra RAM for your desktop computer instead of going for the cluster approach. Although this may not be the best alternative for a startup since it has probably several developers and it’s a high investment to upgrade all of them.
Low cost cluster
If your needs doesn’t stick to what cloud computing services offer you, or you’re definitely going to build a cluster at your home no matter what, then let’s analyze the options.
In the top500.org, which lists the 500 most powerful commercially available computer systems known, the last place (as of nov. 2019) is for a 27.520 cores system that can give 1,142.0 TFlop/s of performance. This cluster uses nodes with a Xeon Platinum 8160 24C 2.1GHz CPU (4700$) and a NVIDIA Tesla V100 GPU ($10,664). This reveals that high performance cluster’s have incredible prices, not to mention the power consumption costs… You can theoretically build a cluster with any type of computer but let’s keep the numbers low and see what we can get.
Talking about getting the most with a low cost (only considering the cost of the hardware), here’s where single board computers are a very interesting option to consider. There are people building High Performance Parallel Personal Clusters with around €500 in off-the-shelf electronics and getting significantly faster results at computationally demanding problems than a single, fast desktop processor. Apart from this you need to consider the power consumption of all the machines you’d be running, not just the actual computers but also fans, switches, etc. For web application purposes getting a good and reliable ISP connection is going to add some additional cost to it.
The cheapest and most compact way to build a cluster that I’ve found is the Cluster HAT. This board is capable of holding up to 4 Pi Zeros using your own RPi as the main controller. You can get this board for ~34€, adding the cost of a Raspberry Pi 2 or 3 and four Raspberry Pi Zero boards.
Single board computers options
These are some of the single boards computers that I’ve seen and I have to say that I’m impressed with all the available options that are out there for big and small projects. In this table I’m focusing on the main aspects that are important for the cluster purpose.
| Board Name | CPU | Cores | GPU | Top RAM | Eth port | Max SD class | Power supply | Price |
|---|---|---|---|---|---|---|---|---|
| Arduino Mega 2560 | ATmega2560 | - | - | 256KB FlashROM | - | - | 7-12V | ~35€ |
| RPi 4B | BCM2711B0 1.5GHz | quad-core | Broadcom VideoCore VI | 4GB LPDDR4 SDRAM | Gb | 10 | 5.1V/3.0A | 59,95€ |
| Banana Pi M64 | Allwinner A64 1.2GHz | quad-core | Mali-400 MP2 | 2GB DDR3 | Gb | 10 | 5V/2A | 63€ |
| Tinker Board | RK3288 | quad-core | ARM Mali-T764 | 2GB DDR3 | Gb | U3 | 5V/3A | ~80€ |
| BeagleBoard-X15 | TI AM5728 | 2×1.5-GHz | DualSGX544 532MHz | 2GB DDR3L | 2xGb | 10 | 12V/5A | 239$ |
| Huawei HiKey 960 | Kirin 960 | 4x2.3GHz A73 and 4x1.8GHz A53 | Mali G71 MP8 | 3GB LPDDR4 SDRAM | - | 10 | 12V/2A | 299$ |
After a lot of research, the board that I would say its the best for a custom built cluster is the RPi4B with 4GB, although other options may have better hardware specifications, you cannot get anything better for this price. Also, one of the main reasons I’d choose this board for a project like this is the community. The RaspberryPi community has an impressive active number of people building awesome stuff. I encourage you to join the RaspberryPi community on Reddit.
One last discovery is this company that sells RPi4 packs ready to construct your own cluster, check it at Pico Cluster.
Conclusion
Building a cluster from scratch at home depends on what your needs are, but I’d say that for a single person, setting up your own cluster at home should be your last option to consider, unless you want to go for it as a personal project. Cloud computing service providers offer very good alternatives for anything a single person or a small startup would need 99% of the times. Nevertheless I have to say that It would be awesome to have a cluster at home, so I don’t discard it as a project for the future.
Some interesting projects
-
Blockchain Signaling System They used 18 Tinker Boards to use blockchain for signaling DDoS attacks in a cooperative and distributed network defense.
-
Wikipedia has a list of distributed computing projects.
-
Jeff Minard home cluster.
Written with 💙 by GuilleQP
Leave a comment and get a cookie 🍪!
What do you think? Are you going to build your own cluster?