In this tutorial we are creating a Proxmox cluster with two physical servers, node1 and node2, and we will add a RaspberryPi as a third Quorum device to make the Proxmox cluster working as intended in High Availability mode with having an odd number of cluster votes. This gives smaller business or a homelab user the possibility to use the full functions of proxmox cluster, but without the huge expanses of three physical servers. You may also use 2 different servers for the cluster, so I for example have a powerful big one as my main node1, and run a minor, lightweight old server as node2 fallback cluster. This way I can have High Availability on my most important VMs like DNS service, pi hole, reverse proxy, unifi controller and more. The high demanding VMs like Plex are just not set for HA migration so the smaller node does not get overloaded. The third Quorum device can also really be anything that can run linux or ubuntu, but should be constantly running. I use a raspberrypi I have running already 24/7.
First we will create a Proxmox Cluster with the two proxmox servers we have. Make sure that node1 and node2 are installed properly, and especially the node you want to join into a cluster, in our case node2 should be a fresh and untouched proxmox install. I am running right now both cluster nodes on version 6.2.11.
Create a Proxmox Cluster with two nodes
To create a cluster login to the web gui of your MAIN cluster, node1 for us, and click on "Datacenter" (step 1 in screen), and there on "Cluster" (step 2 in screen). Then click "Create Cluster" to create the cluster (step 3 in screen).
This will open up a popup where you have to put in your desired name for the cluster and select your LAN device for Cluster communication. This should always be a dedicated LAN device, in an own isolated cluster communication sub-network and not the LAN interface you are running everything else over. Anyway for our use case we just use the one we have:
Then click "Create" and your cluster will be created! This should bring you back to the cluster overview page that shows now that we have one node as part of our cluster:
Next click "Join Information" on the cluster overview page and it will bring up a popup with all the informations you need to join the second node, node2, to the cluster. For easier joining you can just click "Copy information" and go over and login on your node2 web gui:
Now, this happens on your EMPTY node2, that we want to join to the recently created cluster. In the Webinterface go to "Datacenter" -> "Cluster" and than hit "Join Cluster" (see screenshot).
This will bring up the below popup where you copy & paste the key information you just copied from the node1 Join Information tab. It will auto-populate the tab with the infos form cluster 1 and you only need to enter the admin / root password for user on node2 and select the LAN interface over which node2 should communicate with node1. Again this should be a dedicated exclusive network just for backend communication, but in this use case we just use the one Interface we have (see screenshot below). Then click "JOIN Cluster".
Now you will loose connection to node2 on the webinterface. Switch back to node1 webinterface and there you will see node2 very quickly popping up. Now the cluster list on our main node shows us two devices, node1 (master) and node2 that just joined (slave).
Congratulations, we have the first part of our cluster setup, now lets add a third device as the Cluster Quorum to give it odd vote possibility.
Setup third device as Quorum vote
ssh to the Main node1 proxmox server and see the amount of cluster votes with:
pvecm status
As we have just setup our two device cluster, you should see something like this:
The above shows us that we have two nodes and two votes running. But we are still missing a third device for full High Availability status, so lets go ahead and install this first:
apt update && apt install corosync-qdevice
Great! Now we have to do the same on our second cluster node. ssh into the second cluster node, node2 and do this:
apt update && apt install corosync-qdevice
Perfect, now we have to also configure our third "fake" cluster node, in my case a 24/7 running raspberry pi, thats taking the place of node3. ssh into your selceted Quorum host and run as root:-
sudo su
apt update
apt install corosync-qnetd
apt install corosync-qdevice
Now its time to join our raspberry pi as a third quorum device to our proxmox cluster. Go back to your main node and install the quorum sync certificate on all three devices. PLEASE NOTE: the IP I use below needs to be the IP of your third quorum device! We also use the FORCE flag "-f" to ensure that we overwrite any old quorum configuration that might still exist on our third device.
pvecm qdevice setup 192.10.10.5 -f
It will ask you for accepting the fingerprint of your third device and than for the ssh password of the root user for that device as well. Than it runs the script below and should finish with success!
To validate the success of the third quorum device enter:
pvecm status
The output below shows now that our Cluster is up and running with two nodes and three vote devices for the quorum!
Great! Now you have a Two-Node High Availability Proxmox cluster and you can start setting up High Availability rules now for all your virtual machines!
Sources for further references:
https://pve.proxmox.com/wiki/Two-Node_High_Availability_Cluster