Home Lab Hardware 2.0 – New Hosts

    While working on some projects in my home lab recently, I’ve begun encountering resource shortages. Many of these shortages stem from my original intention to utilize repurposed desktop hardware. Although my lab no longer uses VMware, I still utilize it for work and read content, such as William Lam’s blog. While browsing his VCF9 content, I began to notice some relatively inexpensive machines that would meet my needs by providing more CPU and RAM for my workloads. The Minisforum MS-A2 immediately came to consideration due to its 16-core/32-thread AMD processor and the ability to support up to 96GB of RAM. However, shipping is relatively delayed, and I don’t need the smallest form factor available. Further research led me to the Minisforum 795S7, which has similar specifications but in a larger case.

    The 795S7 fits exactly what I was looking for in a lab machine. The processor and RAM mentioned above enable me to reduce the required number of hosts from 3 to 2, while still achieving greater power. Storage-wise, they have 2 NVMe slots each. Since I use a NAS for storage, the hosts only needed 1 256GB drive each. In the future, if I try to go back to a VMware stack, I could populate the second slot to try NVMe tiering. On the network side, the open PCI-e slot allowed me to swap in the 2.5G cards from my old hosts, enabling a second NIC for the storage network. This will also be important for a potential VMware swap, as the onboard hardware is Realtek, which lacks compatibility.

Host Hardware BOM (w/ Amazon Affiliate Links)

Racking and Network Topology

    Reducing the number of hosts in the network allowed me to reduce the size required for my lab. I ended up opening my laser software and designing a rack from 6mm MDF sheets, which I then spray-painted for the hosts to sit in, with both NAS devices sitting on top. The rack creates a bit of a hot/cold aisle situation, as it vents warm air out the back. Swapping the network connections was a one-for-one. Here is a diagram based on current connections-

Installation Process 

      Usually, migrating machines inside XCP-ng is as simple as adding the hosts to the pool and migrating the machines to the new host. In this case, it wasn’t so simple, as the processor family (Intel to AMD) was changing, meaning the hosts could still live in the same pool. I also wanted to reuse the same IP address for hosts. XCP provides the ability to complete a warm migration of the machines, essentially cloning the machine on the new hardware and removing the original when the process is complete.

To accomplish the migration, I took the following steps-

  1. Power down all VMs except the XO appliance machine
  2. Migrate all VMs to host 3 as this IP address would no longer be in use
  3. Change the pool master to host 3
  4. Power down and remove hosts 1 & 2 from the pool
  5. Install XCP-NG on the new hosts*
  6. Add the new hosts to the XO server
  7. Create a new pool for the hosts to reside in
  8. Migrate VMs onto the new pool*
  9. Power down and remove remaining old host from pool
  10. Pull drives and e-waste all hosts 

 Migration Notes and Wrap Up

 

The migration went relatively well, with a few notes and a complaint about XCP-ng. First, while prepping the new hosts, I had to work on some host UEFI issues-

  •  Disable host secure boot -  Setup > Security > Secure Boot > Set to "Disabled"
  • Disable the host Global C-State Control -  Setup > Advanced > AMD CBS > CPU Common Options > Global C-State Control set to "Disabled"

The next issue I encountered was with the migration of VMs. It was significantly slower than I would have liked, and I don’t have a great answer as to how to improve it. I was able to use the internal XO VM for all of them, except that machine. To migrate it, I ended up running a warm migration from the appliance installed in my Windows Subsystem for Linux (WSL) instance. This still ran slower than I had expected, but it ended up working. I also attempted to reattach disks; however, XCP does not store an equivalent of the a VMX file in the data store, meaning that to restore VMs with only the disk, you have to recreate them and attach the old disk. Overall, aside from taking longer than I would have liked, the process was straightforward, and I completed it in an afternoon.


Comments

Post a Comment

Popular posts from this blog

PowerShell Logging Part 2 — Rotating Your Logs

Image
  A few weeks ago, I wrote a post about using PowerShell for logging script outputs and mentioned there’d be a part two. So here we go, finally. PowerShell logging is great for keeping track of data, but log files can quickly get too big. This makes it hard to find what you need or manage storage. With a simple script, you can split logs into separate files when needed. $logPath = "c:\Scripts\NICPingCheck.log" #Check if old log files exist and delete them if (Test-Path "$logPath.old") { Remove-Item "$logPath.old" -Force Write-Host "Old log file removed." } else { Write-Host "No old log file found to remove." } #Rename the current log file to .old if (Test-Path $logPath) { Rename-Item -Path $logPath -NewName "$logPath.old" -Force Write-Host "Current log file renamed to .old." } The script starts by deleting any files ending in .old. Next, it renames the current log file to add .old, so you al...
Read more

PowerShell Logging — Recording the Output of Your Scripts

Image
PowerShell is one of the most powerful tools in an administrator's toolbox, and I use it daily. The more I do with PowerShell, the more I need to record my script and save my script output as reference or proof of the changes made. For example, when did that condition-based automation fire, and what did it do, or what data did you update from that data extract from HR against AD? PowerShell provides several options, and here are my favorites. Write-Host The simplest option is the "Write-Host" command, which returns the output to the command window. This is great for returning simple values; you can watch as the script runs. By including the variable after the write host command, you can print its value at that point in the code for reference- $timestamp = (Get-Date).ToString("yyyy-MM-dd HH:mm:ss") $message = "Logging Blog" Write-Host "$timestamp - $message" Typically, this wouldn't record the output anywhere other than i...
Read more