• 5 min read

Building a macOS Detection Engineering Lab — Part 3: macOS VMs on Proxmox

Running macOS on Proxmox opens up your detection lab to Intel server hardware. This is ideal if you want a dedicated always-on lab environment separate from your daily driver. **Legal note:** Running macOS on non-Apple hardware lives in a gray area (check Apple's EULA). For a *detection lab*, this is typically fine — you're not selling anything or running production workloads.

macos detection-lab detection-engineering

Prerequisites

  • Proxmox VE installed and working
  • Intel CPU with VT-x/VT-d (AMD works but requires more tweaking)
  • 16GB+ RAM available for the macOS VM
  • ~100GB storage for the VM
  • Patience

Step 1: Download macOS Recovery

On your Proxmox host:

cd /tmp
mkdir macos-recovery && cd macos-recovery

# Get Apple's recovery downloader
curl -O https://raw.githubusercontent.com/acidanthera/OpenCorePkg/master/Utilities/macrecovery/macrecovery.py

# Download Ventura recovery
python3 macrecovery.py -b Mac-4B682C642B45593E download

This downloads BaseSystem.dmg (~843MB) and BaseSystem.chunklist.

Step 2: Convert the DMG

Critical step — the downloaded DMG is zlib-compressed. Proxmox needs a raw image:

apt-get install -y dmg2img
dmg2img BaseSystem.dmg /var/lib/vz/images/100/Ventura-recovery.raw

The converted file will be ~3GB. Don’t skip this — a compressed DMG won’t boot!

Step 3: Get OpenCore

OpenCore is the bootloader that makes macOS work on non-Apple hardware.

Download a pre-configured image from thenickdude/KVM-Opencore:

# Download and extract
wget https://github.com/thenickdude/KVM-Opencore/releases/download/v21/OpenCore-v21.iso.gz
gunzip OpenCore-v21.iso.gz

# Convert to raw
qemu-img convert -f raw -O raw OpenCore-v21.iso /var/lib/vz/images/100/OpenCore.raw

Step 4: Create the VM

# Create VM with macOS-specific settings
qm create 100 --name macos-ventura --memory 8192 --cores 4 --cpu host \
  --net0 virtio,bridge=vmbr0 \
  --machine q35 --bios ovmf \
  --ostype other

# Add EFI disk
qm set 100 --efidisk0 local:0,efitype=4m,pre-enrolled-keys=0

# Add main disk (64GB)
qm set 100 --virtio0 local:64,cache=unsafe,discard=on,iothread=1

# Attach OpenCore and recovery
qm set 100 --ide2 local:100/OpenCore.raw,cache=unsafe
qm set 100 --ide0 local:100/Ventura-recovery.raw,cache=unsafe

# Boot order: OpenCore first
qm set 100 --boot order=ide2;ide0;virtio0

# macOS-specific QEMU args (the secret sauce)
qm set 100 --args '-device isa-applesmc,osk="ourhardworkbythesewordsguardedpleasedontsteal(c)AppleComputerInc" -smbios type=2 -device usb-kbd,bus=ehci.0,port=2 -global nec-usb-xhci.msi=off -global ICH9-LPC.acpi-pci-hotplug-with-bridge-support=off -cpu host,vendor=GenuineIntel,+invtsc,+hypervisor,kvm=on,vmware-cpuid-freq=on'

# Disable balloon (unsupported on macOS) and set VGA
qm set 100 --balloon 0
qm set 100 --vga vmware

Step 5: Network Configuration

The macOS installer downloads ~12GB from Apple, so your VM needs internet access.

If your bridge doesn’t have DHCP, configure a static IP in the macOS Recovery Terminal:

ifconfig en0 192.168.1.100 netmask 255.255.255.0 up
route add default 192.168.1.1
echo "nameserver 8.8.8.8" > /etc/resolv.conf

# Test connectivity
ping -c 2 8.8.8.8

Step 6: Install macOS

  1. Start VM and open the console in Proxmox
  2. OpenCore picker appears — select “macOS Base System”
  3. In Recovery:
    • Open Disk Utility
    • Select the VirtIO disk (~64GB)
    • Erase as APFS, GUID Partition Map
    • Name it “Macintosh HD” or whatever you prefer
    • Close Disk Utility
  4. Select “Reinstall macOS Ventura”
  5. Wait for download (~12GB from Apple)
  6. Multiple reboots — after each reboot, select “macOS Installer” in OpenCore
  7. Complete setup wizard when you reach the desktop

Step 7: Convert to qcow2 for Snapshots

Raw disks don’t support snapshots. Convert to qcow2 for easy rollback:

# Stop VM
qm stop 100

# Convert all disks
cd /var/lib/vz/images/100
qemu-img convert -f raw -O qcow2 vm-100-disk-0.raw vm-100-disk-0.qcow2
qemu-img convert -f raw -O qcow2 vm-100-disk-1.raw vm-100-disk-1.qcow2
qemu-img convert -f raw -O qcow2 OpenCore.raw OpenCore.qcow2
qemu-img convert -f raw -O qcow2 Ventura-recovery.raw Ventura-recovery.qcow2

# Update VM config
qm set 100 --efidisk0 local:100/vm-100-disk-0.qcow2,efitype=4m,pre-enrolled-keys=0
qm set 100 --virtio0 local:100/vm-100-disk-1.qcow2,cache=unsafe,discard=on,iothread=1
qm set 100 --ide0 local:100/Ventura-recovery.qcow2,cache=unsafe
qm set 100 --ide2 local:100/OpenCore.qcow2,cache=unsafe

# Remove old raw files
rm *.raw

# Start and create golden snapshot
qm start 100
qm snapshot 100 golden-base --description "Clean Ventura install"

Now you can revert to clean state anytime:

qm rollback 100 golden-base

Remote Access (VNC via Guacamole)

Proxmox’s noVNC console works, but if you want browser-based access through Apache Guacamole, there’s a catch: Guacamole can’t connect to macOS Screen Sharing (Apple’s VNC uses proprietary protocol extensions).

Solution: Vine Server

Install Vine Server (OSXvnc) — an open-source VNC server that uses standard RFB protocol:

# Download and install
curl -L -o /tmp/VineServer.dmg https://github.com/stweil/OSXvnc/releases/download/V5.3.2/VineServer-5.3.2.dmg
hdiutil attach /tmp/VineServer.dmg -nobrowse -quiet
cp -R "/Volumes/VineServer/Vine Server.app" /Applications/
hdiutil detach /Volumes/VineServer

Grant permissions (System Settings → Privacy & Security):

  • Screen Recording → Add Vine Server → Toggle ON
  • Accessibility → Add Vine Server → Toggle ON

Auto-start on login:

cat > ~/Library/LaunchAgents/com.vineserver.plist << 'EOF'
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>Label</key>
    <string>com.vineserver</string>
    <key>ProgramArguments</key>
    <array>
        <string>/Applications/Vine Server.app/Contents/MacOS/OSXvnc-server</string>
        <string>-rfbport</string>
        <string>5901</string>
        <string>-rfbnoauth</string>
    </array>
    <key>RunAtLoad</key>
    <true/>
    <key>KeepAlive</key>
    <true/>
</dict>
</plist>
EOF
launchctl load ~/Library/LaunchAgents/com.vineserver.plist

Guacamole settings:

  • Protocol: VNC
  • Port: 5901
  • Cursor: Local (fixes dot-cursor issue)
  • Color depth: True color (24-bit)

See macOS-VNC-Guacamole-Fix for full troubleshooting guide.

Troubleshooting

OpenCore only shows UEFI Shell and Reset NVRAM

The recovery image isn’t being detected. Common causes:

  • DMG wasn’t converted properly — use file command to check
  • Try pressing Space to reveal hidden boot entries

”bridge vmbr0 does not exist”

Your network bridge has a different name. Check with:

cat /etc/network/interfaces

Update the VM config to use the correct bridge (e.g., vmbr1).

VM not visible in Proxmox web UI

If using Ludus or pools, add the VM to your pool:

pvesh set /pools/YOUR_POOL -vms 100

No internet during install

Configure static IP manually in Recovery Terminal (see Step 5).

Next Steps

Your Proxmox macOS VM is ready. Time to set up log collection.

Next: Part 4 — Collecting macOS Logs →