How to Enable GPU Passthrough on Proxmox 6.2

Hello everyone, I am currently building my small homelabs system using my old AMD FX-8350 processor. I am running the latest Proxmox 6.2.4 and it works very well. But lately, I want to do something different. Proxmox supports the GPU passthrough which is interesting. We can assign and use the power of our GPU to one of the virtual machine. There are some steps to do the GPU passthrough. It is not super simple but doable. We will get through there in a moment.

This tutorial is based on the AMD processor. There are some pretty basic differences between AMD and Intel processor. You can visit the official guide on Proxmox website. If you are new to Proxmox and GPU passthrough things, it is better to read it first.

Steps to Enable GPU Passthrough on Proxmox 6.2

Step 1. BIOS Settings

It is important to set the BIOS prior to the GPU passthrough. You will need to enable the VT-d, Virtualization as well as UEFI.

Step 2. Enable the IOMMU

This step will enable the IOMMU by editing the Grub. I would recommend you to connect to the server via ssh from other computer.

On the host machine, we are going to edit the grub.

nano /etc/default/grub

Now find the following line

GRUB_CMDLINE_LINUX_DEFAULT="quiet"

And then change the line into this one

GRUB_CMDLINE_LINUX_DEFAULT="quiet amd_iommu=on"

Close and save the editor and reboot Proxmox server. After reboot, connect via SSH again and run this command to confirm.

dmesg | grep -e DMAR -e IOMMU

You should see something like this:

If it returns nothing, then something is wrong. You need to check your hardware configuration and/or maybe there are some unsupported hardware.

Step 3. Load Additional Kernel Modules

Here we are going to add some modules to our Proxmox server. Add the following lines to /etc/modules

vfio
vfio_pci
vfio_virqfd

Close and save the file. And then reboot the server again.

Step 4. Verify the IOMMU Interrupt Remapping

It is important to make sure that our system supports the Interrupt Remapping. To check, use this command

dmesg | grep 'remapping'

You should see something like this

AMD-Vi means the interrupt remapping is enabled so we are good to go from here. And you can also use this command for more results.

dmesg | grep -e DMAR -e IOMMU 

Step 5. Get GPU id

Now we need to get our GPU id and configure the Proxmox. Execute this command to get the GPU id and the associated vendor id.

lspci -v

Output:

As you can see, my Graphic card id is 01:00. Now lets run another command to get the vendor id

lspci -n -s 01:00

Output:

01:00.0 0300: 1002:67df (rev ef)
01:00.1 0403: 1002:aaf0

Due take a note of the vendor id shown. In this case, 1002:67df and 1002:aaf0

With this vendor id, we need to assign these two values to vfio. Use this command and don’t forget to change the vendor ids with yours.

echo "options vfio-pci ids=1002:67df,1002:aaf0" > /etc/modprobe.d/vfio.conf

Reboot Proxmox VE. So basically, we were done configuring the Proxmox host to provide the GPU passthrough feature. Next, we are going to create a new VM that will be used for Windows 10.

Step 6. Create a New VM

Here we are going to create a new VM. But do not start it because we will add some configuration to this new VM manually.

Create a new VM from the Proxmox web interface. But make sure you choose or use the following settings:

  • BIOS: OMVF (UEFI)
  • Machine: q35

Follow the wizard until it completes and then install Windows 10 completely. Make sure you set the Remote Desktop on the newly installed Windows 10 because after we configure GPU passthrough, you won’t be able to use the console from the Proxmox web dashboard. Once you set up the remote desktop. Shutdown the VM.

Step 7. Additional Configuration for Windows 10 VM

Now lets SSH to the Proxmox server once again and edit the new Windows 10 vm conf. In this example, my new Windows 10 conf file is in /etc/pve/qemu-server/102.conf. Add the following lines to the config file.

cpu: host,hidden=1,flags=+pcid
hostpci0: 01:00,pcie=1,x-vga=on

Here is my full config screenshot

If we switch to the Proxmox web console in the hardware section, it looks like this.

As you can see, there is a PCI Device (hostpci0) at the end of my hardware list.

Now start the Windows 10 VM and remote desktop to it. Check if the Graphic card is detected and active in your Windows 10 VM.

Alternatively, you can also use a monitor connected to your VGA card and use the Windows 10 VM on that monitor. If you do this, you may also want to passthrough the mouse and keyboard as well.

USB keyboard and mouse passthrough

15 Comments

  1. i make installation of Catalina under proxmox like in your dosc
    then i try to passthrough pcie and
    first time boot my display work but after balck screen
    system show rx 480 card but
    monitor is blank

    • Hi,
      I hope you already setup your macos for remote desktop via VNC. So you can change some settings from other pc.
      I have similar problem. MacOS will detect your RX card and the proxmox display as two separate monitor.
      Connect via VNC and set/change the multiple display settings from there.
      Make your external monitor as the main display.

    • Hi, connect to your Mac via VNC and from there you can change the display settings. You may need to enable the mirroring feature from System Preference >> Display

  2. I cant enter with VNC (connection reset by peer) ((((
    Should i keep enabled ROM-BAR option?
    can i enter by ssh and setup via cli?
    if yes please suggest appropriate commands

  3. Excelent guide! Thank you. I have run into a few problems that I cannot seem to explain.

    First some info:
    -I have run qemu-kvm with gpu passthrough before on Ubuntu with this machine. It worked but IO performance was bad so I am now trying proxmox.
    -I have 2x gpu’s. one nvidia setup in pcie3 slot, bios set to boot from gpu in pcie3 for host to use. Another GPU is an AMD R9 290. setup in pcie1 for passthrough to guest.
    -I have two sets of mice and keyboard
    -I installed LXDE on the host to have a gui. I only have this one machine and cannot connect to the webinterface from another computer.

    I passed through 1 of my 2 Keyboards. One of them is ps/2 for host operations. The other is passed through to the win10 guest. I passed one of my two USB mice to the guest.

    When I start the win10 guest, the mouse and keyboard freezes on the host. The monitor connected to nvidia gpu stops updating frames and is just kinda frozen on whatever whas on the screen at that time. The guest comes up no problems with gpu oassed through. What seems to be causing this peculiar problem of the host monitor/mouse/keyboard freezing up? Shutting down the VM does not change everything back to how it worked before I started the win10 guest.

  4. Pretty good guide, very comprehensive!
    Unfortunately, I’m having issues and the VM doesn’t start.

    On a side note first: I assume this line
    hostpci0: 01:00,pcie=1,x-vga=on
    in the .conf file needs to be adjusted with the actual bus number (i.e. change your 01:00 against whatever else the VGA card reports, 2f:00 in my case), right?

    So ffter I’ve done everything in the guide, my Windows doesn’t even boot anymore: after the “Start” command, the log at the bottom just shows a wheel, and the VNC client cannot connect to it.
    I then tried to start the vm via the command line “qm start 102” to possibly get some error log, however, even the command line is stuck and I can’t even interrupt it with ctrl-C.

    • Ok…so you already have Windoslws vm before you enable the gpu passthrough?
      Its worth to make sure that your Windows vm has the ovmf bios as well as q35 machine. These are important. Check your vm settings.

  5. Yes, that is correct.

    This line
    hostpci0: 2f:00,pcie=1,x-vga=on
    is the culprit, when I add it, then qm gets stuck right away so I have to kill it.

    The rest of my config looks like this, I have double checked it several times:

    agent: 1
    bios: ovmf
    bootdisk: ide0
    cores: 1
    efidisk0: local-lvm:vm-102-disk-0,size=4M
    ide0: /dev/sdb,size=244198584K
    ide2: local:iso/virtio-win-0.1.185.iso,media=cdrom,size=402812K
    machine: q35
    memory: 8192
    name: Win10.local
    net0: virtio=1A:A0:7F:5C:18:E1,bridge=vmbr0,firewall=1
    numa: 0
    ostype: win10
    scsihw: virtio-scsi-pci
    smbios1: uuid=c1c886d7-2506-4260-bcf2-90afa8f4946e
    sockets: 1
    vmgenid: b8258097-3b2a-425d-9f64-ec3eea609477

  6. So as I’m a total newbie to this, I think these instruction are already streamlined to get the desired results, but I want to point out that in my attempts to fix my issue, I noticed that a few things seem to be inconsistent with the official documentation:

    -the Grub changes should be made permanent with the update-grub command
    -I’m, not sure if Grub should get updated at all. As far as I understand it correctly, Grub is only relevant for the legacy SeaBIOS boot. As we want to boot via EFI/q35, I understand the SystemD-boot should be adjusted instead
    -in the original description, there is an additional kernel module vfio_iommu_type1
    -the module change ought to be made permanent in the initramfs via
    update-initramfs -u -k all

    However, with all these steps performed, the VM still refuses to start, so I need to dig more.

  7. So as I’m a total newbie to this, I think these instruction are already streamlined to get the desired results, but I want to point out that in my attempts to fix my issue, I noticed that a few things seem to be inconsistent with the official documentation:

    -the Grub changes should be made permanent with the update-grub command
    -I’m, not sure if Grub should get updated at all. As far as I understand it correctly, Grub is only relevant for the legacy SeaBIOS boot. As we want to boot via EFI/q35, I understand the SystemD-boot should be adjusted instead
    -in the original description, there is an additional kernel module vfio_iommu_type1
    -the module change ought to be made permanent in the initramfs via
    update-initramfs -u -k all

    However, with all these steps performed, the VM still refuses to start, so I need to dig more.

    PS: yes, I tried both possibilities to add the PCIe device, adding the line in the conf file and using the GUI, the results (command line and not working VM) are still the same

5 Trackbacks / Pingbacks

  1. Step by Step Creating Windows 10 Virtual Machine on Proxmox with GPU Passthrough | Manjaro dot site
  2. Tips to Create Ubuntu 20.04 VM on Proxmox with GPU Passthrough
  3. How to Install macOS Big Sur on Proxmox VE | Manjaro dot site
  4. VMware Workstation 16.0 Brings Better Performance to Your Virtual Machines | Manjaro dot site
  5. How to Enable PCI-E Passthrough on Proxmox VE on Intel System | Manjaro dot site

Leave a Reply

Your email address will not be published.


*