qemu-kvm: could not open disk image ' ': Permission denied

[caleyg]

Following the KVM documentation I stumbled across this permissions error.

Running as root on Centos 6.5

[root@dev1 veewee]# bundle exec veewee kvm build 'bamboo-agent' --pool-name=default --network-name=default --force
Building Box bamboo-agent with Definition bamboo-agent:
- debug : false
- cwd : /root/veewee
- force : true
- nogui : false
- auto : false
- checksum : false
- postinstall_include : []
- postinstall_exclude : []
- skip_to_postinstall : false
- use_emulation : false
- network_name : default
- pool_name : default

The isofile CentOS-6.5-x86_64-netinstall.iso already exists.
/usr/local/rvm/gems/ruby-1.9.3-p547@veewee/gems/fog-1.24.0/lib/fog/libvirt/requests/compute/vm_action.rb:7:in `create': Call to virDomainCreateWithFlags failed: Unable to read from monitor: Connection reset by peer (Libvirt::Error)
internal error process exited while connecting to monitor: char device redirected to /dev/pts/19
qemu-kvm: -drive file=/root/veewee/iso/CentOS-6.5-x86_64-netinstall.iso,if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw: could not open disk image /root/veewee/iso/CentOS-6.5-x86_64-netinstall.iso: Permission denied
        from /usr/local/rvm/gems/ruby-1.9.3-p547@veewee/gems/fog-1.24.0/lib/fog/libvirt/requests/compute/vm_action.rb:7:in `vm_action'
        from /usr/local/rvm/gems/ruby-1.9.3-p547@veewee/gems/fog-1.24.0/lib/fog/libvirt/models/compute/server.rb:69:in `start'
        from /root/veewee/lib/veewee/provider/kvm/box/up.rb:8:in `up'
        from /root/veewee/lib/veewee/provider/core/box/build.rb:50:in `kickstart'
        from /root/veewee/lib/veewee/provider/core/box/build.rb:111:in `build'
        from /root/veewee/lib/veewee/provider/kvm/box/build.rb:6:in `build'
        from /root/veewee/lib/veewee/command/kvm.rb:25:in `build'
        from /usr/local/rvm/gems/ruby-1.9.3-p547@veewee/gems/thor-0.19.1/lib/thor/command.rb:27:in `run'
        from /usr/local/rvm/gems/ruby-1.9.3-p547@veewee/gems/thor-0.19.1/lib/thor/invocation.rb:126:in `invoke_command'
        from /usr/local/rvm/gems/ruby-1.9.3-p547@veewee/gems/thor-0.19.1/lib/thor.rb:359:in `dispatch'
        from /usr/local/rvm/gems/ruby-1.9.3-p547@veewee/gems/thor-0.19.1/lib/thor/invocation.rb:115:in `invoke'
        from /usr/local/rvm/gems/ruby-1.9.3-p547@veewee/gems/thor-0.19.1/lib/thor.rb:235:in `block in subcommand'
        from /usr/local/rvm/gems/ruby-1.9.3-p547@veewee/gems/thor-0.19.1/lib/thor/command.rb:27:in `run'
        from /usr/local/rvm/gems/ruby-1.9.3-p547@veewee/gems/thor-0.19.1/lib/thor/invocation.rb:126:in `invoke_command'
        from /usr/local/rvm/gems/ruby-1.9.3-p547@veewee/gems/thor-0.19.1/lib/thor.rb:359:in `dispatch'
        from /usr/local/rvm/gems/ruby-1.9.3-p547@veewee/gems/thor-0.19.1/lib/thor/base.rb:440:in `start'
        from /root/veewee/bin/veewee:24:in `<top (required)>'
        from /usr/local/rvm/gems/ruby-1.9.3-p547@veewee/bin/veewee:23:in `load'
        from /usr/local/rvm/gems/ruby-1.9.3-p547@veewee/bin/veewee:23:in `<main>'
        from /usr/local/rvm/gems/ruby-1.9.3-p547@veewee/bin/ruby_executable_hooks:15:in `eval'
        from /usr/local/rvm/gems/ruby-1.9.3-p547@veewee/bin/ruby_executable_hooks:15:in `<main>'

[caleyg]

SOLVED: found a way. Changing /etc/libvirt/qemu.conf to make things work. Uncomment user/group to work as root. Then restart libvirtd

# The user for QEMU processes run by the system instance. It can be
# specified as a user name or as a user id. The qemu driver will try to
# parse this value first as a name and then, if the name doesn't exist,
# as a user id.
#
# Since a sequence of digits is a valid user name, a leading plus sign
# can be used to ensure that a user id will not be interpreted as a user
# name.
#
# Some examples of valid values are:
#
#       user = "qemu"   # A user named "qemu"
#       user = "+0"     # Super user (uid=0)
#       user = "100"    # A user named "100" or a user with uid=100
#
user = "root"

# The group for QEMU processes run by the system instance. It can be
# specified in a similar way to user.
group = "root"

# Whether libvirt should dynamically change file ownership
# to match the configured user/group above. Defaults to 1.
# Set to 0 to disable file ownership changes.
#dynamic_ownership = 1

[root@dev1 ~]# service libvirtd restart
Stopping libvirtd daemon:                                  [  OK  ]
Starting libvirtd daemon:                                  [  OK  ]

[Germano0]

I don't think it is wise to make such edits letting qemu running as root. You should put the ISO file in a directory that is readable by qemu user, for example /tmp

[FossPrime]

The qemu user needs search permissions all the way up the path, that means chmod a+x. For me the issue was I had ACL's set which blocked qemu... to remove them run "setfacl -b offending-dir/"

You can tell if that's your problem when unix permissions are correct but you can't ls every upstream dir with sudo su -s /bin/bash qemu and ls with an account that does have access yields a result with a permission line that contains a dot such as drwxrwxr-x. meaning extended ACL permissions.

[murlock]

Encountered same issue, after digging ACL issue, I have to grant supplementary right on my HOME

murlock@asus:~/kubernet $ sudo getfacl -e /home/murlock
getfacl: Removing leading '/' from absolute path names
# file: home/murlock
# owner: murlock
# group: murlock
user::rwx
user:libvirt-qemu:--x       #effective:---
group::---          #effective:---
mask::---
other::---

murlock@asus:~/kubernet $ sudo setfacl -m u:libvirt-qemu:rx /home/murlock

murlock@asus:~/kubernet $ sudo getfacl -e /home/murlock
getfacl: Removing leading '/' from absolute path names
# file: home/murlock
# owner: murlock
# group: murlock
user::rwx
user:libvirt-qemu:r-x       #effective:r-x
group::---          #effective:---
mask::r-x
other::---

[davidbeesley]

Murlock's suggestions fixed the same issue for me as well. Only difference was the user was named "qemu" instead of "libvirt-qemo"

[rimsopheap]

@murlock Pefect! That works like a magic.

[lacksfish]

If you copy your ISO image to /var/lib/libvirt/images (and access it from there) all the problems will go away.

[nabeelpredmac]

@murlock thanks ... Magic indeed ...

[mvitale1989]

If none above worked for you (not even having qemu run as root:root; crazy uh?), here's another workaround that did the trick for me: instead of using the libvirt volume provider for your disks, in your machine XMLs, use plain files instead.

E.g., instead of this: <disk type='volume' device='disk'> ... </disk> Use this: <disk type='file' device='disk'> .... </disk>

Edit: turns out it was AppArmour, denying accessing the volume through the libvirt pool interface as stated here https://github.com/jedi4ever/veewee/issues/996#issuecomment-497976612

[jvleminc]

@lacksfish's solution seems to be the easiest one. Worked for me.

[loxK]

I have my images in a drive mounted in /media/user/Data sudo setfacl -m u:libvirt-qemu:rx /media/user fixed it.

[AcidGo]

@murlock NICE. Thanks.

[DMW007]

Using Terraform with the libvirt Provider, I got the same error. But the problem was Apparmour: https://github.com/dmacvicar/terraform-provider-libvirt/issues/97

For testing purpose, I simply set security_driver = "none" in /etc/libvirt/qemu.conf and restart the service service libvirtd restart.

[wwwlookformetop]

If you are using libvirt backup disks on openstack, you need to modify the permissions of users such as nova, glance, and cinder to the backup directory Because openstack USES these users to create backup directories

[wwwlookformetop]

then,reboot

[dperygin]

Using Terraform with the libvirt Provider, I got the same error. But the problem was Apparmour: dmacvicar/terraform-provider-libvirt#97

For testing purpose, I simply set security_driver = "none" in /etc/libvirt/qemu.conf and restart the service service libvirtd restart.

This is what worked for me. Thanks.

[mrcloudhacker]

Simply add your username to libvirt group and then instead of root replace with your username and group in /etc/libvirt/qemu.conf file.

user = "howdy"
group = "rowdy"

No need to mess with acl or run it as root. After making necessary changes simply execute systemctl restart libvirtd and ta-da

[sevillaarvin]

I don't think it is wise to make such edits letting qemu running as root. You should put the ISO file in a directory that is readable by qemu user, for example /tmp

I followed this and did something like:

cd $(mktemp -d)
cp ~/my-installer.iso .
chmod 777 .
sudo virt-install ... --cdrom my-installer.iso

This is to bypass all the ACLs. Please tell me if there's something wrong with this.

[dzintars]

Tnx @codingenesis . Exactly what i needed! :)

sudo groupadd --system libvirt
sudo usermod -a -G libvirt $(whoami)
newgrp libvirt
id $(whoami)
sudo vim /etc/libvirt/libvirtd.conf
unix_sock_group = "libvirt"
unix_sock_rw_perms = "0770"
sudo vim /etc/libvirt/qemu.conf
user = "howdy"
group = "rowdy"
sudo systemctl restart libvirtd.service

[vap0rtranz]

After trying almost all of the above, a very simple "chmod all" fix appeared when someone over at StackExchange explain what kind of perms Qemu needs.

sudo chmod a+x [path_to_volume_pool]

Explanation given:

Qemu requires execution permission along the image file's path

[boostmachines]

After trying almost all of the above, a very simple "chmod all" fix appeared when someone over at StackExchange explain what kind of perms Qemu needs.

sudo chmod a+x [path_to_volume_pool]

Explanation given:

Qemu requires execution permission along the image file's path

Exactly. If admins are storing their ISOs in nontraditional directories, this should be implemented.

[peterwillcn]

Using Terraform with the libvirt Provider, I got the same error. But the problem was Apparmour: dmacvicar/terraform-provider-libvirt#97

For testing purpose, I simply set security_driver = "none" in /etc/libvirt/qemu.conf and restart the service service libvirtd restart.

Thx.

[AnonA2]

it just may be permissions + chmod+x and permissions of folder and home. It helped me to boot os x vm. Now trying to realise GPU passthrough...

[ankitg12]

After trying almost all of the above, a very simple "chmod all" fix appeared when someone over at StackExchange explain what kind of perms Qemu needs.

sudo chmod a+x [path_to_volume_pool]

Explanation given:

Qemu requires execution permission along the image file's path

This was such an effing thing !!!!

[TeaCult]

@vap0rtranz In my case file was /media/username/4TBBackup/testMech.raw setfaclor security_driver = "none" did not resolve it but

sudo chmod a+x /media
sudo chmod a+x /media/username
sudo chmod a+x /media/username/4TBBackup
sudo chmod a+x /media/username/4TBBackup/testMech.raw 

Indeed solved it libvirtd needs execution permissions along the path

But ... Why the hell it needs it ?

[tiziano-pedrazzoli]

none of the solutions you proposed worked. command:

virsh qemu-monitor-command --pretty ubuntu20.04 '
{"execute": "transaction",
   "arguments": {
     "actions": [
       {"type": "block-dirty-bitmap-add",
        "data": {"node": "libvirt-1-format", "name": "bitmap0"}},
       {"type": "drive-backup",
        "data": {"device": "libvirt-1-format", "target": "/home/user/vm/backup.qcow2",
                 "sync": "full", "format": "qcow2"}}
     ]
   }
} '

OUTPUT

{
   "id": "libvirt-14",
   "error": {
     "class": "GenericError",
     "desc": "Could not create file: No such file or directory"
   }
}

I modified /etc/libvirt/qemu.conf by uncommenting user / group. i chmod on the folder putting the permissions to 777 and a + x. i disabled apparmour in /etc/libvirt/qemu.conf with security_driver = "none". i also tried sudo setfacl -m u: libvirt-qemu: rx / home / user / vm. I don't know what to do anymore

[dperygin]

I'm sorry my solutions didn't work for you. Unfortunately I am no longer working with this project and therefore cannot adapt the code further.

Best of luck to you!

Donna Perygin

---

From: tiziano-pedrazzoli @.> Sent: Friday, July 30, 2021 7:29 AM To: jedi4ever/veewee @.> Cc: Donna Perygin @.>; Comment @.> Subject: Re: [jedi4ever/veewee] qemu-kvm: could not open disk image ' ': Permission denied (#996)

none of the solutions you proposed worked. command:

virsh qemu-monitor-command --pretty ubuntu20.04 ' {"execute": "transaction", "arguments": { "actions": [ {"type": "block-dirty-bitmap-add", "data": {"node": "libvirt-1-format", "name": "bitmap0"}}, {"type": "drive-backup", "data": {"device": "libvirt-1-format", "target": "/home/user/vm/backup.qcow2", "sync": "full", "format": "qcow2"}} ] } } '

OUTPUT

{ "id": "libvirt-14", "error": { "class": "GenericError", "desc": "Could not create file: No such file or directory" } }

I modified /etc/libvirt/qemu.conf by uncommenting user / group. i chmod on the folder putting the permissions to 777 and a + x. i disabled apparmour in /etc/libvirt/qemu.conf with security_driver = "none". i also tried sudo setfacl -m u: libvirt-qemu: rx / home / user / vm. I don't know what to do anymore

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fjedi4ever%2Fveewee%2Fissues%2F996%23issuecomment-889860398&data=04%7C01%7Cdperygin%40jsu.edu%7C531a2321a2c24a6df76208d95355ad7b%7C32f44c14cd0e4539b7633c860a76727a%7C0%7C0%7C637632449719424885%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=MuWh2JSsvRxxA2fGz3ACsKw%2FUbwGgL4SF7YZvnjrl2U%3D&reserved=0, or unsubscribehttps://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FADZJ5PN3VLXFM6XFJMQQCGDT2KLKTANCNFSM4AV3ITTQ&data=04%7C01%7Cdperygin%40jsu.edu%7C531a2321a2c24a6df76208d95355ad7b%7C32f44c14cd0e4539b7633c860a76727a%7C0%7C0%7C637632449719434878%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=uLO2v2QwNioCuutHwMOo4eUnIdM4phPYCFG%2FIE9RCaQ%3D&reserved=0.

[DomPro11]

@tiziano-pedrazzoli I did everything as well, but also I decided to change the group and ownership of the file with chgrp username /path/to/file/backup.qcow2 chown username /path/to/file/backup.qcow2 in addition to modifying /etc/libvirt/qemu.conf to use my username and group. After trying this last resort, it ended up working for me. I hope this is helpful to anyone that has at least gotten this far when nothing else worked.

[dperygin]

Sorry. I am no longer maintaining this. I used it a couple of years ago and don't need it anymore.

[juanbomfim22]

In the case I run the VM through the terminal (in my case from OSX-KVM). I do sudo ./Open-Boot.sh and as the docs says: run

sudo setfacl -R -m u:libvirt-qemu:rx /home/$USER
sudo setfacl -R -m u:libvirt-qemu:rx /home/$USER/OSX-KVM

[network-shark]

Keep it simple

sudo usermod -a -G disk $(whoami)

working on debian11

[DauntlessRony]

Error starting domain: internal error: process exited while connecting to monitor: 2022-10-26T09:15:09.633143Z qemu-kvm: -blockdev {"driver":"file","filename":"/home/rony/data/soft/vm/kvm/win10.qcow2","node-name":"libvirt-2-storage","auto-read-only":true,"discard":"unmap"}: Could not open '/home/rony/data/soft/vm/kvm/win10.qcow2': Permission denied

1.

Comment out or assign root user in /etc/libvirt/qemu.conf file.

sudo vim /etc/libvirt/qemu.conf

###################################################################
#user = "root"
user = "root" #this is the user

# The group for QEMU processes run by the system instance. It can be
# specified in a similar way to user.
#group = "root"
group = "root" #this is the group
###################################################################

2.
Now, Restart the libvirtd service.

[user@dell ~]$ sudo systemctl restart libvirtd
[user@dell ~]$ sudo systemctl status libvirtd

[c2b2pss]

@murlock Thanks! Solution worked. Effective, simple.

[Toasterson]

Using Terraform with the libvirt Provider, I got the same error. But the problem was Apparmour: dmacvicar/terraform-provider-libvirt#97

For testing purpose, I simply set security_driver = "none" in /etc/libvirt/qemu.conf and restart the service service libvirtd restart.

If anybody is still looking to fix this and it's neither acl, perms or apparmor, Ubuntu 22.04 now has selinux. So disabling security_driver here is mandatory

[rdonnelly2]

https://github.com/jedi4ever/veewee/issues/996#issuecomment-536519623

mrcloudhacker had the best an easiest solution.

[ThaysonScript]

how do I resolve this?

virsh attach-disk debian125 /home/debian/kvm/disks_kvm/disk1.qcow2 --target vdb --persistent error: Failed to attach disk error: internal error: unable to execute QEMU command 'blockdev-add': Could not open '/home/debian/kvm/disks_kvm/disk1.qcow2': Permission denied

[Toasterson]

Hi, Check my comments above for the rough instructions. 3 things can go wrong, User and group filesystem permissions (libvirt and KVM runs as it's own user) Apparmor or SELunix permissions. Some ACL. Since you are placing the disk inside your home, it's probably security. Try placing it in the same directory as the first disk or /var/lib/libvirt/images alternatively. If that gives a permission error check you pool config in libvirt and place the disk in the correct location.

[ThaysonScript]

Hi, Check my comments above for the rough instructions. 3 things can go wrong, User and group filesystem permissions (libvirt and KVM runs as it's own user) Apparmor or SELunix permissions. Some ACL. Since you are placing the disk inside your home, it's probably security. Try placing it in the same directory as the first disk or /var/lib/libvirt/images alternatively. If that gives a permission error check you pool config in libvirt and place the disk in the correct location.

Thanks friend, I managed to solve it by changing the user and group to root in the /etc/libvirt/qemu.conf file and also changing the permissions of the folder where the disks are with chmod a+x

[ThaysonScript]

Hi, Check my comments above for the rough instructions. 3 things can go wrong, User and group filesystem permissions (libvirt and KVM runs as it's own user) Apparmor or SELunix permissions. Some ACL. Since you are placing the disk inside your home, it's probably security. Try placing it in the same directory as the first disk or /var/lib/libvirt/images alternatively. If that gives a permission error check you pool config in libvirt and place the disk in the correct location.

Thanks friend, I managed to solve it by changing the user and group to root in the /etc/libvirt/qemu.conf file and also changing the permissions of the folder where the disks are with chmod a+x

I also believe that I managed to solve why I changed the security_driver to none, at first it gave an error to manage the vm with libvirt, but I commented it again and finally everything worked fine

[ShayBox]

I'm using Archlinux and it appears libvirt run as root which is why /var/lib/libvirt/images defaults to root:root 0644 but qemu runs as libvirt-qemu and requires o+x.

You can either sudo chmod o+x -R /var/lib/libvirt/images to allow other (non-root) users the additional execute permission, or you can sudo chown libvirt-qemu:root -R /var/lib/libvirt/images and narrow permissions to sudo chmod 0700 -R /var/lib/libvirt/images so only libvirt-qemu and root can access the files.

You could also just use ACL to allow libvirt-qemu access without changing user/group/permissions, but I prefer this.