search

fog / fog-libvirt

libvirt provider for fog
MIT License
16 stars 42 forks source link

Call to virStorageVolGetInfo failed: possible race condition #13

Closed arielsalvo closed 3 years ago

arielsalvo commented 8 years ago

Hi!

I just run into this problem:

-----> Starting Kitchen (v1.4.2)
-----> Cleaning up any prior instances of <default-debian-8>
-----> Destroying <default-debian-8>...
       Finished destroying <default-debian-8> (0m0.00s).
-----> Testing <default-debian-8>
-----> Creating <default-debian-8>...
       Bringing machine 'default' up with 'libvirt' provider...
       /var/lib/jenkins/.vagrant.d/gems/gems/fog-libvirt-0.0.3/lib/fog/libvirt/requests/compute/list_volumes.rb:32:in `info': Call to virStorageVolGetInfo failed: cannot stat file '/var/lib/libvirt/images/vurbia-VAGRANTSLASH-debian-7_vagrant_box_image_20160213024500.img': No such file or directory (Libvirt::RetrieveError)
        from /var/lib/jenkins/.vagrant.d/gems/gems/fog-libvirt-0.0.3/lib/fog/libvirt/requests/compute/list_volumes.rb:32:in `volume_to_attributes'
        from /var/lib/jenkins/.vagrant.d/gems/gems/fog-libvirt-0.0.3/lib/fog/libvirt/requests/compute/list_volumes.rb:10:in `block (2 levels) in list_volumes'
        from /var/lib/jenkins/.vagrant.d/gems/gems/fog-libvirt-0.0.3/lib/fog/libvirt/requests/compute/list_volumes.rb:9:in `each'
        from /var/lib/jenkins/.vagrant.d/gems/gems/fog-libvirt-0.0.3/lib/fog/libvirt/requests/compute/list_volumes.rb:9:in `block in list_volumes'
        from /var/lib/jenkins/.vagrant.d/gems/gems/fog-libvirt-0.0.3/lib/fog/libvirt/requests/compute/list_volumes.rb:44:in `block in raw_volumes'
        from /var/lib/jenkins/.vagrant.d/gems/gems/fog-libvirt-0.0.3/lib/fog/libvirt/requests/compute/list_volumes.rb:42:in `each'
        from /var/lib/jenkins/.vagrant.d/gems/gems/fog-libvirt-0.0.3/lib/fog/libvirt/requests/compute/list_volumes.rb:42:in `raw_volumes'
        from /var/lib/jenkins/.vagrant.d/gems/gems/fog-libvirt-0.0.3/lib/fog/libvirt/requests/compute/list_volumes.rb:8:in `list_volumes'
        from /var/lib/jenkins/.vagrant.d/gems/gems/fog-libvirt-0.0.3/lib/fog/libvirt/models/compute/volumes.rb:11:in `all'
        from /var/lib/jenkins/.vagrant.d/gems/gems/vagrant-libvirt-0.0.32/lib/vagrant-libvirt/action/handle_box_image.rb:63:in `block in call'
        from /var/lib/jenkins/.vagrant.d/gems/gems/vagrant-libvirt-0.0.32/lib/vagrant-libvirt/action/handle_box_image.rb:60:in `synchronize'
        from /var/lib/jenkins/.vagrant.d/gems/gems/vagrant-libvirt-0.0.32/lib/vagrant-libvirt/action/handle_box_image.rb:60:in `call'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/warden.rb:34:in `call'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/builtin/handle_box.rb:56:in `call'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/warden.rb:34:in `call'
        from /var/lib/jenkins/.vagrant.d/gems/gems/vagrant-libvirt-0.0.32/lib/vagrant-libvirt/action/handle_storage_pool.rb:50:in `call'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/warden.rb:34:in `call'
        from /var/lib/jenkins/.vagrant.d/gems/gems/vagrant-libvirt-0.0.32/lib/vagrant-libvirt/action/set_name_of_domain.rb:35:in `call'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/warden.rb:34:in `call'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/warden.rb:95:in `block in finalize_action'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/warden.rb:34:in `call'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/warden.rb:34:in `call'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/builder.rb:116:in `call'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/runner.rb:66:in `block in run'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/util/busy.rb:19:in `busy'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/runner.rb:66:in `run'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/builtin/call.rb:53:in `call'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/warden.rb:34:in `call'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/builtin/config_validate.rb:25:in `call'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/warden.rb:34:in `call'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/builder.rb:116:in `call'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/runner.rb:66:in `block in run'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/util/busy.rb:19:in `busy'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/runner.rb:66:in `run'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/machine.rb:224:in `action_raw'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/machine.rb:199:in `block in action'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/environment.rb:561:in `lock'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/machine.rb:185:in `call'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/machine.rb:185:in `action'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/batch_action.rb:82:in `block (2 levels) in run'
-----> Destroying <default-debian-8>...
       Finished destroying <default-debian-8> (0m0.00s).
>>>>>> ------Exception-------
>>>>>> Class: Kitchen::ActionFailed
>>>>>> Message: Failed to complete #create action: [Expected process to exit with [0], but received '1'
---- Begin output of vagrant up --no-provision --provider libvirt ----
STDOUT: Bringing machine 'default' up with 'libvirt' provider...
STDERR: /var/lib/jenkins/.vagrant.d/gems/gems/fog-libvirt-0.0.3/lib/fog/libvirt/requests/compute/list_volumes.rb:32:in `info': Call to virStorageVolGetInfo failed: cannot stat file '/var/lib/libvirt/images/vurbia-VAGRANTSLASH-debian-7_vagrant_box_image_20160213024500.img': No such file or directory (Libvirt::RetrieveError)
    from /var/lib/jenkins/.vagrant.d/gems/gems/fog-libvirt-0.0.3/lib/fog/libvirt/requests/compute/list_volumes.rb:32:in `volume_to_attributes'
    from /var/lib/jenkins/.vagrant.d/gems/gems/fog-libvirt-0.0.3/lib/fog/libvirt/requests/compute/list_volumes.rb:10:in `block (2 levels) in list_volumes'
    from /var/lib/jenkins/.vagrant.d/gems/gems/fog-libvirt-0.0.3/lib/fog/libvirt/requests/compute/list_volumes.rb:9:in `each'
    from /var/lib/jenkins/.vagrant.d/gems/gems/fog-libvirt-0.0.3/lib/fog/libvirt/requests/compute/list_volumes.rb:9:in `block in list_volumes'
    from /var/lib/jenkins/.vagrant.d/gems/gems/fog-libvirt-0.0.3/lib/fog/libvirt/requests/compute/list_volumes.rb:44:in `block in raw_volumes'
    from /var/lib/jenkins/.vagrant.d/gems/gems/fog-libvirt-0.0.3/lib/fog/libvirt/requests/compute/list_volumes.rb:42:in `each'
    from /var/lib/jenkins/.vagrant.d/gems/gems/fog-libvirt-0.0.3/lib/fog/libvirt/requests/compute/list_volumes.rb:42:in `raw_volumes'
    from /var/lib/jenkins/.vagrant.d/gems/gems/fog-libvirt-0.0.3/lib/fog/libvirt/requests/compute/list_volumes.rb:8:in `list_volumes'
    from /var/lib/jenkins/.vagrant.d/gems/gems/fog-libvirt-0.0.3/lib/fog/libvirt/models/compute/volumes.rb:11:in `all'
    from /var/lib/jenkins/.vagrant.d/gems/gems/vagrant-libvirt-0.0.32/lib/vagrant-libvirt/action/handle_box_image.rb:63:in `block in call'
    from /var/lib/jenkins/.vagrant.d/gems/gems/vagrant-libvirt-0.0.32/lib/vagrant-libvirt/action/handle_box_image.rb:60:in `synchronize'
    from /var/lib/jenkins/.vagrant.d/gems/gems/vagrant-libvirt-0.0.32/lib/vagrant-libvirt/action/handle_box_image.rb:60:in `call'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/warden.rb:34:in `call'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/builtin/handle_box.rb:56:in `call'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/warden.rb:34:in `call'
    from /var/lib/jenkins/.vagrant.d/gems/gems/vagrant-libvirt-0.0.32/lib/vagrant-libvirt/action/handle_storage_pool.rb:50:in `call'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/warden.rb:34:in `call'
    from /var/lib/jenkins/.vagrant.d/gems/gems/vagrant-libvirt-0.0.32/lib/vagrant-libvirt/action/set_name_of_domain.rb:35:in `call'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/warden.rb:34:in `call'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/warden.rb:95:in `block in finalize_action'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/warden.rb:34:in `call'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/warden.rb:34:in `call'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/builder.rb:116:in `call'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/runner.rb:66:in `block in run'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/util/busy.rb:19:in `busy'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/runner.rb:66:in `run'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/builtin/call.rb:53:in `call'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/warden.rb:34:in `call'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/builtin/config_validate.rb:25:in `call'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/warden.rb:34:in `call'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/builder.rb:116:in `call'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/runner.rb:66:in `block in run'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/util/busy.rb:19:in `busy'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/runner.rb:66:in `run'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/machine.rb:224:in `action_raw'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/machine.rb:199:in `block in action'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/environment.rb:561:in `lock'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/machine.rb:185:in `call'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/machine.rb:185:in `action'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/batch_action.rb:82:in `block (2 levels) in run'
---- End output of vagrant up --no-provision --provider libvirt ----
Ran vagrant up --no-provision --provider libvirt returned 1]

If you notice that the error is regarding a volume for a completely different instance you may arrive at the same conclusion I did. I believe this is a race condition caused by bringing up two different instances at the same time that causes lib/fog/libvirt/requests/compute/list_volumes.rb:32 to try to stat the non-existant image file that the other instance is bringing up.

I don't know enough ruby to fix this myself, but I'll help in any way I can.

Thanks! --Ariel

arielsalvo commented 8 years ago

I just opened PR #14 for this issue.

I got a similar race condition in the list_domains.rb file

   Bringing machine 'default' up with 'libvirt' provider...
   /var/lib/jenkins/.vagrant.d/gems/gems/fog-libvirt-0.0.4.pre.1/lib/fog/libvirt/requests/compute/list_domains.rb:66:in `xml_desc': Call to virDomainGetXMLDesc failed: Domain not found: no domain with matching uuid '2f2b986e-7504-4ebd-bc8c-f0392c6fba5b' (kitchen-debian-8-default-debian-8_default) (Libvirt::Error)
    from /var/lib/jenkins/.vagrant.d/gems/gems/fog-libvirt-0.0.4.pre.1/lib/fog/libvirt/requests/compute/list_domains.rb:66:in `domain_to_attributes'
    from /var/lib/jenkins/.vagrant.d/gems/gems/fog-libvirt-0.0.4.pre.1/lib/fog/libvirt/requests/compute/list_domains.rb:16:in `block in list_domains'
    from /var/lib/jenkins/.vagrant.d/gems/gems/fog-libvirt-0.0.4.pre.1/lib/fog/libvirt/requests/compute/list_domains.rb:16:in `map'
    from /var/lib/jenkins/.vagrant.d/gems/gems/fog-libvirt-0.0.4.pre.1/lib/fog/libvirt/requests/compute/list_domains.rb:16:in `list_domains'
    from /var/lib/jenkins/.vagrant.d/gems/gems/fog-libvirt-0.0.4.pre.1/lib/fog/libvirt/models/compute/servers.rb:11:in `all'
    from /var/lib/jenkins/.vagrant.d/gems/gems/vagrant-libvirt-0.0.32/lib/vagrant-libvirt/action/set_name_of_domain.rb:22:in `call'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/warden.rb:34:in `call'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/warden.rb:95:in `block in finalize_action'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/warden.rb:34:in `call'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/warden.rb:34:in `call'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/builder.rb:116:in `call'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/runner.rb:66:in `block in run'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/util/busy.rb:19:in `busy'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/runner.rb:66:in `run'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/builtin/call.rb:53:in `call'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/warden.rb:34:in `call'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/builtin/config_validate.rb:25:in `call'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/warden.rb:34:in `call'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/builder.rb:116:in `call'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/runner.rb:66:in `block in run'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/util/busy.rb:19:in `busy'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/runner.rb:66:in `run'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/machine.rb:224:in `action_raw'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/machine.rb:199:in `block in action'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/environment.rb:561:in `lock'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/machine.rb:185:in `call'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/machine.rb:185:in `action'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/batch_action.rb:82:in `block (2 levels) in run'
arielsalvo commented 8 years ago

I see that PR #15 would fix the list_domains issues. Testing now

github-actions[bot] commented 3 years ago

This issue has been marked inactive and will be closed if no further activity occurs.