Stuck task if no space on device

[sshnaidm]

There is an issue when running task with virt-resize on downloaded image and no much space on root partition (as it's by default in CentOS/Fedora installation). When no space is left on partition the task hangs and never finishes. I see 2 points here: 1) Make possible to define vm.volume_path in playbooks. Maybe it's possible to pass it with --extra-var and "hash_behavior=merge" in ansible.cfg, which allows to merge vars and not overwrite them. 2) To make timeout for the task, that fail it after a specific time to prevent hanging forever. Maybe it's worth to make timeout for every task by default.

[trown]

oh... I did not know about hash_behavior=merge, have you tried that?

[trown]

I made https://review.gerrithub.io/261381 to address this specific task in point 2). I like the idea for point 1) if that works.

[sshnaidm]

tried hash_behavior=merge, seems like working: in ansible.cfg:

[defaults]
hash_behavior=merge

and then pass volume_path as json (maybe there is a better way to do it): ansible-playbook -vv $HOME/.quickstart/usr/local/share/tripleo-quickstart/playbook/quickstart-$RELEASE.yml --extra-vars url=$UNDERCLOUD_QCOW2_LOCATION --extra-vars 'vms["volume_path"]:"/home/img/"' and:

[root@dhcp-0-120 home]# ls /home/img
ceph_0.qcow2  compute_0.qcow2  control_0.qcow2  control_1.qcow2  control_2.qcow

[sshnaidm]

But it failed with another error:

TASK [setup/undercloud : Install undercloud vm storage] ************************
failed: [host0] => (item={u'disk_size': u'50', u'cpus': u'3', u'name': u'undercloud', u'memory': u'12288'}) => {"changed": true, "cmd": "export LIBGUESTFS_BACKEND=direct; virsh vol-create-as default 
undercloud.qcow2 50G --format qcow2; virt-resize --expand /dev/sda1 /home/stack/.quickstart/undercloud.qcow2 '/home/img/''undercloud'.qcow2;",
"delta": "0:00:00.176040", "end": "2016-01-28 16:32:38.063109", "failed": true, "item": {"cpus": "3", "disk_size": "50", "memory": "12288", "name": "undercloud"}, "rc": 1, "start": "2016-01-28 16:32:37.887069", 
"stderr": "virt-resize: error: libguestfs error: /home/img/undercloud.qcow2: No such \nfile or directory\n\nIf reporting bugs, run virt-resize with debugging enabled and include the \ncomplete output:\n\n  virt-resize -v -x [...]", 
"stdout": "Vol undercloud.qcow2 created\n\nExamining /home/stack/.quickstart/undercloud.qcow2 ...", "stdout_lines": ["Vol undercloud.qcow2 created", "", "Examining /home/stack/.quickstart/undercloud.qcow2 ..."], "warnings": []}

Seems like it's also used in playbooks/roles/libvirt/defaults/main.yml and virt-resize expects it to be there: volume_path: /var/lib/libvirt/images/ and it's not replaced there. Not sure what happens

[trown]

I think you will need to also override the vms.volume_pool variable, since by default, it uses 'default' which is probably already created and using /var/lib/libvirt/images for the path.

This step creates an empty volume that we then copy the undercloud image into with virt-resize.

[sshnaidm]

Still fails. I suspect that pool default is used here (volume_pool: default), so its default location is /var/lib/libvirt/images. .quickstart/tripleo-quickstart/playbooks/roles/images/undercloud/templates/volume_pool.xml.j2:

<pool type='dir'>
  <name>default</name>
  <target>
    <path>/var/lib/libvirt/images/</path>
    <permissions>
      <mode>0755</mode>
      <owner>-1</owner>
      <group>-1</group>
    </permissions>
  </target>
</pool>

[trown]

Ah yep, nice find. That is certainly a bug. Want to submit a patch to use the variables there rather than hardcoded values? path should be {{ vms.volume_path }} name should be {{ vms.volume_pool }}

[trown]

oops... that is the image building role... that does not get used when doing the install

[sshnaidm]

I mean this part: virsh vol-create-as {{ vms.volume_pool}} here and path here: volume_pool: default Actually as I see volume pool is "default" everywhere. Maybe it's easier just to define another pool (like "undecloud-pool") for this purpose? Then it will be more explicit and configurable.

[sshnaidm]

Finally succeeded to run it with pool name and volume path overridden: ansible-playbook -vv $HOME/.quickstart/usr/local/share/tripleo-quickstart/playbooks/quickstart.yml --extra-vars url=$UNDERCLOUD_QCOW2_LOCATION --extra-vars '{"vms":{"volume_path": "/home/img/", "volume_pool": "custom"}}'