storage: calltrace observed when set type: partition for storage_pools

[yizhanglinux]

From the description of pull/64

I designed one case about storage_pool with parition type, see my bellow env/playbook/execution log

[1] Fix key for partition pool... This leads to a failure (crash?) any time a pool of type 'partition' is present.

environment: RHEL-8.2

playbook

---
- hosts: all
  become: true
  vars:
    mount_location: '/opt/test1'
    volume_group_size: '5g'
    volume_size: '4g'

  tasks:
    - include_role:
        name: storage

    - include_tasks: get_unused_disk.yml
      vars:
        min_size: "{{ volume_group_size }}"
        max_return: 3

    - name: Test for Fix key for partition pool 
      block:
        - name: Fix key for partition pool
          include_role:
            name: storage
          vars:
            storage_safe_mode: false
            storage_pools:
              - name: vg
                disks: "{{ unused_disks }}"
                type: partition
                volumes:
                  - name: lv 
                    size: "{{ volume_size }}"
                    mount_point: "{{ mount_location }}"

ansible-playbook -i inventory tests/a.yml -vvvv

---snip---
TASK [storage : debug] ***************************************************************************************************************************************************************************************************
task path: /root/test/storage/tasks/main-blivet.yml:84
ok: [localhost] => {
    "_storage_pools": [
        {
            "disks": [
                "sdb",
                "sde",
                "sdf"
            ],
            "encryption": false,
            "encryption_cipher": null,
            "encryption_key_file": null,
            "encryption_key_size": null,
            "encryption_luks_version": null,
            "encryption_passphrase": null,
            "name": "vg",
            "raid_level": null,
            "state": "present",
            "type": "partition",
            "volumes": [
                {
                    "encryption": false,
                    "encryption_cipher": null,
                    "encryption_key_file": null,
                    "encryption_key_size": null,
                    "encryption_luks_version": null,
                    "encryption_passphrase": null,
                    "fs_create_options": "",
                    "fs_label": "",
                    "fs_overwrite_existing": true,
                    "fs_type": "xfs",
                    "mount_check": 0,
                    "mount_device_identifier": "uuid",
                    "mount_options": "defaults",
                    "mount_passno": 0,
                    "mount_point": "/opt/test1",
                    "name": "lv",
                    "pool": "vg",
                    "raid_chunk_size": null,
                    "raid_device_count": null,
                    "raid_level": null,
                    "raid_metadata_version": null,
                    "raid_spare_count": null,
                    "size": "4g",
                    "state": "present",
                    "type": "lvm"
                }
            ]
        }
    ]
}

TASK [storage : debug] ***************************************************************************************************************************************************************************************************
task path: /root/test/storage/tasks/main-blivet.yml:87
ok: [localhost] => {
    "_storage_volumes": []
}

TASK [storage : get required packages] ***********************************************************************************************************************************************************************************
task path: /root/test/storage/tasks/main-blivet.yml:90
<localhost> ESTABLISH LOCAL CONNECTION FOR USER: root
<localhost> EXEC /bin/sh -c 'echo ~root && sleep 0'
<localhost> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /root/.ansible/tmp `"&& mkdir /root/.ansible/tmp/ansible-tmp-1593503378.2319095-10404-78465090616019 && echo ansible-tmp-1593503378.2319095-10404-78465090616019="` echo /root/.ansible/tmp/ansible-tmp-1593503378.2319095-10404-78465090616019 `" ) && sleep 0'
Using module file /root/test/storage/library/blivet.py
<localhost> PUT /root/.ansible/tmp/ansible-local-9913h7i6w68k/tmpm1zzb2ru TO /root/.ansible/tmp/ansible-tmp-1593503378.2319095-10404-78465090616019/AnsiballZ_blivet.py
<localhost> EXEC /bin/sh -c 'chmod u+x /root/.ansible/tmp/ansible-tmp-1593503378.2319095-10404-78465090616019/ /root/.ansible/tmp/ansible-tmp-1593503378.2319095-10404-78465090616019/AnsiballZ_blivet.py && sleep 0'
<localhost> EXEC /bin/sh -c '/usr/libexec/platform-python /root/.ansible/tmp/ansible-tmp-1593503378.2319095-10404-78465090616019/AnsiballZ_blivet.py && sleep 0'
<localhost> EXEC /bin/sh -c 'rm -f -r /root/.ansible/tmp/ansible-tmp-1593503378.2319095-10404-78465090616019/ > /dev/null 2>&1 && sleep 0'
ok: [localhost] => {
    "actions": [],
    "changed": false,
    "crypts": [],
    "invocation": {
        "module_args": {
            "disklabel_type": null,
            "packages_only": true,
            "pools": [
                {
                    "disks": [
                        "sdb",
                        "sde",
                        "sdf"
                    ],
                    "encryption": false,
                    "encryption_cipher": null,
                    "encryption_key_file": null,
                    "encryption_key_size": null,
                    "encryption_luks_version": null,
                    "encryption_passphrase": null,
                    "name": "vg",
                    "raid_level": null,
                    "state": "present",
                    "type": "partition",
                    "volumes": [
                        {
                            "encryption": false,
                            "encryption_cipher": null,
                            "encryption_key_file": null,
                            "encryption_key_size": null,
                            "encryption_luks_version": null,
                            "encryption_passphrase": null,
                            "fs_create_options": "",
                            "fs_label": "",
                            "fs_overwrite_existing": true,
                            "fs_type": "xfs",
                            "mount_check": 0,
                            "mount_device_identifier": "uuid",
                            "mount_options": "defaults",
                            "mount_passno": 0,
                            "mount_point": "/opt/test1",
                            "name": "lv",
                            "pool": "vg",
                            "raid_chunk_size": null,
                            "raid_device_count": null,
                            "raid_level": null,
                            "raid_metadata_version": null,
                            "raid_spare_count": null,
                            "size": "4g",
                            "state": "present",
                            "type": "lvm"
                        }
                    ]
                }
            ],
            "safe_mode": true,
            "use_partitions": null,
            "volumes": []
        }
    },
    "leaves": [],
    "mounts": [],
    "packages": [
        "lvm2",
        "xfsprogs"
    ],
    "pools": [],
    "volumes": []
}

TASK [storage : make sure required packages are installed] ***************************************************************************************************************************************************************
task path: /root/test/storage/tasks/main-blivet.yml:99
Running dnf
<localhost> ESTABLISH LOCAL CONNECTION FOR USER: root
<localhost> EXEC /bin/sh -c 'echo ~root && sleep 0'
<localhost> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /root/.ansible/tmp `"&& mkdir /root/.ansible/tmp/ansible-tmp-1593503384.8154094-10536-209921407276320 && echo ansible-tmp-1593503384.8154094-10536-209921407276320="` echo /root/.ansible/tmp/ansible-tmp-1593503384.8154094-10536-209921407276320 `" ) && sleep 0'
Using module file /usr/lib/python3.6/site-packages/ansible/modules/packaging/os/dnf.py
<localhost> PUT /root/.ansible/tmp/ansible-local-9913h7i6w68k/tmpd6re1p33 TO /root/.ansible/tmp/ansible-tmp-1593503384.8154094-10536-209921407276320/AnsiballZ_dnf.py
<localhost> EXEC /bin/sh -c 'chmod u+x /root/.ansible/tmp/ansible-tmp-1593503384.8154094-10536-209921407276320/ /root/.ansible/tmp/ansible-tmp-1593503384.8154094-10536-209921407276320/AnsiballZ_dnf.py && sleep 0'
<localhost> EXEC /bin/sh -c '/usr/libexec/platform-python /root/.ansible/tmp/ansible-tmp-1593503384.8154094-10536-209921407276320/AnsiballZ_dnf.py && sleep 0'
<localhost> EXEC /bin/sh -c 'rm -f -r /root/.ansible/tmp/ansible-tmp-1593503384.8154094-10536-209921407276320/ > /dev/null 2>&1 && sleep 0'
ok: [localhost] => {
    "changed": false,
    "invocation": {
        "module_args": {
            "allow_downgrade": false,
            "autoremove": false,
            "bugfix": false,
            "conf_file": null,
            "disable_excludes": null,
            "disable_gpg_check": false,
            "disable_plugin": [],
            "disablerepo": [],
            "download_dir": null,
            "download_only": false,
            "enable_plugin": [],
            "enablerepo": [],
            "exclude": [],
            "install_repoquery": true,
            "install_weak_deps": true,
            "installroot": "/",
            "list": null,
            "lock_timeout": 30,
            "name": [
                "lvm2",
                "xfsprogs"
            ],
            "releasever": null,
            "security": false,
            "skip_broken": false,
            "state": "present",
            "update_cache": false,
            "update_only": false,
            "validate_certs": true
        }
    },
    "msg": "Nothing to do",
    "rc": 0,
    "results": []
}

TASK [storage : manage the pools and volumes to match the specified state] ***********************************************************************************************************************************************
task path: /root/test/storage/tasks/main-blivet.yml:104
<localhost> ESTABLISH LOCAL CONNECTION FOR USER: root
<localhost> EXEC /bin/sh -c 'echo ~root && sleep 0'
<localhost> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /root/.ansible/tmp `"&& mkdir /root/.ansible/tmp/ansible-tmp-1593503389.0837007-10552-256949382030857 && echo ansible-tmp-1593503389.0837007-10552-256949382030857="` echo /root/.ansible/tmp/ansible-tmp-1593503389.0837007-10552-256949382030857 `" ) && sleep 0'
Using module file /root/test/storage/library/blivet.py
<localhost> PUT /root/.ansible/tmp/ansible-local-9913h7i6w68k/tmp20hb2ct9 TO /root/.ansible/tmp/ansible-tmp-1593503389.0837007-10552-256949382030857/AnsiballZ_blivet.py
<localhost> EXEC /bin/sh -c 'chmod u+x /root/.ansible/tmp/ansible-tmp-1593503389.0837007-10552-256949382030857/ /root/.ansible/tmp/ansible-tmp-1593503389.0837007-10552-256949382030857/AnsiballZ_blivet.py && sleep 0'
<localhost> EXEC /bin/sh -c '/usr/libexec/platform-python /root/.ansible/tmp/ansible-tmp-1593503389.0837007-10552-256949382030857/AnsiballZ_blivet.py && sleep 0'
<localhost> EXEC /bin/sh -c 'rm -f -r /root/.ansible/tmp/ansible-tmp-1593503389.0837007-10552-256949382030857/ > /dev/null 2>&1 && sleep 0'
The full traceback is:
Traceback (most recent call last):
  File "/root/.ansible/tmp/ansible-tmp-1593503389.0837007-10552-256949382030857/AnsiballZ_blivet.py", line 102, in <module>
    _ansiballz_main()
  File "/root/.ansible/tmp/ansible-tmp-1593503389.0837007-10552-256949382030857/AnsiballZ_blivet.py", line 94, in _ansiballz_main
    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
  File "/root/.ansible/tmp/ansible-tmp-1593503389.0837007-10552-256949382030857/AnsiballZ_blivet.py", line 40, in invoke_module
    runpy.run_module(mod_name='ansible.modules.blivet', init_globals=None, run_name='__main__', alter_sys=True)
  File "/usr/lib64/python3.6/runpy.py", line 205, in run_module
    return _run_module_code(code, init_globals, run_name, mod_spec)
  File "/usr/lib64/python3.6/runpy.py", line 96, in _run_module_code
    mod_name, mod_spec, pkg_name, script_name)
  File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/tmp/ansible_blivet_payload_pv3jltzf/ansible_blivet_payload.zip/ansible/modules/blivet.py", line 1197, in <module>
  File "/tmp/ansible_blivet_payload_pv3jltzf/ansible_blivet_payload.zip/ansible/modules/blivet.py", line 1194, in main
  File "/tmp/ansible_blivet_payload_pv3jltzf/ansible_blivet_payload.zip/ansible/modules/blivet.py", line 1147, in run_module
  File "/tmp/ansible_blivet_payload_pv3jltzf/ansible_blivet_payload.zip/ansible/modules/blivet.py", line 884, in manage_pool
  File "/tmp/ansible_blivet_payload_pv3jltzf/ansible_blivet_payload.zip/ansible/modules/blivet.py", line 790, in manage
  File "/tmp/ansible_blivet_payload_pv3jltzf/ansible_blivet_payload.zip/ansible/modules/blivet.py", line 772, in _manage_volumes
  File "/tmp/ansible_blivet_payload_pv3jltzf/ansible_blivet_payload.zip/ansible/modules/blivet.py", line 357, in manage
  File "/tmp/ansible_blivet_payload_pv3jltzf/ansible_blivet_payload.zip/ansible/modules/blivet.py", line 461, in _create
AttributeError: 'DiskDevice' object has no attribute 'free_space'
fatal: [localhost]: FAILED! => {
    "changed": false,
    "module_stderr": "Traceback (most recent call last):\n  File \"/root/.ansible/tmp/ansible-tmp-1593503389.0837007-10552-256949382030857/AnsiballZ_blivet.py\", line 102, in <module>\n    _ansiballz_main()\n  File \"/root/.ansible/tmp/ansible-tmp-1593503389.0837007-10552-256949382030857/AnsiballZ_blivet.py\", line 94, in _ansiballz_main\n    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n  File \"/root/.ansible/tmp/ansible-tmp-1593503389.0837007-10552-256949382030857/AnsiballZ_blivet.py\", line 40, in invoke_module\n    runpy.run_module(mod_name='ansible.modules.blivet', init_globals=None, run_name='__main__', alter_sys=True)\n  File \"/usr/lib64/python3.6/runpy.py\", line 205, in run_module\n    return _run_module_code(code, init_globals, run_name, mod_spec)\n  File \"/usr/lib64/python3.6/runpy.py\", line 96, in _run_module_code\n    mod_name, mod_spec, pkg_name, script_name)\n  File \"/usr/lib64/python3.6/runpy.py\", line 85, in _run_code\n    exec(code, run_globals)\n  File \"/tmp/ansible_blivet_payload_pv3jltzf/ansible_blivet_payload.zip/ansible/modules/blivet.py\", line 1197, in <module>\n  File \"/tmp/ansible_blivet_payload_pv3jltzf/ansible_blivet_payload.zip/ansible/modules/blivet.py\", line 1194, in main\n  File \"/tmp/ansible_blivet_payload_pv3jltzf/ansible_blivet_payload.zip/ansible/modules/blivet.py\", line 1147, in run_module\n  File \"/tmp/ansible_blivet_payload_pv3jltzf/ansible_blivet_payload.zip/ansible/modules/blivet.py\", line 884, in manage_pool\n  File \"/tmp/ansible_blivet_payload_pv3jltzf/ansible_blivet_payload.zip/ansible/modules/blivet.py\", line 790, in manage\n  File \"/tmp/ansible_blivet_payload_pv3jltzf/ansible_blivet_payload.zip/ansible/modules/blivet.py\", line 772, in _manage_volumes\n  File \"/tmp/ansible_blivet_payload_pv3jltzf/ansible_blivet_payload.zip/ansible/modules/blivet.py\", line 357, in manage\n  File \"/tmp/ansible_blivet_payload_pv3jltzf/ansible_blivet_payload.zip/ansible/modules/blivet.py\", line 461, in _create\nAttributeError: 'DiskDevice' object has no attribute 'free_space'\n",
    "module_stdout": "",
    "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error",
    "rc": 1
}

PLAY RECAP ***************************************************************************************************************************************************************************************************************
localhost                  : ok=35   changed=0    unreachable=0    failed=1    skipped=15   rescued=0    ignored=0  

[pcahyna]

Thank you but are partition pools really supported? They are not documented in the README. @dwlehman

[yizhanglinux]

Here is the commit which seems suppot it[1], and already have some cases in tests_create_partition_volume_then_remove.yml and tests_disk_errors.yml[2]

[1] b3ee952 Fix key for partition pool class lookup.

[2]

$ cat tests_create_partition_volume_then_remove.yml
---
- hosts: all
  become: true
  vars:
    storage_safe_mode: false
    mount_location: '/opt/test1'

  tasks:
    - include_role:
        name: storage

    - include_tasks: get_unused_disk.yml
      vars:
        max_return: 1

    - name: Create a partition device mounted on "{{ mount_location }}"
      include_role:
        name: storage
      vars:
        storage_pools:
          - name: "{{ unused_disks[0] }}"
            type: partition
            disks: "{{ unused_disks }}"
            volumes:
              - name: test1
                type: partition
                mount_point: "{{ mount_location }}"

[yizhanglinux]

From the log, seems it's due to the "size:" paramter for partition

ansible/modules/blivet.py\", line 461, in _create\nAttributeError: 'DiskDevice' object has no attribute 'free_space'\n"

[pcahyna]

The code is there, but I think it was intentionally not announced to users.

[dwlehman]

You have a volume of type 'lvm' inside a pool of type 'partition', which does not make sense. Obviously we should handle this more gracefully.

[richm]

does https://github.com/linux-system-roles/storage/pull/214 fix this issue?