fix: image transformer none value in requirements

[dav-pascual]

When OS image in metadata is not found in provisioning config, there was a traceback causing mrack to fail. Adding default value solves the problem and prints expected message.

[Tiboris]

Hi @dav-pascual, thanks for the patch. Could you please paste before/after run log?

[dav-pascual]

Before:

2022-04-04 12:29:57,489 mrack.transformers.transformer INFO OpenStack: Created requirement(s): [
    {
        "config_drive": true,
        "flavor": "ci.standard.medium",
        "group": "ipaserver",
        "image": "idm-rhel-8.5-ga-latest",
        "key_name": "idm-jenkins",
        "name": "r85.mrack.test",
        "network": "IPv4",
        "os": "rhel-8.5"
    },
    {
        "config_drive": true,
        "flavor": "ci.standard.medium",
        "group": "ipaserver",
        "image": null,
        "key_name": "idm-jenkins",
        "name": "r73.mrack.test",
        "network": "IPv4",
        "os": "rhel-7.3"
    },
    {
        "config_drive": true,
        "flavor": "ci.standard.medium",
        "group": "ipaserver",
        "image": "idm-rhel-9.0-Server-nightly-latest",
        "key_name": "idm-jenkins",
        "name": "r9.mrack.test",
        "network": "IPv4",
        "os": "rhel-9.0"
    },
    {
        "config_drive": true,
        "flavor": "ci.standard.medium",
        "group": "ipaserver",
        "image": "idm-Fedora-Cloud-Base-33-latest",
        "key_name": "idm-jenkins",
        "name": "f33.mrack.test",
        "network": "IPv4",
        "os": "fedora-33"
    }
]
2022-04-04 12:29:57,489 mrack.providers.provider INFO OpenStack: Preparing provider resources
2022-04-04 12:29:57,490 mrack.actions.up ERROR An unexpected exception occurred while provisioning
2022-04-04 12:29:57,490 mrack.run ERROR sequence item 0: expected str instance, NoneType found
Traceback (most recent call last):
  File "/home/davherna/src/idmops/mrack/src/mrack/run.py", line 201, in handle
    ret_code = func(*args, **kwargs)
  File "/home/davherna/src/idmops/mrack/src/mrack/run.py", line 225, in run
    mrackcli(obj={})  # pylint: disable=no-value-for-parameter,unexpected-keyword-arg
  File "/home/davherna/src/idmops/idm-ci/venv/lib64/python3.10/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/home/davherna/src/idmops/idm-ci/venv/lib64/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/home/davherna/src/idmops/idm-ci/venv/lib64/python3.10/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/davherna/src/idmops/idm-ci/venv/lib64/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/davherna/src/idmops/idm-ci/venv/lib64/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/home/davherna/src/idmops/idm-ci/venv/lib64/python3.10/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/davherna/src/idmops/mrack/src/mrack/run.py", line 62, in wrapper
    return loop.run_until_complete(func(*args, **kwargs))
  File "/usr/lib64/python3.10/asyncio/base_events.py", line 646, in run_until_complete
    return future.result()
  File "/home/davherna/src/idmops/mrack/src/mrack/run.py", line 122, in up
    await up_action.provision()
  File "/home/davherna/src/idmops/mrack/src/mrack/actions/up.py", line 106, in provision
    raise results
  File "/home/davherna/src/idmops/mrack/src/mrack/providers/provider.py", line 367, in provision_hosts
    if not await self.prepare_provisioning(reqs):
  File "/home/davherna/src/idmops/mrack/src/mrack/providers/openstack.py", line 478, in prepare_provisioning
    im_list = ", ".join(prepare_images)
TypeError: sequence item 0: expected str instance, NoneType found

[dav-pascual]

After fix:

2022-04-04 12:48:12,289 mrack.transformers.transformer INFO OpenStack: Created requirement(s): [
    {
        "config_drive": true,
        "flavor": "ci.standard.medium",
        "group": "ipaserver",
        "image": "idm-rhel-8.5-ga-latest",
        "key_name": "idm-jenkins",
        "name": "r85.mrack.test",
        "network": "IPv4",
        "os": "rhel-8.5"
    },
    {
        "config_drive": true,
        "flavor": "ci.standard.medium",
        "group": "ipaserver",
        "image": "rhel-7.3",
        "key_name": "idm-jenkins",
        "name": "r73.mrack.test",
        "network": "IPv4",
        "os": "rhel-7.3"
    },
    {
        "config_drive": true,
        "flavor": "ci.standard.medium",
        "group": "ipaserver",
        "image": "idm-rhel-9.0-Server-nightly-latest",
        "key_name": "idm-jenkins",
        "name": "r9.mrack.test",
        "network": "IPv4",
        "os": "rhel-9.0"
    },
    {
        "config_drive": true,
        "flavor": "ci.standard.medium",
        "group": "ipaserver",
        "image": "idm-Fedora-Cloud-Base-33-latest",
        "key_name": "idm-jenkins",
        "name": "f33.mrack.test",
        "network": "IPv4",
        "os": "fedora-33"
    }
]
2022-04-04 12:48:12,945 mrack.providers.openstack INFO OpenStack: Validating host: {
    "config_drive": true,
    "flavor": "ci.standard.medium",
    "group": "ipaserver",
    "image": "idm-rhel-8.5-ga-latest",
    "key_name": "idm-jenkins",
    "name": "r85.mrack.test",
    "network": "shared_net_6",
    "os": "rhel-8.5"
}
2022-04-04 12:48:12,945 mrack.providers.openstack INFO OpenStack: r85.mrack.test - OK
2022-04-04 12:48:12,945 mrack.providers.openstack INFO OpenStack: Validating host: {
    "config_drive": true,
    "flavor": "ci.standard.medium",
    "group": "ipaserver",
    "image": "rhel-7.3",
    "key_name": "idm-jenkins",
    "name": "r73.mrack.test",
    "network": "shared_net_6",
    "os": "rhel-7.3"
}
2022-04-04 12:48:12,946 mrack.actions.up ERROR An unexpected exception occurred while provisioning
2022-04-04 12:48:12,946 mrack.run ERROR Image not found image: rhel-7.3, ref: None

[pvoborni]

This solution is better than before, but I don't think it is correct. The issue is in os-image mapping. I.e. there is no image defined for the os, e.g. "rhel-7.3". Thus the error message should be something like: "Image name not found for os: rhel-7.3".

ERROR Image not found image: rhel-7.3. Should be raised when there is no image with name rhel-7.3 in OpenStack.

The validation error might be raised in the transformer itself as it is the place where the issue happens.

[Tiboris]

@pvoborni Hi, i do not agree with your comment that it is not correct behavior. Before https://github.com/neoave/mrack/commit/c3a91ad9c36113ad14d4ebbab8985616a0a1359c commit the behavior was exactly the same and the idea was to use the 'os' as the fallback 'image' and try to provision like this (with for example pointing to exact ami/openstack image/distro etc.). This was always something like hidden feature and the error was correctly (assuming the above behavior present in the releases up to https://github.com/neoave/mrack/releases/tag/v1.2.0 is the correct)

[dav-pascual]

If we fallback to 'os' when there is no image, then it's not a transformer issue, but a provider issue because it doesnt find an image called 'rhel-7.3' ('os' metadata value).

[pvoborni]

OK, using 'os' as image fallback is probably better then having a "default" os defined (possible without this MR) as it might be more confusing. Anyway we should document this.

[Tiboris]

Definitely a good point to document this.

[Tiboris]

slightly updating the docs in https://github.com/neoave/mrack/pull/174