ds-identify does not identify openstack Open Telecom Cloud BMS

[ubuntu-server-builder]

This bug was originally filed in Launchpad as LP: #1828236

Launchpad details

affected_projects = []
assignee = None
assignee_name = None
date_closed = None
date_created = 2019-05-08T13:50:56.955867+00:00
date_fix_committed = None
date_fix_released = None
id = 1828236
importance = medium
is_complete = False
lp_url = https://bugs.launchpad.net/cloud-init/+bug/1828236
milestone = None
owner = sabrina-mueller
owner_name = Sabrina
private = False
status = triaged
submitter = sabrina-mueller
submitter_name = Sabrina
tags = ['dsid']
duplicates = []

Launchpad user Sabrina(sabrina-mueller) wrote on 2019-05-08T13:50:56.955867+00:00

relating to cloud-init bugs #1669675 #1756471

The OpenStack detection fails on BMS (Bare Metal) hosts in Open Telekom Cloud.

cloud-init version 18.5 detection fails cloud-init Version 18.2 still works

/usr/lib/python3/dist-packages/cloudinit/sources/DataSourceOpenStack.py searches for (...) 

OpenStack DMI constants

DMI_PRODUCT_NOVA = 'OpenStack Nova' DMI_PRODUCT_COMPUTE = 'OpenStack Compute' VALID_DMI_PRODUCT_NAMES = [DMI_PRODUCT_NOVA, DMI_PRODUCT_COMPUTE] DMI_ASSET_TAG_OPENTELEKOM = 'OpenTelekomCloud' VALID_DMI_ASSET_TAGS = [DMI_ASSET_TAG_OPENTELEKOM] (...)   BMS server in OTC have none of theses tags set. Manual edit DataSourceOpenStack.py only for testing to accept our host, all is working fine.

How we can get back OTC BMS hosts detected as valid OpenStack hosts in cloud-init?

[ubuntu-server-builder]

Launchpad user Sabrina(sabrina-mueller) wrote on 2019-05-08T13:50:56.955867+00:00

Launchpad attachments: cloud-init.tar.gz

[ubuntu-server-builder]

Launchpad user Scott Moser(smoser) wrote on 2019-05-13T19:40:32.249390+00:00

I'm confused. From the uploaded .tar.gz: $ cat run/cloud-init/ds-identify.log [up 29.06s] ds-identify policy loaded: mode=search report=false found=first maybe=all notfound=disabled /etc/cloud/cloud.cfg.d/92_bms_configdrivedisable.cfg set datasource_list: [ NoCloud, OpenStack, Ec2, OpenNebula, Azure, AltCloud, OVF, MAAS, GCE, CloudSigma, CloudStack, None ] DMI_PRODUCT_NAME=RH2288H V3 DMI_SYS_VENDOR=Huawei DMI_PRODUCT_SERIAL=2102311GHQ10H6000507 DMI_PRODUCT_UUID=13D686BA-5D98-11E7-BBB8-F87588B6B9D4 PID_1_PRODUCT_NAME=unavailable DMI_CHASSIS_ASSET_TAG=To be filled by O.E.M. FS_LABELS=cloudimg-rootfs,config-2 ISO9660_DEVS=/dev/sda2=config-2 KERNEL_CMDLINE=BOOT_IMAGE=/boot/vmlinuz-4.15.0-48-generic root=LABEL=cloudimg-rootfs ro console=ttyS0,115200n8 console=tty0 console=tty1 console=ttyS0 VIRT=none UNAME_KERNEL_NAME=Linux UNAME_KERNEL_RELEASE=4.15.0-48-generic UNAME_KERNEL_VERSION=#51-Ubuntu SMP Wed Apr 3 08:28:49 UTC 2019 UNAME_MACHINE=x86_64 UNAME_NODENAME=sab-u18-ci1805 UNAME_OPERATING_SYSTEM=GNU/Linux DSNAME= DSLIST=NoCloud OpenStack Ec2 OpenNebula Azure AltCloud OVF MAAS GCE CloudSigma CloudStack None MODE=search ON_FOUND=first ON_MAYBE=all ON_NOTFOUND=disabled pid=725 ppid=706 is_container=false is_ds_enabled(IBMCloud) = false. ec2 platform is 'Unknown'. No ds found [mode=search, notfound=disabled]. Disabled cloud-init [1] [up 29.09s] returning 1

I dont know why the 'has_fs_with_label' cchekc in check_configdrive_v2 would not have found 'config-2'.

[ubuntu-server-builder]

Launchpad user Sabrina(sabrina-mueller) wrote on 2019-05-14T07:46:34.978564+00:00

Configdrive works so far. BMS hosts works a little bit special in OTC as they needs to load datasources two times. First: configDrive Second: openStack

At first boot DataSource configdrive is used to get network information. After network works cloud-init config is changed by an OTC script to ignore configdrive from now. Cloud-init is triggered to scan and run again. Normally it now finds OpenStack (where we get needed vendordata config).

ConfigDrive will be disabled by script directly in /etc/cloud/cloud.cfg.d/92_bms_configdrivedisable.cfg /etc/cloud/ds-identify.cfg

Scripts doing that can be installed by package bms-network-setup https://download.opensuse.org/repositories/Cloud:/OTC:/Tools/xUbuntu_18.04/

[ubuntu-server-builder]

Launchpad user Scott Moser(smoser) wrote on 2019-05-16T14:51:38.291483+00:00

@Sabrina,

I think what you're saying is that this platform essentially needs to get network information off the config drive and the remainder of the information off of openstack metadata service.

I had somewhat expected this sort of thing would happen.

I think the right approach is probably to extend the OpenStack metadata source for this use case.

[ubuntu-server-builder]

Launchpad user Sabrina(sabrina-mueller) wrote on 2019-05-20T12:37:43.395716+00:00

@Scott,

Did you mean "OpenStack metadata source" or shouldn't it bea "cloud-init DataSourceOpenStack" May I did not get your thought. Independent of this double datasource load I think cloud-init should be able to detect OTC BMS hosts. The datasource openStack is there, reachable and working, only ignored by DataSourceOpenStack.py. We should find a way to detect it or to enforce a custom defined dataSource to be tested again.

[ubuntu-server-builder]

Launchpad user Scott Moser(smoser) wrote on 2019-05-23T13:14:06.864856+00:00

@Sabrina, you were right. when I said "OpenStack metadata source", I meant "DataSourceOpenStack".

Cloud-init no longer "polls" network resources looking for a DataSource. Instead, it requires that it positively identify that it is running on the provided cloud, and thus should expect to find the datasource present.

From the data collected above, I'm not sure of a way that we can positively identify this cloud platform other than the presense of the config drive, and possibly some information inside it. Bug 1788487 describes that cloud-init should use a specific datasource if it is so configured, but that wouldn't even solve your problem as you need network information to come from the config drive and other info from the metadata service.

I can't see any way around the need to extend the OpenStack datasource to identify this scenario.