search

canonical / cloud-init

Official upstream for the cloud-init: cloud instance initialization
https://cloud-init.io/
Other
2.88k stars 857 forks source link

cloud-init - Hosts in softlayer receiving warning #2829

Closed ubuntu-server-builder closed 1 year ago

ubuntu-server-builder commented 1 year ago

This bug was originally filed in Launchpad as LP: #1673637

Launchpad details 
affected_projects = ['cloud-init (Ubuntu)', 'cloud-init (Ubuntu Xenial)', 'cloud-init (Ubuntu Yakkety)', 'cloud-init (Ubuntu Zesty)', 'cloud-init (Ubuntu Artful)']
assignee = None
assignee_name = None
date_closed = 2017-09-23T02:29:52.727898+00:00
date_created = 2017-03-17T00:49:28.864370+00:00
date_fix_committed = 2017-05-25T14:12:01.657977+00:00
date_fix_released = 2017-09-23T02:29:52.727898+00:00
id = 1673637
importance = high
is_complete = True
lp_url = https://bugs.launchpad.net/cloud-init/+bug/1673637
milestone = None
owner = jadunham1
owner_name = Jeff Dunham
private = False
status = fix_released
submitter = jadunham1
submitter_name = Jeff Dunham
tags = ['dsid', 'id-58cc1e3bea5b5c7f55e04e65', 'verification-done-xenial', 'verification-done-yakkety', 'verification-done-zesty']
duplicates = []

Launchpad user Jeff Dunham(jadunham1) wrote on 2017-03-17T00:49:28.864370+00:00

=== Begin cloud-init SRU Template === [Impact] Softlayer user datasource is undetected when only openstack/latest config-drive directory is provided (due to softlayer user-data being provided).

[Test Case]

For cloud-init, the easiest way to demonstrate this is to create a lxc container and populate it with a '/config-drive' that only contains a openstack/latest path.

wget https://git.launchpad.net/~smoser/cloud-init/+git/sru-info/plain/bin/lxc-proposed-snapshot chmod 755 lxc-proposed-snapshot wget https://git.launchpad.net/~smoser/cloud-init/+git/sru-info/plain/tools/make-configdrive-dir chmod 755 make-configdrive-dir name=test-proposed

for release in xenial yaketty zesty; do  ref=$release-proposed;  ./lxc-proposed-snapshot --proposed --publish $release $ref;  ./lxc init $ref $name;

populate a /config-drive with attached 'make-configdrive-dir'

and push it to the container

 d=$(mktemp -d)  ./make-configdrive-dir "$d" "$name"  rm -Rf "$d"  lxc file pull $name/etc/cloud/cloud.cfg.d/90_dpkg.cfg - |     sed 's/NoCloud, //' |     lxc file push - $name/etc/cloud/cloud.cfg.d/90_dpkg.cfg;

 lxc start $name;  sleep 10;  lxc exec $name -- rm -rf /config-drive/openstack/2015-10-15/;  lxc exec $name -- cp -r /config-drive /var/lib/cloud/seed/config_drive;  lxc exec $name -- sudo DEBUG_LEVEL=2 DI_LOG=stderr /usr/lib/cloud-init/ds-identify --force 2>&1 | egrep 'latest|Found' ;

 # should produce config drive seeded directory had only 'latest' done

[Regression Potential] Minimal as this code is exercised for softlayer

[Other Info] Related bugs:

=== End cloud-init SRU Template ===

ubuntu-server-builder commented 1 year ago

Launchpad user Scott Moser(smoser) wrote on 2017-03-22T13:11:30.914170+00:00

Hi, can you please collect the following from the system:

/var/log/cloud-init.log /run/cloud-init/ dmesg > dmesg.txt journalctl -o short-precise > journal.txt

Then, please run: sudo /usr/lib/ds-identify --force

And collect the (now updated) files: /run/cloud-init/cloud.cfg /run/cloud-init/ds-identify.log

Thank you.

ubuntu-server-builder commented 1 year ago

Launchpad user Scott Moser(smoser) wrote on 2017-03-22T13:12:01.716148+00:00

Hi, and once you've done that, please set the status of the bug to 'Confirmed. Thanks.

ubuntu-server-builder commented 1 year ago

Launchpad user Scott Moser(smoser) wrote on 2017-03-22T16:00:52.824959+00:00

Jeff, Also, could you also let me know what cloud platform (OpenStack for example) that you're running on softlayer ? How did the NoCloud datasource get created?

ubuntu-server-builder commented 1 year ago

Launchpad user Scott Moser(smoser) wrote on 2017-03-22T21:02:41.539571+00:00

I dug around some on this, and talked a bit to Robert Jennings who gave me some more info. softlayer works to feed cloud-init a datasource in 2 ways a.) /var/lib/cloud/seed/nocloud-net/ has data in it in the image. /etc/cloud/cloud.cfg.d/99_networklayer_common.cfg contained datasource_list: [ ConfigDrive, NoCloud ]

b.) if user provides metadata (or userdata) then a disk (xvdh1) is attached that has openstack config drive like data on it.

of note here is that it contains *only* 'latest'  as the version.

I really dont like that they provided the versioned information 'latest'. I consider it a bug that cloud-init reads that at all.

The best way forward here is either: 1.) modify the openstack datasource (and ds-identify) to consider this 'METADATA' disk to be a ConfigDrive. That will actually mean that Ubuntu images do not need the tricks they have to mount that disk over /var/lib/cloud/seed/config-drive and would also remove some possible race conditions there. What I don't like about this is that the word 'METADATA' is so generic, and ds-identify doesn't actually mount anything, meaning it could return true for this datasource on someone elses system that just happened to have a disk named METADATA.

2.) create a specific datasource for softlayer that basically just does the above, but only in more specific scenario and not enabled by default.

Heres some information for my own reference later.

sudo blkid

/dev/xvdh1: SEC_TYPE="msdos" LABEL="METADATA" UUID="681B-8C5D" TYPE="vfat" PARTUUID="3d631e09-01" /dev/xvdb1: LABEL="SWAP-xvdb1" UUID="d51fcca0-6b10-4934-a572-f3898dfd8840" TYPE="swap" PARTUUID="00025cdb-01" /dev/xvda1: LABEL="cloudimg-bootfs" UUID="85caed83-8815-4770-94b5-853ca2e03431" TYPE="ext3" PARTUUID="974bb19a-01" /dev/xvda2: LABEL="cloudimg-rootfs" UUID="22ca45b9-13f2-481d-9251-0dd08d36eace" TYPE="ext4" PARTUUID="974bb19a-02"

mount /dev/xvdh1 /mnt

( cd /mnt && for f in $(find * -type f); do echo == $f ==; cat $f; echo; done )

== meta.js == ["#!\/bin\/sh\r\nssh-import-id smoser"] == openstack/latest/user_data ==

!/bin/sh

ssh-import-id smoser == openstack/latest/meta_data.json == {"files":[[]],"network_config":{"content_path":"\/content\/interfaces"},"hostname":"smfoo2","name":"smfoo2","domain":"softlayer.brickies.net","meta":{"dsmode":"net"},"uuid":"f20e93bb-212e-2a17-6d70-c33dbf891c12"} == openstack/content/interfaces == auto lo iface lo inet loopback

auto eth0 allow-hotplug eth0 iface eth0 inet static address 10.170.230.200 netmask 255.255.255.192 post-up route add -net 10.0.0.0 netmask 255.0.0.0 gw 10.170.230.193 post-up route add -net 161.26.0.0 netmask 255.255.0.0 gw 10.170.230.193

auto eth1 allow-hotplug eth1 iface eth1 inet static address 169.45.206.71 netmask 255.255.255.240 gateway 169.45.206.65

rm -f /run/cloud-init/ds-identify.log

sudo DEBUG_LEVEL=3 /usr/lib/cloud-init/ds-identify --force

cat /run/cloud-init/ds-identify.log

cat /run/cloud-init/ds-identify.log

[up 11015.19s] ds-identify --force policy loaded: mode=report report=false found=all maybe=all notfound=enabled /etc/cloud/cloud.cfg.d/99_networklayer_common.cfg set datasource_list: [ ConfigDrive, NoCloud ] DMI_PRODUCT_NAME=unavailable DMI_SYS_VENDOR=unavailable DMI_PRODUCT_SERIAL=unavailable DMI_PRODUCT_UUID=unavailable PID_1_PLATFORM=unavailable FS_LABELS=METADATA,SWAP-xvdb1,cloudimg-bootfs,cloudimg-rootfs KERNEL_CMDLINE=root=UUID=22ca45b9-13f2-481d-9251-0dd08d36eace ro console=hvc0 elevator=noop
VIRT=xen UNAME_KERNEL_NAME=Linux UNAME_KERNEL_RELEASE=4.4.0-45-generic UNAME_KERNEL_VERSION=#66-Ubuntu SMP Wed Oct 19 14:12:37 UTC 2016 UNAME_MACHINE=x86_64 UNAME_NODENAME=smfoo2 UNAME_OPERATING_SYSTEM=GNU/Linux DSNAME= DSLIST=ConfigDrive NoCloud MODE=report ON_FOUND=all ON_MAYBE=all ON_NOTFOUND=enabled pid=10895 ppid=10894 is_container=false Checking for datasource 'ConfigDrive' via 'dscheck_ConfigDrive' check for 'ConfigDrive' returned not-found[1] Checking for datasource 'NoCloud' via 'dscheck_NoCloud' check for 'NoCloud' returned found found=NoCloud maybe= Found single datasource: NoCloud [up 11015.24s] returning 0

ubuntu-server-builder commented 1 year ago

Launchpad user Jeff Dunham(jadunham1) wrote on 2017-03-22T21:43:35.888437+00:00

Do you still need my logs or do you have enough?

ubuntu-server-builder commented 1 year ago

Launchpad user Scott Moser(smoser) wrote on 2017-03-29T21:00:30.493890+00:00

I'm not sure what to do here. I believe this is fixed in current trunk. If you see this in any package versioned > 0.7.9-82-g0e2030ca-0ubuntu1 .

The fix that was put in place was just to specifically allow 'latest' in ds-identify.

There very well could be race conditions though on the mounting of that disk to /var/lib/cloud

ubuntu-server-builder commented 1 year ago

Launchpad user Chris Brinker(chris-brinker) wrote on 2017-04-27T20:07:29.375615+00:00

I'm still seeing this behavior with recently created xenial machines running later versions:

$ dpkg -s cloud-init Package: cloud-init Status: install ok installed Priority: extra Section: admin Installed-Size: 1455 Maintainer: Scott Moser smoser@ubuntu.com Architecture: all Version: 0.7.9-90-g61eb03fe-0ubuntu1~16.04.1

What can I contribute to help debug this persistent issue? I'm hesitant to suppress the warnings, but maybe that is the best course of action right now?

Thanks, -Chris

Warning upon ssh login:

A new feature in cloud-init identified possible datasources for

this system as:

['Ec2', 'None']

However, the datasource used was: NoCloud

In the future, cloud-init will only attempt to use datasources that

are identified or specifically configured.

For more information see

https://bugs.launchpad.net/bugs/1669675

If you are seeing this message, please file a bug against

cloud-init at

https://bugs.launchpad.net/cloud-init/+filebug?field.tags=dsid

Make sure to include the cloud provider your instance is

running on.

After you have filed a bug, you can disable this warning by launching

your instance with the cloud-config below, or putting that content

into /etc/cloud/cloud.cfg.d/99-warnings.cfg

cloud-config

warnings:

dsid_missing_source: off

ubuntu-server-builder commented 1 year ago

Launchpad user Steve Langasek(vorlon) wrote on 2017-05-08T19:04:52.363608+00:00

Hello Jeff, or anyone else affected,

Accepted cloud-init into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/0.7.9-113-g513e99e0-0ubuntu1~16.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

ubuntu-server-builder commented 1 year ago

Launchpad user Steve Langasek(vorlon) wrote on 2017-05-11T20:30:07.847681+00:00

Hello Jeff, or anyone else affected,

Accepted cloud-init into yakkety-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/0.7.9-113-g513e99e0-0ubuntu1~16.10.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

ubuntu-server-builder commented 1 year ago

Launchpad user Steve Langasek(vorlon) wrote on 2017-05-11T20:37:05.601558+00:00

Hello Jeff, or anyone else affected,

Accepted cloud-init into zesty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/0.7.9-113-g513e99e0-0ubuntu1~17.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

ubuntu-server-builder commented 1 year ago

Launchpad user Chad Smith(chad.smith) wrote on 2017-05-11T20:43:20.070725+00:00

Validated xenial properly discovers configdrive when only the "latest" subdir is present: $ name=test-proposed $ release=xenial $ d=$(mktemp -d) $ wget https://git.launchpad.net/~smoser/cloud-init/+git/sru-info/plain/tools/make-configdrive-dir make-configdrive-di 100%[===================>] 1.66K --.-KB/s in 0s $ chmod 755 make-configdrive-dir lxc init $ref $name Creating test-proposed $ ./make-configdrive-dir $d $name config-drive/ config-drive/openstack/ config-drive/openstack/latest/ config-drive/openstack/latest/meta_data.json config-drive/openstack/latest/vendor_data.json config-drive/openstack/latest/user_data config-drive/openstack/2015-10-15/ config-drive/openstack/2015-10-15/meta_data.json config-drive/openstack/2015-10-15/vendor_data.json config-drive/openstack/2015-10-15/user_data $ rm -Rf $d $ lxc file pull $name/etc/cloud/cloud.cfg.d/90_dpkg.cfg - | sed 's/NoCloud, //' | lxc file push - $name/etc/cloud/cloud.cfg.d/90_dpkg.cfg $ lxc exec $name -- sudo DEBUG_LEVEL=2 DI_LOG=stderr /usr/lib/cloud-init/ds-identify --force 2>&1 | egrep 'latest|Found' config drive seeded directory had only 'latest' config drive seeded directory had only 'latest' Found single datasource: ConfigDrive $ lxc exec $name -- dpkg -l cloud-init ... ii cloud-init 0.7.9-113-g5 all Init scripts for cloud instances

ubuntu-server-builder commented 1 year ago

Launchpad user Chad Smith(chad.smith) wrote on 2017-05-12T19:24:19.872396+00:00

Validated yakkety and zesty---- $ for release in yakkety zesty; do ref=$release-proposed; lxc init $ref $name; d=$(mktemp -d); ./make-configdrive-dir $d $name; rm -rf $d; lxc file pull $name/etc/cloud/cloud.cfg.d/90_dpkg.cfg - | sed 's/NoCloud, //' | lxc file push - $name/etc/cloud/cloud.cfg.d/90_dpkg.cfg; lxc start $name; sleep 10; lxc exec $name -- rm -rf /config-drive/openstack/2015-10-15/; lxc exec $name -- cp -r /config-drive /var/lib/cloud/seed/config_drive; lxc exec $name -- sudo DEBUG_LEVEL=2 DI_LOG=stderr /usr/lib/cloud-init/ds-identify --force 2>&1 | egrep 'latest|Found' ; lxc exec $name -- dpkg -l cloud-init; lxc exec $name -- grep VERSION= /etc/os-release; lxc stop $name; lxc delete $name; done;

Creating proposed-test config-drive/ config-drive/openstack/ config-drive/openstack/latest/ config-drive/openstack/latest/meta_data.json config-drive/openstack/latest/vendor_data.json config-drive/openstack/latest/user_data config-drive/openstack/2015-10-15/ config-drive/openstack/2015-10-15/meta_data.json config-drive/openstack/2015-10-15/vendor_data.json config-drive/openstack/2015-10-15/user_data config drive seeded directory had only 'latest' config drive seeded directory had only 'latest' Found single datasource: ConfigDrive Desired=Unknown/Install/Remove/Purge/Hold | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad) ||/ Name Version Architecture Description +++-==============-============-============-================================= ii cloud-init 0.7.9-113-g5 all Init scripts for cloud instances VERSION="16.10 (Yakkety Yak)" Creating proposed-test config-drive/ config-drive/openstack/ config-drive/openstack/latest/ config-drive/openstack/latest/meta_data.json config-drive/openstack/latest/vendor_data.json config-drive/openstack/latest/user_data config-drive/openstack/2015-10-15/ config-drive/openstack/2015-10-15/meta_data.json config-drive/openstack/2015-10-15/vendor_data.json config-drive/openstack/2015-10-15/user_data config drive seeded directory had only 'latest' config drive seeded directory had only 'latest' Found single datasource: ConfigDrive Desired=Unknown/Install/Remove/Purge/Hold | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad) ||/ Name Version Architecture Description +++-==============-============-============-================================= ii cloud-init 0.7.9-113-g5 all Init scripts for cloud instances VERSION="17.04 (Zesty Zapus)"

ubuntu-server-builder commented 1 year ago

Launchpad user Launchpad Janitor(janitor) wrote on 2017-05-17T02:58:17.479191+00:00

This bug was fixed in the package cloud-init - 0.7.9-113-g513e99e0-0ubuntu1~16.04.1

cloud-init (0.7.9-113-g513e99e0-0ubuntu1~16.04.1) xenial-proposed; urgency=medium

ubuntu-server-builder commented 1 year ago

Launchpad user Chris Halse Rogers(raof) wrote on 2017-05-17T02:58:52.270382+00:00

The verification of the Stable Release Update for cloud-init has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

ubuntu-server-builder commented 1 year ago

Launchpad user Launchpad Janitor(janitor) wrote on 2017-05-17T03:07:38.254895+00:00

This bug was fixed in the package cloud-init - 0.7.9-113-g513e99e0-0ubuntu1~16.10.1

cloud-init (0.7.9-113-g513e99e0-0ubuntu1~16.10.1) yakkety; urgency=medium

ubuntu-server-builder commented 1 year ago

Launchpad user Launchpad Janitor(janitor) wrote on 2017-05-17T03:14:07.523948+00:00

This bug was fixed in the package cloud-init - 0.7.9-113-g513e99e0-0ubuntu1~17.04.1

cloud-init (0.7.9-113-g513e99e0-0ubuntu1~17.04.1) zesty; urgency=medium

ubuntu-server-builder commented 1 year ago

Launchpad user Scott Moser(smoser) wrote on 2017-09-23T02:29:55.052045+00:00

This bug is believed to be fixed in cloud-init in 17.1. If this is still a problem for you, please make a comment and set the state back to New

Thank you.