ibmcb / cbtool

Cloud Rapid Experimentation and Analysis Toolkit
Apache License 2.0
77 stars 49 forks source link

PLM: Ubuntu 22.04 workload vms having duplicate IP addresses #443

Open rayx opened 2 weeks ago

rayx commented 2 weeks ago

While following the instructions on this page to create workload images, I observe that, after I create cb_nullworkload image, if I run vmattach tinyvm twice, the two vms created have the same IP addresses from DHCP server, because:

1) Ubuntu 22.04 system uses systemd-networkd service to get IP address from DHCP server. It sends machine id (/etc/machine-id) by default.

2) cb_nullworkload image has /etc/machine-id. Since there isn't a mechanism to remove or recreate the machine id when creating new vms, all vms created from the image have the same machine id and hence the same IP addresses.

I considered a few approaches:

1) Use cloud-init to recreate machine-id on vm first start-up. Unfortuantely it's impossible. See the discussion here

2) Use cloud-init to change the default behavior of systemd-networkd. cloud-init (netplan actually) has a dhcp-identifier directive. If its value is 'mac', cloud-init adds "ClientIdentifier=mac" line to systemd-networkd config file and changes its defualt behavior. So I used this approach to create initial image and thought systemd-networkd config should persist. Unfortunately that's not the case. As cbtool also uses cloud-init when creating vm and doesn't provide network-config file, it looks like that's considered a config change by cloud-init and new systemd-networkd config file is generated which doesn't include the "ClientIdentifier=mac" line.

3) So I ended up removing machine-id manually after running vmcapture:

 \# guestmount -a cb_nullworkload -i --rw /mnt
 \# truncate -s 0 /mnt/etc/machine-id /mnt/var/lib/dbus/machine-id
 \# guestunmount /mnt

(The command suggested [here](https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1563951/comments/9) perhpas works too).

I wonder if it's possible to run the "truncate -s0 ..." command (or perhaps just "echo -n >...") in CBTOOL before it captures vm image?

rayx commented 6 days ago

It turns out that install script has this support already. See its --cleanupimageid option. Below is the code in lib/auxiliary/dependencies.py which implements the option:

        if options.cleanupimageid :
            _cmd += "; sudo truncate -s 0 /etc/machine-id; sudo rm -rf /var/lib/dbus/machine-id; sudo ln -s /etc/machine-id /var/lib/dbus/machine-id"
rayx commented 5 days ago

I'm reopening the issue to request a small change. When user runs "vmattach check:::" command, CBTOOL should add "--cleanupimageid" option to the generated command. Example:

"~/cbtool/pre_install.sh; ~/cbtool/install --role workload --wks nullworkload,cassandra_ycsb,mongo_ycsb,redis_ycsb --cleanupimageid" 

I believe this can be implemented by modifying initialize_object function in lib/operations/base_operations.py.