search

canonical / cloud-init

Official upstream for the cloud-init: cloud instance initialization
https://cloud-init.io/
Other
2.85k stars 854 forks source link

Strictly disallow import-time code execution #5344

Open holmanb opened 3 months ago

holmanb commented 3 months ago

Bug report

A recent PR fixed some issues in cc_disk_setup.py where cloud-init was executing code during module import which would crawl PATH for binaries - even when this module isn't used. This inspired me to write a tiny static analysis tool to help identify other code execution during import.

It found more than a couple of callsites. We should resolve these issues. No all of these need to change. Assigning a partial to a global variable isn't horrid, however if it can be avoided that would be better. Running code at import time that isn't strictly needed in all cases carries unnecessary cost.

find cloudinit -name '*.py' -type f | xargs ~/git/customs/src/customs.py
__init__.py:51:0: W001: Import invoked a Call
__init__.py:52:0: W001: Import invoked a Call
events.py:35:0: W001: Import invoked a Call
handlers.py:445:0: W001: Import invoked a Call
handlers.py:442:0: W001: Import invoked a Call
handlers.py:444:0: W001: Import invoked a Call
handlers.py:441:0: W001: Import invoked a Call
handlers.py:443:0: W001: Import invoked a Call
templater.py:43:0: W001: Import invoked a Call
templater.py:42:0: W001: Import invoked a Call
show.py:48:0: W001: Import invoked a Call
__init__.py:61:0: W001: Import invoked a Call
cloud_config.py:39:0: W001: Import invoked a Call
renderer.py:24:0: W001: Import invoked a Call
util.py:1214:0: W001: Import invoked a Call
util.py:2779:0: W001: Import invoked a Call
__init__.py:11:0: W001: Import invoked a Call
subp.py:15:0: W001: Import invoked a Call
cc_rh_subscription.py:79:0: W001: Import invoked a Call
cc_apt_configure.py:162:0: W001: Import invoked a Call
cc_final_message.py:60:0: W001: Import invoked a Call
cc_zypper_add_repo.py:79:0: W001: Import invoked a Call
cc_write_files.py:118:0: W001: Import invoked a Call
cc_keys_to_console.py:69:0: W001: Import invoked a Call
cc_ubuntu_drivers.py:53:0: W001: Import invoked a Call
cc_wireguard.py:94:0: W001: Import invoked a Call
cc_wireguard.py:98:0: W001: Import invoked a Call
cc_wireguard.py:16:0: W001: Import invoked a Call
cc_chef.py:67:0: W001: Import invoked a Call
cc_chef.py:27:0: W001: Import invoked a Call
cc_chef.py:79:0: W001: Import invoked a Call
cc_chef.py:37:0: W001: Import invoked a Call
cc_chef.py:95:0: W001: Import invoked a Call
cc_chef.py:149:0: W001: Import invoked a Call
cc_chef.py:68:0: W001: Import invoked a Call
cc_grub_dpkg.py:59:0: W001: Import invoked a Call
cc_phone_home.py:94:0: W001: Import invoked a Call
cc_update_hostname.py:91:0: W001: Import invoked a Call
cc_users_groups.py:186:0: W001: Import invoked a Call
cc_scripts_per_instance.py:40:0: W001: Import invoked a Call
cc_scripts_vendor.py:61:0: W001: Import invoked a Call
cc_ssh.py:164:0: W001: Import invoked a Call
cc_ssh.py:158:0: W001: Import invoked a Call
cc_install_hotplug.py:59:0: W001: Import invoked a Call
cc_rsyslog.py:91:0: W001: Import invoked a Call
cc_rsyslog.py:122:0: W001: Import invoked a Call
cc_rsyslog.py:121:0: W001: Import invoked a Call
cc_set_passwords.py:88:0: W001: Import invoked a Call
cc_set_passwords.py:93:0: W001: Import invoked a Call
cc_ntp.py:324:0: W001: Import invoked a Call
cc_ntp.py:321:0: W001: Import invoked a Call
cc_lxd.py:161:0: W001: Import invoked a Call
cc_landscape.py:113:0: W001: Import invoked a Call
cc_update_etc_hosts.py:97:0: W001: Import invoked a Call
cc_seed_random.py:76:0: W001: Import invoked a Call
cc_salt_minion.py:72:0: W001: Import invoked a Call
cc_package_update_upgrade_install.py:68:0: W001: Import invoked a Call
cc_yum_add_repo.py:126:0: W001: Import invoked a Call
cc_snap.py:113:0: W001: Import invoked a Call
cc_scripts_per_once.py:39:0: W001: Import invoked a Call
cc_fan.py:57:0: W001: Import invoked a Call
cc_ssh_import_id.py:52:0: W001: Import invoked a Call
cc_ca_certs.py:129:0: W001: Import invoked a Call
cc_scripts_user.py:40:0: W001: Import invoked a Call
cc_disk_setup.py:102:0: W001: Import invoked a Call
cc_ssh_authkey_fingerprints.py:40:0: W001: Import invoked a Call
cc_timezone.py:37:0: W001: Import invoked a Call
cc_resizefs.py:53:0: W001: Import invoked a Call
schema.py:371:0: W001: Import invoked a Call
schema.py:116:0: W001: Import invoked a Call
schema.py:372:0: W001: Import invoked a Call
cc_scripts_per_boot.py:39:0: W001: Import invoked a Call
cc_disable_ec2_metadata.py:42:0: W001: Import invoked a Call
cc_keyboard.py:24:0: W001: Import invoked a Call
cc_keyboard.py:67:0: W001: Import invoked a Call
cc_resolv_conf.py:100:0: W001: Import invoked a Call
cc_spacewalk.py:43:0: W001: Import invoked a Call
cc_power_state_change.py:81:0: W001: Import invoked a Call
cc_puppet.py:110:0: W001: Import invoked a Call
cc_mounts.py:123:0: W001: Import invoked a Call
cc_mounts.py:77:0: W001: Import invoked a Call
cc_mounts.py:127:0: W001: Import invoked a Call
cc_mounts.py:119:0: W001: Import invoked a Call
cc_mounts.py:126:0: W001: Import invoked a Call
cc_growpart.py:108:0: W001: Import invoked a Call
cc_growpart.py:100:0: W001: Import invoked a Call
cc_mcollective.py:88:0: W001: Import invoked a Call
cc_runcmd.py:76:0: W001: Import invoked a Call
cc_ubuntu_pro.py:22:0: W001: Import invoked a Call
cc_ubuntu_pro.py:135:0: W001: Import invoked a Call
cc_ubuntu_autoinstall.py:71:0: W001: Import invoked a Call
cc_set_hostname.py:80:0: W001: Import invoked a Call
cc_locale.py:51:0: W001: Import invoked a Call
__init__.py:102:0: W001: Import invoked a Call
status.py:60:0: W001: Import invoked a Call
logs.py:30:0: W001: Import invoked a Call
dmi.py:21:0: W001: Import invoked a Call
DataSourceConfigDrive.py:29:0: W001: Import invoked a Call
__init__.py:172:0: W001: Import invoked a Call
__init__.py:182:0: W001: Import invoked a Call
DataSourceEc2.py:32:0: W001: Import invoked a Call
ec2.py:16:0: W001: Import invoked a Call
azure.py:38:0: W001: Import invoked a Call
azure.py:32:0: W001: Import invoked a Call
netlink.py:32:0: W001: Import invoked a Call
netlink.py:51:0: W001: Import invoked a Call
netlink.py:33:0: W001: Import invoked a Call
netlink.py:50:0: W001: Import invoked a Call
netlink.py:52:0: W001: Import invoked a Call
DataSourceOracle.py:48:0: W001: Import invoked a Call
Analyzed 259 files
holmanb commented 3 months ago

Aside: In the mid to near term, I don't think that it would be difficult to get this tool into a state where it can be part of CI. All that it would take (I think) is to add the ability to disable warnings inline, as well as have ignored function calls (via pyproject.toml config), then publish to PyPI. I'd prefer to have at least one co-maintainer if we go that route.