search

pyinfra-dev / pyinfra

pyinfra turns Python code into shell commands and runs them on your servers. Execute ad-hoc commands and write declarative operations. Target SSH servers, local machine and Docker containers. Fast and scales from one server to thousands.
https://pyinfra.com
MIT License
3.84k stars 374 forks source link

vagrant connector: `--limit` not used early enough and fail in `get_vagrant_config` of unavailble VMs #1024

Open paul-ri opened 10 months ago

paul-ri commented 10 months ago

Describe the bug

With only 1 Vagrant VM existing and up, and an inventory referring to multiple Vagrant VMs by name, --limit is not applied early enough and the Vagrant config of non-existing VMs tries to be fetched but fails

I think the error starts from this line where the Vagrant connector tries to fetch a VM config before applying the limits

https://github.com/pyinfra-dev/pyinfra/blob/7540c41fa0284b8dd80dcbea24d3616414484d60/pyinfra/api/inventory.py#L101

My use case: well it's quite an edge case. I was trying something so replaced all our SSH targets with variations of @vagrant/XXX in order to be sure to not touch the real ones. I was trying to learn how --limit worked. Using mock @ssh/foo works fine as a workaround for me.

To Reproduce

Start one Vagrant VM. Have an inventory referring to two VMs where one doesn't exist yet, then apply a limit.

# inventory.py
prod_a = [
    "@vagrant/default",
]

prod_b = [
    "@vagrant/foo",
]

calling pyinfra with pyinfra inventory.py <some_operation> --limit prod_a will fail due to pyinfra trying to get the config info on prod_b anyways

image

Expected behavior

I would expect for pyinfra to handle the --limit argument and not try to make a connection to VMs not mentioned in --limit

Meta

> pyinfra --support
--> Support information:

    If you are having issues with pyinfra or wish to make feature requests, please
    check out the GitHub issues at https://github.com/Fizzadar/pyinfra/issues .
    When adding an issue, be sure to include the following:

    System: Linux
      Platform: Linux-6.5.9-arch2-1-x86_64-with-glibc2.38
      Release: 6.5.9-arch2-1
      Machine: x86_64
    pyinfra: v2.7
    Executable: /home/paul/git/<REDACTED>/.venv/bin/pyinfra
    Python: 3.11.2 (CPython, GCC 13.1.1 20230429)

image

Fizzadar commented 9 months ago

Interesting bug, unfortunately the way connectors generate inventory means it will always try to load before we can apply a limit. But maybe we shouldn't be failing here at all and actually it should fail at connect time, since the VM config exists.

I'll need to do some testing with Vagrant for this.