search

pyinfra-dev / pyinfra

pyinfra turns Python code into shell commands and runs them on your servers. Execute ad-hoc commands and write declarative operations. Target SSH servers, local machine and Docker containers. Fast and scales from one server to thousands.
https://pyinfra.com
MIT License
3.91k stars 383 forks source link

No running Vagrant instances found #644

Closed int0x80 closed 3 years ago

int0x80 commented 3 years ago

Describe the bug

pyinfra cannot find a running Vagrant instance.

To Reproduce

Steps to reproduce the behavior (include code & usage example):

Vagrantfile:

Vagrant.configure("2") do |config|
  config.vm.box = "generic/ubuntu2004"
  config.vm.hostname = "lol"
end

inventory.py:

my_hosts = ['@vagrant/default', '@vagrant/focal64', 'lol', '@vagrant/ubuntu2004']  # define a group as a list of hosts

deploy.py:

from pyinfra.operations import apt, server

server.shell(
    name='Run an ad-hoc command',  # optional name for the operation
    commands='id',
)

# Define some state - this operation will do nothing on subsequent runs
apt.packages(
    name='Ensure the apt packages are installed',
    packages=['git', 'i3', 'tmux', 'vim'],
    sudo=True,  # use sudo when installing the packages
)

Running:

$ vagrant status --machine-readable
1629155671,default,metadata,provider,vmware_desktop
1629155672,default,provider-name,vmware_desktop
1629155672,default,state,running
1629155672,default,state-human-short,running
1629155672,default,state-human-long,The VM is running. To stop this VM%!(VAGRANT_COMMA) you can run `vagrant halt` to\nshut it down%!(VAGRANT_COMMA) or you can run `vagrant suspend` to simply suspend\nthe virtual machine. In either case%!(VAGRANT_COMMA) to restart it again%!(VAGRANT_COMMA) run\n`vagrant up`.
1629155672,,ui,info,Current machine states:\n\ndefault                   running (vmware_desktop)\n\nThe VM is running. To stop this VM%!(VAGRANT_COMMA) you can run `vagrant halt` to\nshut it down%!(VAGRANT_COMMA) or you can run `vagrant suspend` to simply suspend\nthe virtual machine. In either case%!(VAGRANT_COMMA) to restart it again%!(VAGRANT_COMMA) run\n`vagrant up`.

$ pyinfra inventory.py deploy.py
--> Loading config...
--> Loading inventory...
    Getting Vagrant config...
    Getting Vagrant config...
--> pyinfra error: No running Vagrant instances found!

Expected behavior

Expected pyinfra to run id on the VM and install packages and dependencies via apt: git, i3, tmux, vim.

Meta

$ pyinfra --support
--> Support information:

    If you are having issues with pyinfra or wish to make feature requests, please
    check out the GitHub issues at https://github.com/Fizzadar/pyinfra/issues .
    When adding an issue, be sure to include the following:

    System: Darwin
      Platform: macOS-10.15.7-x86_64-i386-64bit
      Release: 19.6.0
      Machine: x86_64
    pyinfra: v1.4.12
    Executable: /usr/local/bin/pyinfra
    Python: 3.8.2 (CPython, Clang 12.0.0 (clang-1200.0.32.21))
$ sudo pip3 install pyinfra
$ pyinfra --version
pyinfra: v1.4.12
$ pyinfra -vvv --debug --debug-facts --debug-operations inventory.py deploy.py
    [pyinfra_cli.main] Checking potential directory:
    [pyinfra_cli.main] Deploy directory remains as cwd
--> Loading config...
--> Loading inventory...
    [pyinfra_cli.inventory] Creating fake inventory...
    Getting Vagrant config...
localhost: >>> vagrant status --machine-readable
localhost: 1629155407,default,metadata,provider,vmware_desktop
localhost: 1629155409,default,provider-name,vmware_desktop
localhost: 1629155409,default,state,running
localhost: 1629155409,default,state-human-short,running
localhost: 1629155409,default,state-human-long,The VM is running. To stop this VM%!(VAGRANT_COMMA) you can run `vagrant halt` to\nshut it down%!(VAGRANT_COMMA) or you can run `vagrant suspend` to simply suspend\nthe virtual machine. In either case%!(VAGRANT_COMMA) to restart it again%!(VAGRANT_COMMA) run\n`vagrant up`.
localhost: 1629155409,,ui,info,Current machine states:\n\ndefault                   running (vmware_desktop)\n\nThe VM is running. To stop this VM%!(VAGRANT_COMMA) you can run `vagrant halt` to\nshut it down%!(VAGRANT_COMMA) or you can run `vagrant suspend` to simply suspend\nthe virtual machine. In either case%!(VAGRANT_COMMA) to restart it again%!(VAGRANT_COMMA) run\n`vagrant up`.
[pyinfra.api.connectors.util] --> Waiting for exit status...
[pyinfra.api.connectors.util] --> Command exit status: 0
    [pyinfra.api.connectors.vagrant] Loading SSH config for default
localhost: >>> vagrant ssh-config default
localhost: Host default
localhost:   HostName 127.0.0.1
localhost:   User vagrant
localhost:   Port 2231
localhost:   UserKnownHostsFile /dev/null
localhost:   StrictHostKeyChecking no
localhost:   PasswordAuthentication no
localhost:   IdentityFile /private/tmp/lol/.vagrant/machines/default/vmware_desktop/private_key
localhost:   IdentitiesOnly yes
localhost:   LogLevel FATAL
localhost:
[pyinfra.api.connectors.util] --> Waiting for exit status...
[pyinfra.api.connectors.util] --> Command exit status: 0
    [pyinfra.api.connectors.vagrant] Got Vagrant SSH info:
['Host default', 'HostName 127.0.0.1', 'User vagrant', 'Port 2231', 'UserKnownHostsFile /dev/null', 'StrictHostKeyChecking no', 'PasswordAuthentication no', 'IdentityFile /private/tmp/lol/.vagrant/machines/default/vmware_desktop/private_key', 'IdentitiesOnly yes', 'LogLevel FATAL', '']
    Getting Vagrant config...
localhost: >>> vagrant status --machine-readable
localhost: 1629155412,default,metadata,provider,vmware_desktop
localhost: 1629155413,default,provider-name,vmware_desktop
localhost: 1629155413,default,state,running
localhost: 1629155413,default,state-human-short,running
localhost: 1629155413,default,state-human-long,The VM is running. To stop this VM%!(VAGRANT_COMMA) you can run `vagrant halt` to\nshut it down%!(VAGRANT_COMMA) or you can run `vagrant suspend` to simply suspend\nthe virtual machine. In either case%!(VAGRANT_COMMA) to restart it again%!(VAGRANT_COMMA) run\n`vagrant up`.
localhost: 1629155413,,ui,info,Current machine states:\n\ndefault                   running (vmware_desktop)\n\nThe VM is running. To stop this VM%!(VAGRANT_COMMA) you can run `vagrant halt` to\nshut it down%!(VAGRANT_COMMA) or you can run `vagrant suspend` to simply suspend\nthe virtual machine. In either case%!(VAGRANT_COMMA) to restart it again%!(VAGRANT_COMMA) run\n`vagrant up`.
[pyinfra.api.connectors.util] --> Waiting for exit status...
[pyinfra.api.connectors.util] --> Command exit status: 0
    [pyinfra.api.connectors.vagrant] Got Vagrant SSH info:
[]
--> pyinfra error: No running Vagrant instances found!

Verifying:

$ vagrant ssh
vagrant@lol:~$ dpkg -l | grep -c 'i3-wm'
0
int0x80 commented 3 years ago

Resolved. My approach was incorrect with inventory.py. 

my_hosts = ['@vagrant/default']  # define a group as a list of hosts

Running:

$ pyinfra -vv --debug inventory.py deploy.py
    [pyinfra_cli.main] Checking potential directory:
    [pyinfra_cli.main] Deploy directory remains as cwd
--> Loading config...
--> Loading inventory...
    [pyinfra_cli.inventory] Creating fake inventory...
    Getting Vagrant config...
localhost: >>> vagrant status --machine-readable
[pyinfra.api.connectors.util] --> Waiting for exit status...
[pyinfra.api.connectors.util] --> Command exit status: 0
    [pyinfra.api.connectors.vagrant] Loading SSH config for default
localhost: >>> vagrant ssh-config default
[pyinfra.api.connectors.util] --> Waiting for exit status...
[pyinfra.api.connectors.util] --> Command exit status: 0
    [pyinfra.api.connectors.vagrant] Got Vagrant SSH info:
['Host default', 'HostName 127.0.0.1', 'User vagrant', 'Port 2231', 'UserKnownHostsFile /dev/null', 'StrictHostKeyChecking no', 'PasswordAuthentication no', 'IdentityFile /private/tmp/lol/.vagrant/machines/default/vmware_desktop/private_key', 'IdentitiesOnly yes', 'LogLevel FATAL', '']
    [pyinfra.api.connectors.vagrant] Got Vagrant SSH info:
['Host default', 'HostName 127.0.0.1', 'User vagrant', 'Port 2231', 'UserKnownHostsFile /dev/null', 'StrictHostKeyChecking no', 'PasswordAuthentication no', 'IdentityFile /private/tmp/lol/.vagrant/machines/default/vmware_desktop/private_key', 'IdentitiesOnly yes', 'LogLevel FATAL', '']

--> Connecting to hosts...
    [pyinfra.api.connectors.ssh] Connecting to: @vagrant/default ({'allow_agent': False, 'look_for_keys': False, 'hostname': '127.0.0.1', 'username': 'vagrant', 'port': 2231, 'timeout': 10, 'pkey': <paramiko.rsakey.RSAKey object at 0x10742c1c0>})
    [@vagrant/default] Connected
    [pyinfra.api.state] Activating host: @vagrant/default

--> Preparing operations...
    Loading: deploy.py
    [pyinfra.api.operation] Adding operation, {'Run an ad-hoc command'}, opOrder=(3,), opHash=08743582456b52abe1182f5a5a3e12b457ba28b8
    [pyinfra.api.operation] Adding operation, {'Ensure the apt packages are installed'}, opOrder=(9,), opHash=c11b7766912fe844d36569f005f08e2e2799e6c3
    [pyinfra.api.facts] Getting fact: deb_packages () (ensure_hosts: (Host(@vagrant/default),))
    [pyinfra.api.connectors.ssh] Running command on @vagrant/default: (pty=False) sudo -H -n sh -c '! command -v dpkg >/dev/null || dpkg -l'
[@vagrant/default] >>> sudo -H -n sh -c '! command -v dpkg >/dev/null || dpkg -l'
    [pyinfra.api.connectors.ssh] Waiting for exit status...
    [pyinfra.api.connectors.ssh] Command exit status: 0
    Loaded fact deb_packages ()
    [@vagrant/default] noop: package git is installed (1:2.25.1-1ubuntu3.1)
    [@vagrant/default] noop: package tmux is installed (3.0a-2ubuntu0.3)
    [@vagrant/default] noop: package vim is installed (2:8.1.2269-1ubuntu5)
    [@vagrant/default] Ready: deploy.py

--> Proposed changes:
    Groups: inventory / @vagrant / my_hosts
    [@vagrant/default]   Operations: 2   Commands: 2

--> Beginning operation run...
--> Starting operation: Run an ad-hoc command
    [pyinfra.api.operations] Starting operation Run an ad-hoc command on @vagrant/default
    [pyinfra.api.connectors.ssh] Running command on @vagrant/default: (pty=None) sh -c id
[@vagrant/default] >>> sh -c id
    [pyinfra.api.connectors.ssh] Waiting for exit status...
    [pyinfra.api.connectors.ssh] Command exit status: 0
    [@vagrant/default] Success

--> Starting operation: Ensure the apt packages are installed
    [pyinfra.api.operations] Starting operation Ensure the apt packages are installed on @vagrant/default
    [pyinfra.api.connectors.ssh] Running command on @vagrant/default: (pty=None) sudo -H -n sh -c 'DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install i3'
[@vagrant/default] >>> sudo -H -n sh -c 'DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install i3'
    [pyinfra.api.connectors.ssh] Waiting for exit status...
    [pyinfra.api.connectors.ssh] Command exit status: 0
    [@vagrant/default] Success

--> Results:
    Groups: inventory / @vagrant / my_hosts
    [@vagrant/default]   Successful: 2   Errors: 0   Commands: 2/2
Fizzadar commented 3 years ago

@int0x80 thank you for raising this - the error in the first instance was definitely not helpful. I've just pushed/released an improvement for that in https://github.com/Fizzadar/pyinfra/commit/aed9f111dc80cd0244f285c45e69f6a677c32d37 / v1.4.13.