voxpupuli / puppet-python

Puppet module for installing and managing Python, pip, virtualenvs and Gunicorn virtual hosts.
https://forge.puppetlabs.com/puppet/python
Apache License 2.0
199 stars 375 forks source link

Wrong pip referenced inside virtualenv #505

Closed jameskirsop closed 4 years ago

jameskirsop commented 5 years ago

Affected Puppet, Ruby, OS and module versions/distributions

How to reproduce (e.g Puppet code you use)

python::virtualenv {'es_stats_venv':
    version => '3.4',
    virtualenv => 'virtualenv-3.4',
    venv_dir => '/etc/zabbix/es_stats_venv',
    owner => 'zabbix',
}

python::pip { 'es_stats_zabbix' :
    ensure => 'present',
    pkgname  => 'es_stats_zabbix',
    owner => 'root',
    pip_provider  => 'pip3',
    virtualenv => '/etc/zabbix/es_stats_venv',
}

python::pip { 'es_stats' :
    ensure => 'present',
    pkgname  => 'es_stats',
    owner => 'root',
    pip_provider  => 'pip3',
    virtualenv => '/etc/zabbix/es_stats_venv',
}

What are you seeing

Error: /Stage[main]/Graylog_els_cluster_node/Python::Pip[es_stats_zabbix]/Exec[pip_install_es_stats_zabbix]: Could not evaluate: Could not find command '/etc/zabbix/es_stats_venv/bin/pip3.6'
Error: /Stage[main]/Graylog_els_cluster_node/Python::Pip[es_stats]/Exec[pip_install_es_stats]: Could not evaluate: Could not find command '/etc/zabbix/es_stats_venv/bin/pip3.6'

What behaviour did you expect instead

No errors, execution of pip at the valid path of /etc/zabbix/es_stats_venv/bin/pip3 which exists inside the venv.

I've also tried to specify pip3.4 as the pip_provider, which is valid inside the venv. Specifying this has no affect on the outcome.

Other virtualenvs are affected also, this is just an example showing similar symptoms.

aeva-assured commented 4 years ago

Seeing what I think may be the same bug

  python::virtualenv { '/var/adp/collector2':
    ensure        => present,
    version       => '3.6',
    owner         => 'portaluser',
    group         => 'portalgroup',
    venv_dir      => '/home/portaluser/virtualenvs',
    systempkgs    => false,
   }

  python::pip { 'mysql-connector-python':
    pkgname       => 'mysql-connector-python',
    ensure        => '8.0.16',
    virtualenv    => '/home/portaluser/virtualenvs',
    pip_provider  => 'pip3',
    url           => 'http://repository/Pip/mysql_connector_python-8.0.16-cp36-cp36m-manylinux1_x86_64.whl',
    require        => [ Package['python36-pip'], ],
   }

Puppet Run result:

Error: /Stage[main]/Adp_collector::Install/Python::Pip[mysql-connector-python]/Exec[pip_install_mysql-connector-python]: Could not evaluate: Could not find command '/home/portaluser/virtualenvs/bin/pip2'

View of the contents of that directory:

[root@hostname bin]# pwd; ls -l
/home/portaluser/virtualenvs/bin
total 56
-rw-r--r--. 1 portaluser portaluser  2087 Oct 15 19:40 activate
-rw-r--r--. 1 portaluser portaluser  1029 Oct 15 19:40 activate.csh
-rw-r--r--. 1 portaluser portaluser  2183 Oct 15 19:40 activate.fish
-rw-r--r--. 1 portaluser portaluser  1137 Oct 15 19:40 activate_this.py
-rwxr-xr-x. 1 portaluser portaluser   260 Oct 15 19:40 easy_install
-rwxr-xr-x. 1 portaluser portaluser   260 Oct 15 19:40 easy_install-3.6
-rwxr-xr-x. 1 portaluser portaluser   247 Oct 15 19:40 pip
-rwxr-xr-x. 1 portaluser portaluser   247 Oct 15 19:40 pip3
-rwxr-xr-x. 1 portaluser portaluser   247 Oct 15 19:40 pip3.6
lrwxrwxrwx. 1 portaluser portaluser     9 Oct 15 19:40 python -> python3.6
lrwxrwxrwx. 1 portaluser portaluser     9 Oct 15 19:40 python3 -> python3.6
-rwxr-xr-x. 1 portaluser portaluser 11408 Oct 15 19:40 python3.6
-rwxr-xr-x. 1 portaluser portaluser  2346 Oct 15 19:40 python-config
-rwxr-xr-x. 1 portaluser portaluser   239 Oct 15 19:40 wheel
[root@hostname bin]#

I was able to use a temporary workaround by dropping in a symbolic link from pip2 -> pip3.

jplindquist commented 4 years ago

Running into the same issue, and it seems to be from this resource collector: https://github.com/voxpupuli/puppet-python/blob/b88ea1947aafb9d8333d07a6927586baea51fb40/manifests/install.pp#L294-L296 (fixed line copy)

I commented that snippet out locally, and it ran flawlessly for what I was doing. I don't know if that is there for a reason, but I'm happy to write up a PR if removing that is a reasonable solution

fnoop commented 4 years ago

Also happens in the global env, not a virtualenv. pip_provider is overriden by 'pip' - was driving me mad until I found this issue and workaround. Thanks :)

dd5154 commented 4 years ago

Same issue, used workaround