Error when using latest version of pip

[yizhang-zen]

See https://github.com/poise/poise-python/blob/master/lib/poise_python/resources/python_package.rb#L41

I ran into error, since pip has new releases recently

STDERR: The python_package resource requires pip >= 6.1.0, currently 10.0.0b2

it should check something like ^[0-6]\\.0 so it does not match 10

[collinpeters]

This is breaking our build as well. Any way to work around this?

[yizhang-zen]

I added python_runtime block with a pip version in the our recipe, as a temp fix. It works for me

[yizhang-zen]

@coderanger Could you have a look please? :)

[collinpeters]

@yizhang-zen Could you provide an example? I'm not having any luck myself.

[yizhang-zen]

sorry..same here...add the block I mentioned before didn't work. :/

[chris-redekop]

I encountered this error and worked around it by adding the indicated pip_version line to my recipe.

case node['platform_family']
when 'amazon'
  package_name = 'python36'
else
  package_name = 'python3'
end

python_runtime '3' do
  provider :system

  # See https://github.com/poise/poise-python/issues/107
  pip_version '9.0.3'

  options :system, package_name: package_name
end

[barracksbuilder]

Any word on this getting fixed and merged into master. This breaks the AWS code deploy agent which takes down our entire environment. We use berks to manage dependencies and I haven't found a way to overwrite the pip version from an attributes wrapper.

[bcg62]

also running into this issue

STDERR: The python_package resource requires pip >= 6.1.0, currently 10.0.0

[coderanger]

Literally everyone is running into this issue, please stop with the me toos. It takes a few days to test with new major releases and whatnot, for now you’ll need to pin back to pip 9 to use this cookbook.

[barracksbuilder]

@coderanger, Is there a way to pin pip 9 through an attribute? I see override['poise-python']['options']['pip_version'] = false can I use my wrapper to pin from there?

Little more background we don't use poise-python directly we are using the aws-codedeploy-agent and poise-python is a dependency

[coderanger]

Yes, you can set any provider option via that system.

[et304383]

Serious question, what on earth was that regex trying to match?

[andy-v-h]

To get around this issue I had to specify a different pip_url like so, default['poise-python']['options']['get_pip_url'] = 'https://raw.githubusercontent.com/pypa/get-pip/master/2.6/get-pip.py'

[davidmontgom]

this is a major issue. I had a server die and I had to rector 10 cookbooks to install via bash. Is this cookbook no longer maintained?

[coderanger]

@davidmontgom I already responded to you once. Stop it.

[raghav8888]

@coderanger Needed a bit of guidance since I'm pretty new to chef. Went through the concept of wrapper cookbooks. From the comment, all I need to do is:

1. Include dependency in metadata.rb
2. In my cookbook's default.rb, include_recipe 'poise-python::default'
3. Include the line below line in my cookbook's attribute file default['poise-python']['options']['get_pip_url'] = 'https://raw.githubusercontent.com/pypa/get-pip/master/2.6/get-pip.py'

Could you confirm?

[coderanger]

@raghav8888 override['poise-python']['options']['pip_version'] = '9.0.3' will lock the version back to pip 9.

[raghav8888]

oh cool. Thanks. Will implement that till this gets fixed.

[eyalzek]

@coderanger thanks for the info, might I suggest to make the documentation for the pip_version attribute a bit more clear?

pip_version – Version of pip to install. If set to true, use the latest. If set to false, do not install pip. For backward compatibility, can also be set to a URL instead of get_pip_url. (default: true)

It makes sense that it takes a version string since it says: Version of pip to install, but I find the second sentence to be a bit misleading in that it makes me think that this attribute should be a boolean.

[coderanger]

@eyalzek It can be a boolean, true means "latest version" and false means "don't touch pip, I'll do it myself" :) It's definitely a bit obtuse as designs go, there's a whole bunch of config info compressed into a single value because it makes life a bit easier for the majority of cases. An example would probably be the way to go, barring writing more formal reference docs which I just don't have the bandwidth for. Though as a tip for the future, I'm usually pretty good about annotating my code with YARD markup which shows type info and sometimes a more verbose description:

https://github.com/poise/poise-python/blob/2870ae1744cd78620079984d88b49d4def3d18ad/lib/poise_python/resources/python_runtime.rb#L55-L63

[et304383]

I'm having a little trouble understanding the lack of sense of urgency here. Many (all) people are affected by this who updated to pip 10. Yet, it's been over two weeks and no fix is in place nor has the PR which could resolve this been merged.

I get that everyone is busy and what not, but @coderanger you've replied several times to this thread indicating you have at least some availability. I understand wanting to test and what not, but at this point what are you worried about? Making the cookbook more broken? This is hotfix territory. Push out a fix that hopefully works and worry about thoroughly testing and/or refining it later.

[barracksbuilder]

@et304383, There is urgency here but @coderanger is only one person. The version check isn't the only issue with PIP 10 they changed some of the API calls. You can pin PIP to 9.0.3 to get it running again. If you use a wrapper like I do because I'm really only using this cookbook for the aws-codedeploy-agent you can use the override attribute stated above. If you are actually using the cookbook each item has the ability to set the PIP version so pin it there.

[gsdevme]

@barracksbuilder @et304383 Open source software!

Im attempting to use the attribute work around on CentOS 6.9 though and running into

       Mixlib::ShellOut::ShellCommandFailed
       ------------------------------------
       python_runtime_pip[2] (/tmp/kitchen/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb line 118) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
       ---- Begin output of ["/usr/bin/python", "/tmp/get-pip20180417-84-oxao2a.py", "--upgrade", "--force-reinstall", "pip==9.0.3"] ----
       STDOUT: 
       STDERR: Traceback (most recent call last):
         File "/tmp/get-pip20180417-84-oxao2a.py", line 20636, in <module>
           main()
         File "/tmp/get-pip20180417-84-oxao2a.py", line 197, in main
           bootstrap(tmpdir=tmpdir)
         File "/tmp/get-pip20180417-84-oxao2a.py", line 82, in bootstrap
           import pip._internal
         File "/tmp/tmpUQb23G/pip.zip/pip/_internal/__init__.py", line 42, in <module>
         File "/tmp/tmpUQb23G/pip.zip/pip/_internal/cmdoptions.py", line 16, in <module>
         File "/tmp/tmpUQb23G/pip.zip/pip/_internal/index.py", line 526
           {str(c.version) for c in all_candidates},
                      ^
       SyntaxError: invalid syntax
       ---- End output of ["/usr/bin/python", "/tmp/get-pip20180417-84-oxao2a.py", "--upgrade", "--force-reinstall", "pip==9.0.3"] ----
       Ran ["/usr/bin/python", "/tmp/get-pip20180417-84-oxao2a.py", "--upgrade", "--force-reinstall", "pip==9.0.3"] returned 1

       Cookbook Trace:
       ---------------
       /tmp/kitchen/cookbooks/poise/files/halite_gem/poise/utils/shell_out.rb:86:in `tap'
       /tmp/kitchen/cookbooks/poise/files/halite_gem/poise/utils/shell_out.rb:86:in `poise_shell_out!'
       /tmp/kitchen/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_runtime_pip.rb:131:in `block in bootstrap_pip'
       /tmp/kitchen/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_runtime_pip.rb:107:in `bootstrap_pip'
       /tmp/kitchen/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_runtime_pip.rb:84:in `action_install'
       /tmp/kitchen/cookbooks/poise/files/halite_gem/poise/helpers/notifying_block.rb:69:in `notifying_block'
       /tmp/kitchen/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb:50:in `action_install'

[coderanger]

@gsdevme That's another pip 10 issue, they dropped support for Python 2.6 which is what the distro python package there is using. Solution is default['poise-python']['options']['get_pip_url'] = 'https://bootstrap.pypa.io/2.6/get-pip.py' as well as also pinning to 9.0.3.

[paustin01]

For others here I came across this issue via the cloudcli chef cookbook which depends on this cookbook (both at latest versions). Adding the following to the attributes file of my wrapper cookbook did the trick.

override['poise-python']['options']['pip_version'] = '9.0.3'

[jeffbyrnes]

In case anyone is using the get-pip.py from pypa.io, here’s how I dealt with that:

remote_file file_cache_path('get-pip.py') do
  source 'https://bootstrap.pypa.io/get-pip.py'
  notifies :run, "python_execute[#{file_cache_path('get-pip.py')} 'pip<10']", :immediately
end

python_execute "#{file_cache_path('get-pip.py')} 'pip<10'" do
  action :nothing
end

[scalp42]

@jeffbyrnes I'm confused in what order do you run this?

I also tried to set pip_version to false in the python_runtime resource but no dice.

python_execute requires a runtime, so not sure how you managed to run it? Or maybe you have a system python package installed before?

Debugging:

python_runtime '2' do
  options(
    pip_version: false,
    setuptools: false
  )
end

remote_file %|#{Chef::Config[:file_cache_path]}/get-pip.py| do
  source 'https://bootstrap.pypa.io/get-pip.py'
  notifies :run, "python_execute[#{Chef::Config[:file_cache_path]}/get-pip.py 'pip<10']", :immediately
end

python_execute "#{Chef::Config[:file_cache_path]}/get-pip.py 'pip<10'" do
  action :nothing
  python '2'
end

No dice:

INFO: Processing python_package[setuptools] action install (/var/tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb line 138)

           ================================================================================
           Error executing action `install` on resource 'python_package[setuptools]'
           ================================================================================

           Mixlib::ShellOut::ShellCommandFailed
           ------------------------------------
           Expected process to exit with [0], but received '1'
           ---- Begin output of ["/usr/bin/python2.7", "-", "setuptools"] ----
           STDOUT:
           STDERR: The python_package resource requires pip >= 6.1.0, currently 10.0.0
           ---- End output of ["/usr/bin/python2.7", "-", "setuptools"] ----
           Ran ["/usr/bin/python2.7", "-", "setuptools"] returned 1

[coderanger]

Poise-python 1.7.0 is up with full support for both Pip 10 and Chef 14.