Setuptools version Python mismatch on Shippable

[joshuaohana]

I can't tell exactly where my problem is arising, but as of about a week ago I've been unable to deploy my Node app via Shippable.

I was running an older version of Node when the issue first arose. Since then I've tried to upgrade my Node version to 10.21.0 and the same issues present. I've also tried to mess with different version of things in my shippable file to no avail

The error I get it

Downloading/unpacking setuptools>=20.0 (from awsebcli==3.7.8) Running setup.py (path:/tmp/pip_build_root/setuptools/setup.py) egg_info for package setuptools pkg_resources/py2_warn.py:15: UserWarning: Setuptools no longer works on Python 2

I tried several things in my shippable file, pasted in full below, commenting out the bits that did't help. I also tried to target the version for setuptools but that gave a different error

# python:
#   - 2.7

node_js:
  - 10.21.0   

install:
  - shippable_retry npm install
  - npm install -g gulp
  # - pip uninstall -y setuptools
  # - pip install setuptools=42.0.2
  # - pip install --force-reinstall awsebcli==3.14.5
  # - pip install --force-reinstall botocore==1.11.9

I'm perfectly content sticking with python2 but don't know how to tell Shippable to use the older version of setuptools. Or I'm perfectly content upgrading to python3 but don't know how to tell shippable to do that.

I tried to follow the instructions on https://setuptools.readthedocs.io/en/latest/python%202%20sunset.html but they didn't seem to cooperate with shippable

[a-murphy]

If you can use Node 10.15.1, I would suggest specifying python as the language in your shippable.yml and a Python 3 version. The v7.2.4 Python image has Python 3.7.2 and Node 10.15.1 pre-installed. If you are relying on the default script section, you will need to add a script section at the same level as install with npm test to avoid running the Python default instead.

If Node 10.15.1 won't work for you, adding --no-dependencies to the --force-reinstall installs should prevent reinstalling dependencies and updating them to newer versions, however you will likely run into problems with other dependencies and will have to determine which versions of the dependencies you need to install.

[joshuaohana]

thanks @a-murphy but that didn't seem to resolve, I changed to 10.15.1 but I'm getting the same error that

Downloading/unpacking setuptools>=20.0 (from awsebcli==3.7.8) Running setup.py (path:/tmp/pip_build_root/setuptools/setup.py) egg_info for package setuptools pkg_resources/py2_warn.py:15: UserWarning: Setuptools no longer works on Python 2

[a-murphy]

My suggestion with Node 10.15.1 was to specify the Python version (changing the language to python in the YML and replacing the node_js version section with python) and use the default version of NodeJS installed in the latest Python images, which happens to be 10.15.1.

[joshuaohana]

If I only specify python version (3.7.2 for example) then shippable shows Node version as node_js 0.12 and then everything fails completely big surprise. If I specify both python and node versions then we're back to the same error :(

[a-murphy]

What build image name is in the pull_job_image section of the logs? I wasn't able to find your project on Shippable. If it's shippable/minv2, that would explain problems, and I'd suggest adding build_image: drydock/u16pytall:v7.2.4 to your YML to use the latest Python image. If it's newer, you should be able to update the machine image in your node pool to v7.2.4 to use v7.2.4 images by default.

[joshuaohana]

@a-murphy pull_job_image says shippable_retry sudo docker pull drydock/u14nodall:prod

[a-murphy]

That's not quite as old as shippable/minv2, but has never been a default image for Python. Would you mind including either the full shippable.yml file or the ID from the bottom of the project page?

[joshuaohana]

sure @a-murphy thanks for the help, project ID is 59b3161a8a02980600066e8f, you'll see the failures from jobs 600-621

[a-murphy]

In 619, it looks like it was still finding language: node_js instead of language: python, which is why it was trying to use a NodeJS image. Perhaps the indentation somewhere in the shippable.yml file lead to the parser missing something. And in 621, it looks like the language was still node_js, but with the python image configured in the YML. However, the error in 621 had to do with the environment to which it was deploying, not the build environment, and it looks like you've resolved that.

Changing the machine image in your subscription node pool to v7.2.4 would save the time spent pulling the build image, but could also change the image versions used for your other projects. So, I would only suggest updating that if you don't mind updating the images used in your other projects.