ImportError: cannot import name 'DEPRECATED_SERVICE_NAMES' from 'botocore.docs'

[metametadata]

awsume suddenly started failing on CI server after the Docker build container was rebuilt. This is unexpected as I have most of the dependencies in the container pinned to specific versions.

Steps

1) Install using pip3 install awsume==4.5.3. 2) Run awsume.

Expected

Success.

Actual

Warning: the awsume shell script is not being sourced, please use awsume-configure to install the alias
Traceback (most recent call last):
  File "/usr/local/bin/awsumepy", line 6, in <module>
    from awsume.awsumepy.main import main
  File "/usr/local/lib/python3.7/dist-packages/awsume/awsumepy/__init__.py", line 1, in <module>
    from . import app
  File "/usr/local/lib/python3.7/dist-packages/awsume/awsumepy/app.py", line 9, in <module>
    import boto3
  File "/usr/local/lib/python3.7/dist-packages/boto3/__init__.py", line 17, in <module>
    from boto3.session import Session
  File "/usr/local/lib/python3.7/dist-packages/boto3/session.py", line 25, in <module>
    from .resources.factory import ResourceFactory
  File "/usr/local/lib/python3.7/dist-packages/boto3/resources/factory.py", line 17, in <module>
    from ..docs import docstring
  File "/usr/local/lib/python3.7/dist-packages/boto3/docs/__init__.py", line 15, in <module>
    from botocore.docs import DEPRECATED_SERVICE_NAMES
ImportError: cannot import name 'DEPRECATED_SERVICE_NAMES' from 'botocore.docs' (/usr/local/lib/python3.7/dist-packages/botocore/docs/__init__.py)
/usr/local/bin/awsume: line 183: return: can only `return' from a function or sourced script

Cause

Supposedly it's caused by the bug in the latest boto3 version: https://github.com/boto/boto3/issues/3648.

Suggested Fix

Pin the version of boto3 dependency to the stable one.

And I guess it's a good idea to pin all the other dependencies too.

Workaround

The same as in the referenced upstream issue: downgrade boto3 to the good version via pip3 install 'boto3==1.26.101'.

[mtskillman]

@metametadata apologies for the long delay here, but, respectfully, this issue simply stems from installing awsume via pip. we "officially recommend" using pipx for reasons like this. eg ➝ necessary to isolate the python environment of awsume from other things being installed.

basically these issues are due to having other stuff installed "globally" that happens to cause install of particular versions of deps that causing issues for awsume. in pipx, this would be a non-issue as each env is isolated

And I guess it's a good idea to pin all the other dependencies too.

this is a great idea. I'll bring this up internally

[metametadata]

this issue simply stems from installing awsume via pip

I don't remember exactly, but I could have reproduced it in the isolated Docker container.

Not reproducible anymore, maybe due to boto updates.

we "officially recommend" using pipx for reasons like this. eg ➝ necessary to isolate the python environment of awsume from other things being installed.

Thanks, I didn't know pipx provides isolated envs, will switch to it.

this is a great idea. I'll bring this up internally

Yes, I suppose even with pipx there's a risk to run into fresh upstream issues from the unpinned dependencies.

FWIW, installing pipx was a headache, this is what I ended up with in my Dockerfile:

# https://hub.docker.com/_/debian
FROM --platform=linux/amd64 debian:buster-20210816

ENV LANG C.UTF-8

# pip
RUN set -eux \
  ; apt-get update \
  ; apt-get install -y python3-pip \
  ; rm -rf /var/lib/apt/lists/*

# python3-venv, needed for pipx to install packages
RUN set -eux \
  ; apt-get update \
  ; apt-get install -y python3-venv \
  ; rm -rf /var/lib/apt/lists/*

# pipx, per https://github.com/pypa/pipx/pull/972 due to issue https://github.com/pypa/pipx/issues/982
RUN set -eux \
  ; apt-get update \
  ; apt-get install -y pipx \
  ; pipx ensurepath \
  ; rm -rf /var/lib/apt/lists/*

# Per instructions printed by pipx ensurepath
ENV PATH=/root/.local/bin:$PATH

# AWSume, https://awsu.me/general/quickstart.html#installation
RUN set -eux \
  ; pipx install awsume==4.5.3

[mtskillman]

no bug identified -- closing issue