boto / s3transfer

Amazon S3 Transfer Manager for Python
Apache License 2.0
209 stars 133 forks source link

ImportError: cannot import name 'ReadTimeoutError' #130

Closed snowman2 closed 5 years ago

snowman2 commented 5 years ago

Pins:

botocore>=1.12.89,<1.13.0
s3transfer>=0.2.0,<0.3.

Just started happening today:

from boto3.s3.transfer import create_transfer_manager
File "/var/runtime/boto3/s3/transfer.py", line 129, in <module>
from s3transfer.manager import TransferConfig as S3TransferConfig
File "/var/task/s3transfer/manager.py", line 21, in <module>
from s3transfer.utils import get_callbacks
File "/var/task/s3transfer/utils.py", line 27, in <module>
from botocore.exceptions import ReadTimeoutError
ImportError: cannot import name 'ReadTimeoutError'
snowman2 commented 5 years ago

This pin seems to work (probably a looser version exists that works as well):

boto3==1.9.87
botocore==1.12.83
jamesls commented 5 years ago

Can you share the specific version of botocore, s3transfer, and boto3 that were causing those issues for you? I tried a few options, but they're all working for me:

Using the version ranges you've provided:

$ pip install 'botocore>=1.12.89,<1.13.0'
$ pip install 's3transfer>=0.2.0,<0.3.'
$ python -c "from boto3.s3.transfer import create_transfer_manager"

With the oldest versions in the range:

$ pip install botocore==1.12.89 s3transfer==0.2.0 boto3
$ python -c "from boto3.s3.transfer import create_transfer_manager"

That ReadTimeoutError exception was added in botocore 1.11.0, so there shouldn't be an issue with a version range of botocore>=1.12.89,<1.13.0.

willtsui commented 5 years ago

It also fail with AWS Lambda python 2.7 botocore==1.10.74 boto3==1.7.74

Comparing utils.py with download.py and init.py, the later import ReadTimeoutError from botocore.vendored.requests.packages.urllib3.exceptions instead of botocore.exceptions.

snowman2 commented 5 years ago

Not sure - it didn't show up in the build logs. By all means it doesn't make sense with the pins. Must be an underlying pin and something pip is doing when resolving dependencies.

jcrawford commented 5 years ago

I am seeing this issue as well and I am running the following versions

boto3==1.7.70
botocore==1.10.82
s3transfer==0.2.0

This is the stack trace that is failing. This just recently started happening. I noticed above that it was mentined that this exception ReadTimeoutError was added in 1.11.0 but if that is the case why am I getting the errors using 1.10.82?

  File "/.../venv/lib/python3.6/site-packages/s3transfer/manager.py", line 21, in <module>
    from s3transfer.utils import get_callbacks
  File "/.../venv/lib/python3.6/site-packages/s3transfer/utils.py", line 27, in <module>
    from botocore.exceptions import ReadTimeoutError
ImportError: cannot import name 'ReadTimeoutError'

Updating my botocore to 1.11.0 has solved the issue for me, however I am not sure why I was required to upgrade locally yet on our qa and production instances it is still running 1.10.82 without any issues.