piskvorky / smart_open

Utils for streaming large files (S3, HDFS, gzip, bz2...)
MIT License
3.22k stars 383 forks source link

Do not touch botocore unless it is installed #803

Closed ddelange closed 8 months ago

ddelange commented 8 months ago

Title

Raise a proper S3 import exception (leftover from #785).

Motivation

Leftover from #785, not raising the proper Exception anymore:

  File "/usr/share/python3/app/lib/python3.11/site-packages/smart_open/s3.py", line 121, in <module>
    RETRY = Retry()
            ^^^^^^^
  File "/usr/share/python3/app/lib/python3.11/site-packages/smart_open/s3.py", line 86, in __init__
    self.exceptions: List[Exception] = [botocore.exceptions.EndpointConnectionError]
                                        ^^^^^^^^
NameError: name 'botocore' is not defined

Tests

Work in progress

Checklist

Before you create the PR, please make sure you have:

Workflow

ddelange commented 8 months ago

cc @mpenkov this error was raised when importing smart_open on a smart_open[azure] container.

ddelange commented 8 months ago

hmm, now we're getting

smart_open/transport.py:22: in <module>
    _REGISTRY = {NO_SCHEME: smart_open.local_file}
E   AttributeError: partially initialized module 'smart_open' has no attribute 'local_file' (most likely due to a circular import)

What would be the proper fix?

edit: 252430c

ddelange commented 8 months ago

is there a point in time where we can test import smart_open before any optional deps are installed? to catch this case?

edit: 947bbe0

mpenkov commented 8 months ago

Fixes #804

mpenkov commented 8 months ago

Thank you @ddelange !

ddelange commented 8 months ago

thanks too for the quick action! https://http.cat/202