liormizr / s3path

s3path is a pathlib extension for AWS S3 Service
Apache License 2.0
207 stars 39 forks source link

Version 0.3.0 issue with custom configuration with smart-open - open method #66

Closed liormizr closed 3 years ago

liormizr commented 3 years ago

Raised from @LeoQuote:

the original code won't work using custom s3 endpoint. Please consider adding my proposal code.

Sugestion:

        s3path_session = boto3.Session()
        s3path_client = s3path_session.client(
            's3',
            endpoint_url=resource_kwargs['endpoint_url'],
            aws_access_key_id=resource_kwargs['aws_access_key_id'],
            aws_secret_access_key=resource_kwargs['aws_secret_access_key'],
            aws_session_token=resource_kwargs['aws_session_token']
        )
        file_object = smart_open.open(
            uri=path.as_uri(),
            mode=mode,
            buffering=buffering,
            encoding=encoding,
            errors=errors,
            newline=newline,
            ignore_ext=True,
            transport_params={
                'session': s3path_session,
                'client': s3path_client,
                'resource_kwargs': resource_kwargs,
liormizr commented 3 years ago

@LeoQuote: Fixed it in PR: #65 Let me know if you think that there will still be an issue

liormizr commented 3 years ago

Fixed and deployed in version 0.3.01