Create a separate boto client for generating unsigned URLs - Githubissues

NYPL-Simplified / server_core

Shared data model and utilities for Library Simplified server applications

7 stars 11 forks source link

Create a separate boto client for generating unsigned URLs #1183

Closed vbessonov closed 4 years ago

vbessonov commented 4 years ago

Description

This PR changes S3Uploader initialisation logic, it creates two boto3 clients:

the first one without authentication used for generating unsigned URLs
the second one with authentication for working with S3: uploading files, etc.

Motivation and Context

This PR fixes the authentication issue in S3Uploader caused by PR 1179. Steps to reproduce the issue:

Create a MARC or ONIX collection
Try to import it using bin/directory_import with valid S3 credentials In the output you'll see an authentication error.

How Has This Been Tested?

I tested it by importing an ONIX collection into S3 with bin/directory_import script using the steps described above.

Checklist:

[ ] I have updated the documentation accordingly.
[x] All new and existing tests passed.

leonardr commented 4 years ago

There's one test failure that seems pretty minor. Looks good otherwise.

leonardr commented 4 years ago

Great, it makes sense now.