shotgunsoftware / python-api

A Python-based library for accessing Flow Production Tracking API.
https://developer.shotgridsoftware.com/python-api
Other
308 stars 198 forks source link

Uploading files that share the same base filename to S3 can overwrite each other #294

Open briansmith-epic opened 1 year ago

briansmith-epic commented 1 year ago

When performing an upload via the SG API, we have multiple files that share a common filename but are in completely different paths. If we try to upload both of those simultaneously, the AWS upload URL is the same for both.

In _upload_to_storage(), os.path.basename(path) is passed to the _get_attachment_upload_info call. On the AWS side, that means that no matter the parent directory, any file that shares the base name will get a duplicate upload URL.

This only occurs if the _upload_to_storage() call happens concurrently : I'm guessing that the SG backend uses a timestamp to differentiate calls, because currently we do this about 30 times, but only when multiple get called at the same time do we get a duplicate upload URL.

The culprit line is here.

Recommend changing: filename = os.path.basename(path) to something like: filename = os.path.abspath(path)