search

bradleyg / django-s3direct

Directly upload files to S3 compatible services with Django.
MIT License
653 stars 234 forks source link

Duplication problem #218

Closed lucasvazq closed 3 years ago

lucasvazq commented 3 years ago

Hello, good day, I am having a duplication problem with my files in S3. I do not know if this is the place to look for a solution, but I wanted to leave my doubt here in case someone has an answer to give me. I am using this library to upload videos to S3. Many times it happens that I upload a video for the first time, and when I try to upload it again elsewhere, instead of re-uploading and duplicating, it returns the URL of the original video. This seems fine to me, it prevents the same video from being duplicated.

The problem I am having is that this does not work consistently for me and I do not know why. For example, yesterday I was uploading a video many times. The first time I uploaded it, it returned a URL that ended in the following way: b597c6e8174b576a01.mp4 All other times it returned this first URL to me (the video was not duplicated in S3). However, today I am trying to upload the same video again, hoping it will return the same URL that it returned yesterday and I get a new one (the video is now duplicated in S3).

bradleyg commented 3 years ago

Hi there,

Using the setting allow_existence_optimization in your config file should emit the behaviour you mention (does not re-upload the file).

django-s3direct uses [EvaporateJS](https://github.com/TTLabs/EvaporateJS/wiki/Evaporate.create()) under the hood, and I wonder if the setting s3FileCacheHoursAgo is affecting something here (by default it's set to 12 hours, which sounds similar to your issue) - reading the EvaporateJS docs it shouldn't do, but might be worth experimenting with...

I'll have a dig into it when I get chance.

lucasvazq commented 3 years ago

Hello, thank you for the quick reply. We have found the problem and it was due to an error in the implementation that we were making of this library. It was not related to any configuration, it was our javascript problem. If any other type of problem arises, I will take your words into consideration.