tus / tusd

Reference server implementation in Go of tus: the open protocol for resumable file uploads
https://tus.github.io/tusd
MIT License
3.09k stars 480 forks source link

Mismatched offset caused by duplicate part name with old file use s3store. #1109

Closed OwynWang closed 5 months ago

OwynWang commented 7 months ago

Describe the bug Preondition: s3store & specificed FileInfoChanges.ID from filename in PreUploadCreateCallback. An old file named example.file.part, could be an object, could be the last failed upload remained. If it exists, client can't upload the file named example.file. Error message is ERR_MISMATCHED_OFFSET: mismatched offset.

To Reproduce Steps to reproduce the behavior: Method 1:

  1. Start a tusd server that use s3store & specificed FileInfoChanges.ID from filename in PreUploadCreateCallback.
  2. Upload a file named example.file.part.
  3. Upload a file named example.file Error.

Method 2:

  1. Start a tusd server that use s3store & specificed FileInfoChanges.ID in PreUploadCreateCallback & PreferredPartSize=10MB.
  2. Upload(chunk_size=11MB) an object named example.file(size>50MB), and inertrupt upload when part file appear.
  3. Upload a file named example.file Error.

Expected behavior Upload example.file Success.

Setup details Please provide following details, if applicable to your situation:

Acconut commented 5 months ago

First of all, apologies for my late response! My first thought here is that this is not directly what I consider a bug in tusd. When you are specifying a custom upload ID, its your task to ensure that this ID does not cause collisions with resources from other uploads. This can either achieved by deleting the resources belonging to the other uploads, or - what we usually recommend - by including a random part in the upload ID. If you really want to keep the original filename (which is already questionable), you can use IDs with the format {RANDOM UUID}/{FILENAME}.{EXTENSION}

That being said, I will see if we can improve the documentation for FileInfoChanges.ID to make these caveats clearer

Acconut commented 5 months ago

Added documentation in https://github.com/tus/tusd/commit/57d06cf2901de3bdde208d40cb388adaeece6196.

OwynWang commented 5 months ago

You are correct, and the original file name may contain many special characters, leading to numerous exceptions if used as an UploadID. I ultimately resolved this issue through the PreFinishResponseCallback.