tus / tusd

Reference server implementation in Go of tus: the open protocol for resumable file uploads
https://tus.github.io/tusd
MIT License
3.07k stars 477 forks source link

How to delete files from s3? #122

Closed MurtadhaS closed 7 years ago

MurtadhaS commented 7 years ago

I want to ask about the way to delete files from s3 ?

Acconut commented 7 years ago

The S3 storage supports the Termination extension (http://tus.io/protocols/resumable-upload.html#termination) which will remove all associated objects for an upload in your S3 bucket. To use it, you send a HTTP DELETE request to the upload URL.

MurtadhaS commented 7 years ago

I do that but I get : [tusd] event="ResponseOutgoing" status="404" method="DELETE" path="964b7ba2df1b3af780b388f0b0a3008f+2~0qStk6gMAaV-XhEhd-gmQVVYVOJGuFu" error="NoSuchKey: which I think because I use a custom endpoint (my storage is only s3 compatible).

MurtadhaS commented 7 years ago

I fixed the key and now I get : [tusd] event="ResponseOutgoing" status="404" method="DELETE" path="91342343321ba6f795432aa11914ee49" error="upload not found"

Acconut commented 7 years ago

It looks like both of your upload URLs are not correct. The upload IDs for the S3 Store are relatively long which results in URLs, such as: https://localhost:1080/files/be9a4af34da64debff5baaa6f3723371+wk4JyusTIDtFKAAELYtQNM3po0iqlB_HDLxmh5DDvpRtKh2QveYBm6Enn4k498E17x8XDjkmW4dZL71z8o29zQ--

which I think because I use a custom endpoint (my storage is only s3 compatible).

That should not be a problem. tusd can work with any S3-compatible backend.

MurtadhaS commented 7 years ago

I am putting the name of the file in the storage, is that the same as "upload ID" ?

MurtadhaS commented 7 years ago

After some debugging I am sure now that the problem is in the ListParts function that is used by the GetInfo function , all of those are contianed in ("s3store.go") file.

Acconut commented 7 years ago

I am putting the name of the file in the storage, is that the same as "upload ID" ?

Frankly, I don't understand what you mean. Which storage are you referring to? The upload ID is the last part of the upload URL. For example, if the upload URL is https://localhost:1080/file/foobar, the upload ID is foobar.

MurtadhaS commented 7 years ago

Hi, I am still having the problem , that I can't download or delete the file after it is fully downloaded. Notice that I am using the demo tus-js client.

MurtadhaS commented 7 years ago

I found the problem, in the s3store.go file, in GetInfo function the code: if isAwsError(err, "NoSuchUpload") { info.Offset = info.Size return info, nil } else { return info, err } is wrong because when the file is completed it returns NoSuchKey, So the right thing is : if isAwsError(err, "NoSuchKey") { info.Offset = info.Size return info, nil } else { return info, err }

https://github.com/tus/tusd/pull/124

Acconut commented 7 years ago

I am not able to reproduce your issues. Would you be so kind and provide a detailed explanation how one may reproduce your situation? After trying various scenarios using tus-js-client, everything works as intended.

MurtadhaS commented 7 years ago

It may be a problem with my storage producing the wrong error code. Any way thank you bro.

Acconut commented 7 years ago

That could be the case. Which storage service are you using?

MurtadhaS commented 7 years ago

ceph