search

Backblaze / b2-sdk-java

The official Java SDK for using Backblaze's B2 Storage APIs
Other
94 stars 25 forks source link

When is a file successfully uploaded? #120

Closed bollerdominik closed 4 years ago

bollerdominik commented 4 years ago

I want to upload a bunch of small and larger files to B2 and then delete my local copy once the file has been uploaded successfully.

Is it safe to delete the file after the call to uploadSmallFile() or uploadLargeFile()? I ask because I have noticed the B2 web dashboard still listing some of my files as "Unfinished Large Files" after the uploadXFile()call has returned.

certainmagic commented 4 years ago

Hi Dominik!

If one of those upload methods returns normally, without throwing an exception, and the file hasn't been uploaded successfully, that's a serious bug.

Let's learn more about the unfinished large files you're seeing. When an uploadLargeFile() call fails, are you using B2StorageClient.finishUploadingLargeFile() to resume the upload or one of the cancelLargeFile() methods to clean up the failed upload? If not, perhaps these unfinished large files correspond to failed previous upload attempts. Are there successfully uploaded versions of these files as well?

thanks, ab

ps. The existing uploadLargeFile() api is awkward because we don't return the large file's fileId that you'd need to easily call finishUploadingLargeFile() or cancelLargeFile(). You end up having to look for it yourself. :(

bollerdominik commented 4 years ago

Hi Thanks for your response. The upload method calls returned without any exception. I noticed the file is available on my bucket but the B2 web dashboard still shows it as "Unfinished Large Files" for some time. Maybe the web GUI cashes the list? My bucket is set to only keep the last version of the file.

Just to confirm your answer: Am I correct to assume that after successfully calling uploadSmallFile() or uploadLargeFile() the file has been uploaded completely and it is safe to delete the local copy?

certainmagic commented 4 years ago

Hi Dominik --

I work for a file storage company. I don't really believe in deleting files. ;)

But...on a more serious note... When B2 returns a 200 response with a file version from an upload, it is confirming that the file is reliably stored in B2. If the SDK is working properly, when an upload method returns without an exception, B2 has accepted your file.

I think it might be useful to figure out where your unfinished large files are coming from. That will let us know it's safe to clean them up help rule out a bug.

I don't think the web ui caches the "unfinished large files" for very long. You can call B2StorageClient.unfinishedLargeFiles() to see what unfinishedLargeFiles() you have. There's no caching for that. There's a similar operation in the b2 cli.

ttfn, ab

bollerdominik commented 4 years ago

Thanks for the answer.