certainmagic
commented
4 years ago Hi Marko --
I'm sorry you're having trouble. :(
Occasionally, we have internal errors that prevent you from uploading. We try to track them down and fix them.
When you get an error that is preventing you from uploading a large file, the SDK has already done all of the retrying it can do in a reasonable period of time. If you call B2StorageClient.finishUploadingLargeFile() the SDK will try to finish uploading only the parts it hadn't uploaded yet.
Here are some things you should consider when using large files in general:
- If you don't "cancel" or delete unfinished large files, they will continue to exist. We provide the unfinishedLargeFiles() iterator to help you find them and cancelLargeFile() for you to delete any you want to get rid of.
- It probably goes without saying, but please don't change the large files between when you start uploading them and when you finish uploading them. If you do, the results aren't well-defined.
does this help?
thanks, ab
markoglasgow
Hi guys,
When we upload large amounts of files to B2 using the Java SDK, we periodically get the following error:
First example:
Exception in thread "main" <B2Exception 500 internal_error: incident id 0c6058816065-iYokSPwuQYGT0tClXcbcOA==> at com.backblaze.b2.client.exceptions.B2Exception.create(B2Exception.java:83) at com.backblaze.b2.client.webApiHttpClient.B2WebApiHttpClientImpl.extractExceptionFromErrorResponse(B2WebApiHttpClientImpl.java:266) at com.backblaze.b2.client.webApiHttpClient.B2WebApiHttpClientImpl.postAndReturnString(B2WebApiHttpClientImpl.java:220) at com.backblaze.b2.client.webApiHttpClient.B2WebApiHttpClientImpl.postDataReturnJson(B2WebApiHttpClientImpl.java:87) at com.backblaze.b2.client.B2StorageClientWebifierImpl.uploadPart(B2StorageClientWebifierImpl.java:304) at com.backblaze.b2.client.B2LargeFileUploader.lambda$uploadOnePart$4(B2LargeFileUploader.java:294) at com.backblaze.b2.client.B2Retryer.doRetry(B2Retryer.java:85) at com.backblaze.b2.client.B2LargeFileUploader.uploadOnePart(B2LargeFileUploader.java:262) at com.backblaze.b2.client.B2LargeFileUploader.lambda$uploadPartsAndFinish$1(B2LargeFileUploader.java:209) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
Second example:
Exception in thread "main" <B2Exception 500 internal_error: incident id 0c6058816065-Oa7JCWAaRfilmex+EMgnoA==> at com.backblaze.b2.client.exceptions.B2Exception.create(B2Exception.java:83) at com.backblaze.b2.client.webApiHttpClient.B2WebApiHttpClientImpl.extractExceptionFromErrorResponse(B2WebApiHttpClientImpl.java:266) at com.backblaze.b2.client.webApiHttpClient.B2WebApiHttpClientImpl.postAndReturnString(B2WebApiHttpClientImpl.java:220) at com.backblaze.b2.client.webApiHttpClient.B2WebApiHttpClientImpl.postDataReturnJson(B2WebApiHttpClientImpl.java:87) at com.backblaze.b2.client.B2StorageClientWebifierImpl.uploadPart(B2StorageClientWebifierImpl.java:304) at com.backblaze.b2.client.B2LargeFileUploader.lambda$uploadOnePart$4(B2LargeFileUploader.java:294) at com.backblaze.b2.client.B2Retryer.doRetry(B2Retryer.java:85) at com.backblaze.b2.client.B2LargeFileUploader.uploadOnePart(B2LargeFileUploader.java:262) at com.backblaze.b2.client.B2LargeFileUploader.lambda$uploadPartsAndFinish$1(B2LargeFileUploader.java:209) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
It uploads thousands of files for hours correctly before we get this error.
What causes the error? How should we handle it? Should we restart the upload of the affected file?
I posted to B2 Zendesk support about this, but they told me to make a Github issue here.
-Marko