search

aws-amplify / aws-sdk-android

AWS SDK for Android. For more information, see our web site:
https://docs.amplify.aws
Other
1.03k stars 549 forks source link

couldn't reset the stream to retry while invoking a pdf upload using transfer utility #136

Closed ghost closed 8 years ago

ghost commented 8 years ago

Issue is occuring when I am trying to upload a pdf file located in external storage .Weird thing is file gets deleted from the mobile directory automatically when this execution fails or success. I am using Xiaomi,Android api 4.4.4 API level 19

These are the logs where the exception occurs. com.amazonaws.AmazonClientException: Encountered an exception and couldn't reset the stream to retry at com.amazonaws.http.AmazonHttpClient.resetRequestAfterError(AmazonHttpClient.java:499) at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:439) at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:199) at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4221) at com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:1628) at com.amazonaws.mobileconnectors.s3.transferutility.UploadTask.uploadSinglePartAndWaitForCompletion(UploadTask.java:205) at com.amazonaws.mobileconnectors.s3.transferutility.UploadTask.call(UploadTask.java:86) at com.amazonaws.mobileconnectors.s3.transferutility.UploadTask.call(UploadTask.java:45)

fosterzhang commented 8 years ago

TransferUtility doesn't automatically delete files. Do you see similar behavior on other devices? Would you please provide code snippet to reproduce this issue?

ghost commented 8 years ago

I have not tried on other devices. But on this device I am getting this issue most of the time. This is the snippet code I am using

Filepath list contain pdf files.

    public void beginUpload(String[] filePath, String[] fileName) {
        for(int i = 0; i < filePath.length;i++) {
            File file = new File(filePath[i]);
            Log.e(TAG, "File Path: " + filePath[i] + "Files name: " + fileName[i]);
            TransferObserver observer = mTransferUtility.upload(mCredentialsVo.bucket, fileName[i] ,
                    file);
            observer.setTransferListener(new UploadListener(observer.getId()));
            mObservers.add(observer);
        }
    }
ghost commented 8 years ago

Some of the snippet of the filePath values /storage/emulated/0/Android/data/com.google.android.apps.docs/cache/projector/Algorithm-Ch6-Heapsort.pdf When visited again,not able to find the files.

fosterzhang commented 8 years ago

@iit2009060 Your code seems fine. Is your device running low on storage? The pdf files are in cache directory and might be purged once the system runs out of space. Does your code have logic to delete these files?

ghost commented 8 years ago

No we dont have any logic to delete the files. It automatically gets deleted once we run transfer utility. I will check on the storage thing.But why after running the transfer utility (fail or success) files gets deleted,before that it was on the mobile storage. One more thing, why this error occur "couldn't reset the stream to retry" ,it happens intermittently but most of the time.

ghost commented 8 years ago

Currently we are using this as to calculate freeMemory() public float freeMemory() { StatFs statFs = new StatFs(Environment.getExternalStorageDirectory().getAbsolutePath()); long free = (statFs.getAvailableBlocks() * statFs.getBlockSize()); return free / (1024.f * 1024.f); } But this function practically giving me space is less around 10 mb ,but when I check on the mobile it shows 26.29 GB available ,so I currently I am not using this function to check for memory,and keep storing pdf files,and then run the transfer utility. my filePath[] array also contains audio files ,which are not gets deleted and they are get successfully uploaded.Problem occurs when I tried to upload pdf files.

fosterzhang commented 8 years ago

@iit2009060 I suggest you try the same code on different device.

ghost commented 8 years ago

These are the logs while uploading a pdf and an audio file in an different device(Spice spice MI498H Android 6.0.1 API Level 23).It works for two times,but after crashes.

05-18 11:48:56.831 31868-31868/com.arpit E/AWSUploadUtility: File Path: /storage/9016-4EF8/37a619aa803d282a-732507370795810584-1-user.pdf Files name: 570534365574979738/user/732507370795810584/37a619aa803d282a/Draft_732507370795810584_37a619aa803d282a_1_20160518114856user.pdf 05-18 11:48:56.923 31868-31868/com.arpit E/AWSUploadUtility: File Path: /storage/9016-4EF8/37a619aa803d282a-732507370795810584-1-audioRecorder.mp3 Files name: 570534365574979738/user/732507370795810584/37a619aa803d282a/Draft_732507370795810584_37a619aa803d282a_1_20160518_114856_complete.mp3

05-18 11:48:57.729 31868-30716/com.arpit E/UploadTask: Failed to upload: 1 due to Encountered an exception and couldn't reset the stream to retry 05-18 11:48:57.734 31868-31868/com.arpit E/AWSUploadUtility: Error during upload: 1 com.amazonaws.AmazonClientException: Encountered an exception and couldn't reset the stream to retry at com.amazonaws.http.AmazonHttpClient.resetRequestAfterError(AmazonHttpClient.java:499) at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:439) at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:199) at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4221) at com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:1628) at com.amazonaws.mobileconnectors.s3.transferutility.UploadTask.uploadSinglePartAndWaitForCompletion(UploadTask.java:205) at com.amazonaws.mobileconnectors.s3.transferutility.UploadTask.call(UploadTask.java:86) at com.amazonaws.mobileconnectors.s3.transferutility.UploadTask.call(UploadTask.java:45) at java.util.concurrent.FutureTask.run(FutureTask.java:237) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1113) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:588) at java.lang.Thread.run(Thread.java:818) Caused by: java.io.IOException: read failed: ENOENT (No such file or directory) at libcore.io.IoBridge.read(IoBridge.java:478) at java.io.FileInputStream.read(FileInputStream.java:177) at com.amazonaws.services.s3.internal.RepeatableFileInputStream.read(RepeatableFileInputStream.java:145) at com.amazonaws.internal.SdkFilterInputStream.read(SdkFilterInputStream.java:73) at com.amazonaws.util.LengthCheckInputStream.read(LengthCheckInputStream.java:95) at com.amazonaws.internal.SdkFilterInputStream.read(SdkFilterInputStream.java:73) at com.amazonaws.event.ProgressReportingInputStream.read(ProgressReportingInputStream.java:115) at com.amazonaws.auth.AwsChunkedEncodingInputStream.setUpNextChunk(AwsChunkedEncodingInputStream.java:290) at com.amazonaws.auth.AwsChunkedEncodingInputStream.read(AwsChunkedEncodingInputStream.java:152) at java.io.InputStream.read(InputStream.java:162) at com.amazonaws.http.UrlHttpClient.write(UrlHttpClient.java:171) at com.amazonaws.http.UrlHttpClient.writeContentToConnection(UrlHttpClient.java:129) at com.amazonaws.http.UrlHttpClient.execute(UrlHttpClient.java:65) at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:360) at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:199)  at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4221)  at com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:1628)  at com.amazonaws.mobileconnectors.s3.transferutility.UploadTask.uploadSinglePartAndWaitForCompletion(UploadTask.java:205)  at com.amazonaws.mobileconnectors.s3.transferutility.UploadTask.call(UploadTask.java:86)  at com.amazonaws.mobileconnectors.s3.transferutility.UploadTask.call(UploadTask.java:45)  at java.util.concurrent.FutureTask.run(FutureTask.java:237)  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1113)  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:588)  at java.lang.Thread.run(Thread.java:818)  Caused by: android.system.ErrnoException: read failed: ENOENT (No such file or directory) at libcore.io.Posix.readBytes(Native Method) at libcore.io.Posix.read(Posix.java:169) at libcore.io.BlockGuardOs.read(BlockGuardOs.java:230) at libcore.io.IoBridge.read(IoBridge.java:468) at java.io.FileInputStream.read(FileInputStream.java:177)  at com.amazonaws.services.s3.internal.RepeatableFileInputStream.read(RepeatableFileInputStream.java:145)  at com.amazonaws.internal.SdkFilterInputStream.read(SdkFilterInputStream.java:73)  at com.amazonaws.util.LengthCheckInputStream.read(LengthCheckInputStream.java:95)  at com.amazonaws.internal.SdkFilterInputStream.read(SdkFilterInputStream.java:73)  at com.amazonaws.event.ProgressReportingInputStream.read(ProgressReportingInputStream.java:115)  at com.amazonaws.auth.AwsChunkedEncodingInputStream.setUpNextChunk(AwsChunkedEncodingInputStream.java:290)  at com.amazonaws.auth.AwsChunkedEncodingInputStream.read(AwsChunkedEncodingInputStream.java:152)  at java.io.InputStream.read(InputStream.java:162)  at com.amazonaws.http.UrlHttpClient.write(UrlHttpClient.java:171)  at com.amazonaws.http.UrlHttpClient.writeContentToConnection(UrlHttpClient.java:129)  at com.amazonaws.http.UrlHttpClient.execute(UrlHttpClient.java:65)  at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:360)  at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:199)  at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4221)  at com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:1628)  at com.amazonaws.mobileconnectors.s3.transferutility.UploadTask.uploadSinglePartAndWaitForCompletion(UploadTask.java:205)  at com.amazonaws.mobileconnectors.s3.transferutility.UploadTask.call(UploadTask.java:86)  at com.amazonaws.mobileconnectors.s3.transferutility.UploadTask.call(UploadTask.java:45)  at java.util.concurrent.FutureTask.run(FutureTask.java:237)  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1113)  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:588)  at java.lang.Thread.run(Thread.java:818) 

fosterzhang commented 8 years ago

@iit2009060 The log says

Caused by: java.io.IOException: read failed: ENOENT (No such file or directory)

This indicates the file is removed while the upload is in progress. I recommend you check your code and your hardware. Also try this sample https://github.com/awslabs/aws-sdk-android-samples/tree/master/S3TransferUtilitySample.

ghost commented 8 years ago

This is the sample logs when Istarted using aws-sdk-android S3Transfer Utility sample. Can you helpme fixing this issue ,then I will try to reproduce this issue on the sample. 05-18 16:36:49.312 24702-24702/com.amazonaws.demo.s3transferutility D/TransferSerivce: Starting Transfer Service 05-18 16:36:49.336 24702-24702/com.amazonaws.demo.s3transferutility D/TransferSerivce: Network connected: true 05-18 16:36:49.337 24702-18176/com.amazonaws.demo.s3transferutility D/TransferSerivce: Loading transfers from database 05-18 16:36:49.339 24702-18176/com.amazonaws.demo.s3transferutility D/TransferSerivce: 0 transfers are loaded from database 05-18 16:37:30.289 24702-24715/com.amazonaws.demo.s3transferutility W/CursorWrapperInner: Cursor finalized without prior close() 05-18 16:37:34.806 20722-20722/com.amazonaws.demo.s3transferutility W/System: ClassLoader referenced unknown path: /data/app/com.amazonaws.demo.s3transferutility-1/lib/arm 05-18 16:37:40.833 20722-20722/com.amazonaws.demo.s3transferutility D/TransferSerivce: Starting Transfer Service 05-18 16:37:40.907 20722-20722/com.amazonaws.demo.s3transferutility D/TransferSerivce: Network connected: true 05-18 16:37:40.909 20722-21136/com.amazonaws.demo.s3transferutility D/TransferSerivce: Loading transfers from database 05-18 16:37:40.911 20722-21136/com.amazonaws.demo.s3transferutility D/TransferSerivce: 0 transfers are loaded from database 05-18 16:37:40.911 20722-20722/com.amazonaws.demo.s3transferutility D/UploadActivity: onStateChanged: 8, IN_PROGRESS 05-18 16:37:40.912 20722-20722/com.amazonaws.demo.s3transferutility D/UploadActivity: onProgressChanged: 8, total: 334234, current: 0 05-18 16:37:41.883 20722-20722/com.amazonaws.demo.s3transferutility D/UploadActivity: onProgressChanged: 8, total: 334234, current: 139264 05-18 16:37:42.945 20722-20722/com.amazonaws.demo.s3transferutility D/UploadActivity: onProgressChanged: 8, total: 334234, current: 334234 05-18 16:37:44.903 20722-20722/com.amazonaws.demo.s3transferutility D/UploadActivity: onProgressChanged: 8, total: 334234, current: 668468 05-18 16:37:46.847 20722-20722/com.amazonaws.demo.s3transferutility D/UploadActivity: onProgressChanged: 8, total: 334234, current: 1002702 05-18 16:37:47.527 20722-20734/com.amazonaws.demo.s3transferutility W/CursorWrapperInner: Cursor finalized without prior close() 05-18 16:37:48.075 20722-20722/com.amazonaws.demo.s3transferutility D/UploadActivity: onProgressChanged: 8, total: 334234, current: 1336936 05-18 16:37:48.076 20722-21137/com.amazonaws.demo.s3transferutility E/UploadTask: Failed to upload: 8 due to Unable to execute HTTP request: Write error: ssl=0xb8119b90: I/O error during system call, Connection reset by peer 05-18 16:37:48.116 20722-20722/com.amazonaws.demo.s3transferutility E/UploadActivity: Error during upload: 8 com.amazonaws.AmazonClientException: Unable to execute HTTP request: Write error: ssl=0xb8119b90: I/O error during system call, Connection reset by peer at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:428) at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:199) at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4221) at com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:1628) at com.amazonaws.mobileconnectors.s3.transferutility.UploadTask.uploadSinglePartAndWaitForCompletion(UploadTask.java:205) at com.amazonaws.mobileconnectors.s3.transferutility.UploadTask.call(UploadTask.java:86) at com.amazonaws.mobileconnectors.s3.transferutility.UploadTask.call(UploadTask.java:45) at java.util.concurrent.FutureTask.run(FutureTask.java:237) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1113) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:588) at java.lang.Thread.run(Thread.java:818) Caused by: javax.net.ssl.SSLException: Write error: ssl=0xb8119b90: I/O error during system call, Connection reset by peer at com.android.org.conscrypt.NativeCrypto.SSL_write(Native Method) at com.android.org.conscrypt.OpenSSLSocketImpl$SSLOutputStream.write(OpenSSLSocketImpl.java:771) at com.android.okhttp.okio.Okio$1.write(Okio.java:76) at com.android.okhttp.okio.AsyncTimeout$1.write(AsyncTimeout.java:155) at com.android.okhttp.okio.RealBufferedSink.flush(RealBufferedSink.java:221) at com.android.okhttp.internal.http.HttpConnection$FixedLengthSink.flush(HttpConnection.java:308) at com.android.okhttp.okio.RealBufferedSink.flush(RealBufferedSink.java:223) at com.android.okhttp.okio.RealBufferedSink$1.flush(RealBufferedSink.java:204) at com.amazonaws.http.UrlHttpClient.writeContentToConnection(UrlHttpClient.java:130) at com.amazonaws.http.UrlHttpClient.execute(UrlHttpClient.java:65)

ghost commented 8 years ago

Got it fixed.Will try to reproduce this issue on the sample.

ghost commented 8 years ago

I am not able to reproduce the issue on the sample.But on the transfer utility when i change the key to be file.getName() , it gets uploaded successfully ,but file gets deleted from the local storage in mobile.

ghost commented 8 years ago

And it is happening for uploading the pdf files.

ghost commented 8 years ago

Founded the root cause.Its happening due to mupdf library. Please close the issue