Kinesis saveRecord failing with null pointer exception

[meavydev]

Describe the bug Rather than the ACE exception, it looks like the internal code is failing with a null pointer: 05-24 16:33:50.234 12079 7514 E AndroidRuntime: java.lang.NullPointerException 05-24 16:33:50.234 12079 7514 E AndroidRuntime: at java.io.FileOutputStream.<init>(FileOutputStream.java:227) 05-24 16:33:50.234 12079 7514 E AndroidRuntime: at com.amazonaws.mobileconnectors.kinesis.kinesisrecorder.FileManager.newOutputStream(FileManager.java:115) 05-24 16:33:50.234 12079 7514 E AndroidRuntime: at com.amazonaws.mobileconnectors.kinesis.kinesisrecorder.FileRecordStore.tryInitializeWriter(FileRecordStore.java:121) 05-24 16:33:50.234 12079 7514 E AndroidRuntime: at com.amazonaws.mobileconnectors.kinesis.kinesisrecorder.FileRecordStore.put(FileRecordStore.java:80) 05-24 16:33:50.234 12079 7514 E AndroidRuntime: at com.amazonaws.mobileconnectors.kinesis.kinesisrecorder.AbstractKinesisRecorder.saveRecord(AbstractKinesisRecorder.java:100) 05-24 16:33:50.234 12079 7514 E AndroidRuntime: at com.amazonaws.mobileconnectors.kinesis.kinesisrecorder.KinesisRecorder.saveRecord(KinesisRecorder.java:240) To Reproduce It is difficult to say what is triggering it, but it is failing after running for a few hours.

Which AWS service(s) are affected? Kinesis

Expected behavior No crash or an ACE exception.

Environment Information (please complete the following information):

• AWS Android SDK Version: com.amazonaws:aws-android-sdk-kinesis:2.18.0 other AWS SDKs are 2.19.0

• Device: Pixel 3A

• Android Version: 11

• Specific to simulators: No

[meavydev]

Also seeing a related submitAllRecords error, but this is returning the AmazonClientException : com.amazonaws.AmazonClientException: Cannot find records file at com.amazonaws.mobileconnectors.kinesis.kinesisrecorder.FileRecordStore$RecordIterator.hasNext(FileRecordStore.java:272) at com.amazonaws.mobileconnectors.kinesis.kinesisrecorder.AbstractKinesisRecorder.submitAllRecords(AbstractKinesisRecorder.java:128) at com.navenio.sdk.aws.kinesis.KinesisStream$sendKinesisIbeaconStream$1.invokeSuspend(KinesisStream.kt:264) at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106) at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:571) at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750) at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:678) at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665) Caused by: java.io.FileNotFoundException: /data/user/0/com.navenio.ils/cache/KinesisRecorder/kinesis_stream_records: open failed: ENOENT (No such file or directory) at libcore.io.IoBridge.open(IoBridge.java:492) at java.io.FileInputStream.<init>(FileInputStream.java:160) at com.amazonaws.mobileconnectors.kinesis.kinesisrecorder.FileManager.newInputStream(FileManager.java:103) at com.amazonaws.mobileconnectors.kinesis.kinesisrecorder.FileRecordStore$RecordIterator.tryOpenReader(FileRecordStore.java:216) at com.amazonaws.mobileconnectors.kinesis.kinesisrecorder.FileRecordStore$RecordIterator.hasNext(FileRecordStore.java:244) at com.amazonaws.mobileconnectors.kinesis.kinesisrecorder.AbstractKinesisRecorder.submitAllRecords(AbstractKinesisRecorder.java:128)

[meavydev]

Having noticed an update AWS SDK of 2.26.0, we still see exceptions: com.amazonaws.AmazonClientException: Failed to remove read records at com.amazonaws.mobileconnectors.kinesis.kinesisrecorder.AbstractKinesisRecorder.submitAllRecords(AbstractKinesisRecorder.java:206) at com.navenio.sdk.aws.kinesis.KinesisStream$sendKinesisWifiStream$1.invokeSuspend(KinesisStream.kt:203) at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106) at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:571) at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750) at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:678) at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665) Caused by: java.io.IOException: Failed to delete read records and persist unread records at com.amazonaws.mobileconnectors.kinesis.kinesisrecorder.FileRecordStore.deleteReadRecords(FileRecordStore.java:184) at com.amazonaws.mobileconnectors.kinesis.kinesisrecorder.FileRecordStore.access$300(FileRecordStore.java:43) at com.amazonaws.mobileconnectors.kinesis.kinesisrecorder.FileRecordStore$RecordIterator.removeReadRecords(FileRecordStore.java:339) at com.amazonaws.mobileconnectors.kinesis.kinesisrecorder.AbstractKinesisRecorder.submitAllRecords(AbstractKinesisRecorder.java:167) at com.navenio.sdk.aws.kinesis.KinesisStream$sendKinesisWifiStream$1.invokeSuspend(KinesisStream.kt:203)  at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)  at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)  at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:571)  at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750)  at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:678)  at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665)  2021-07-06 11:49:41.679 21356-21847/com.navenio.ils E/KinesisStream$sendKinesisWifiStream: [nvsdk] Kinesis WiFi save failed com.amazonaws.AmazonClientException: Cannot find records file at com.amazonaws.mobileconnectors.kinesis.kinesisrecorder.FileRecordStore$RecordIterator.hasNext(FileRecordStore.java:272) at com.amazonaws.mobileconnectors.kinesis.kinesisrecorder.AbstractKinesisRecorder.submitAllRecords(AbstractKinesisRecorder.java:128) at com.navenio.sdk.aws.kinesis.KinesisStream$sendKinesisWifiStream$1.invokeSuspend(KinesisStream.kt:203) at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106) at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:571) at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750) at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:678) at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665) Caused by: java.io.FileNotFoundException: /data/user/0/com.navenio.ils/cache/KinesisRecorder/kinesis_stream_records: open failed: ENOENT (No such file or directory) at libcore.io.IoBridge.open(IoBridge.java:497) at java.io.FileInputStream.<init>(FileInputStream.java:160) at com.amazonaws.mobileconnectors.kinesis.kinesisrecorder.FileManager.newInputStream(FileManager.java:103) at com.amazonaws.mobileconnectors.kinesis.kinesisrecorder.FileRecordStore$RecordIterator.tryOpenReader(FileRecordStore.java:216) at com.amazonaws.mobileconnectors.kinesis.kinesisrecorder.FileRecordStore$RecordIterator.hasNext(FileRecordStore.java:244) at com.amazonaws.mobileconnectors.kinesis.kinesisrecorder.AbstractKinesisRecorder.submitAllRecords(AbstractKinesisRecorder.java:128)  at com.navenio.sdk.aws.kinesis.KinesisStream$sendKinesisWifiStream$1.invokeSuspend(KinesisStream.kt:203)  at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)  at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)  at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:571)  at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750)  at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:678)  at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665)  Caused by: android.system.ErrnoException: open failed: ENOENT (No such file or directory) at libcore.io.Linux.open(Native Method) at libcore.io.ForwardingOs.open(ForwardingOs.java:239) at libcore.io.BlockGuardOs.open(BlockGuardOs.java:273) at libcore.io.ForwardingOs.open(ForwardingOs.java:239) at android.app.ActivityThread$AndroidOs.open(ActivityThread.java:7682) at libcore.io.IoBridge.open(IoBridge.java:483) at java.io.FileInputStream.<init>(FileInputStream.java:160)  at com.amazonaws.mobileconnectors.kinesis.kinesisrecorder.FileManager.newInputStream(FileManager.java:103)  at com.amazonaws.mobileconnectors.kinesis.kinesisrecorder.FileRecordStore$RecordIterator.tryOpenReader(FileRecordStore.java:216)  at com.amazonaws.mobileconnectors.kinesis.kinesisrecorder.FileRecordStore$RecordIterator.hasNext(FileRecordStore.java:244)  at com.amazonaws.mobileconnectors.kinesis.kinesisrecorder.AbstractKinesisRecorder.submitAllRecords(AbstractKinesisRecorder.java:128)  at com.navenio.sdk.aws.kinesis.KinesisStream$sendKinesisWifiStream$1.invokeSuspend(KinesisStream.kt:203)  at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)  at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)  at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:571)  at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750)  at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:678)  at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665)  2021-07-06 11:49:42.558 21356-22087/com.navenio.ils E/KinesisStream$sendKinesisBleStream: [nvsdk] Kinesis BLE save failed java.lang.NullPointerException at java.io.FileOutputStream.<init>(FileOutputStream.java:227) at com.amazonaws.mobileconnectors.kinesis.kinesisrecorder.FileManager.newOutputStream(FileManager.java:115) at com.amazonaws.mobileconnectors.kinesis.kinesisrecorder.FileRecordStore.tryInitializeWriter(FileRecordStore.java:121) at com.amazonaws.mobileconnectors.kinesis.kinesisrecorder.FileRecordStore.put(FileRecordStore.java:80) at com.amazonaws.mobileconnectors.kinesis.kinesisrecorder.AbstractKinesisRecorder.saveRecord(AbstractKinesisRecorder.java:100) at com.amazonaws.mobileconnectors.kinesis.kinesisrecorder.KinesisRecorder.saveRecord(KinesisRecorder.java:240) at com.navenio.sdk.aws.kinesis.KinesisStream$sendKinesisBleStream$1.invokeSuspend(KinesisStream.kt:244) at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106) at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:571) at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750) at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:678) at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665) 2021-07-06 11:49:45.059 21356-22052/com.navenio.ils E/KinesisStream$sendKinesisWifiStream: [nvsdk] Kinesis WiFi save failed com.amazonaws.AmazonClientException: Cannot find records file at com.amazonaws.mobileconnectors.kinesis.kinesisrecorder.FileRecordStore$RecordIterator.hasNext(FileRecordStore.java:272) at com.amazonaws.mobileconnectors.kinesis.kinesisrecorder.AbstractKinesisRecorder.submitAllRecords(AbstractKinesisRecorder.java:128) at com.navenio.sdk.aws.kinesis.KinesisStream$sendKinesisWifiStream$1.invokeSuspend(KinesisStream.kt:203) at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106) at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:571) at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750) at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:678) at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665) Caused by: java.io.FileNotFoundException: /data/user/0/com.navenio.ils/cache/KinesisRecorder/kinesis_stream_records: open failed: ENOENT (No such file or directory) at libcore.io.IoBridge.open(IoBridge.java:497) at java.io.FileInputStream.<init>(FileInputStream.java:160) at com.amazonaws.mobileconnectors.kinesis.kinesisrecorder.FileManager.newInputStream(FileManager.java:103) at com.amazonaws.mobileconnectors.kinesis.kinesisrecorder.FileRecordStore$RecordIterator.tryOpenReader(FileRecordStore.java:216) at com.amazonaws.mobileconnectors.kinesis.kinesisrecorder.FileRecordStore$RecordIterator.hasNext(FileRecordStore.java:244) at com.amazonaws.mobileconnectors.kinesis.kinesisrecorder.AbstractKinesisRecorder.submitAllRecords(AbstractKinesisRecorder.java:128)  at com.navenio.sdk.aws.kinesis.KinesisStream$sendKinesisWifiStream$1.invokeSuspend(KinesisStream.kt:203)  at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)  at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)  at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:571)  at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750)  at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:678)  at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665)  Caused by: android.system.ErrnoException: open failed: ENOENT (No such file or directory) at libcore.io.Linux.open(Native Method) at libcore.io.ForwardingOs.open(ForwardingOs.java:239) at libcore.io.BlockGuardOs.open(BlockGuardOs.java:273) at libcore.io.ForwardingOs.open(ForwardingOs.java:239) at android.app.ActivityThread$AndroidOs.open(ActivityThread.java:7682) at libcore.io.IoBridge.open(IoBridge.java:483) at java.io.FileInputStream.<init>(FileInputStream.java:160)  at com.amazonaws.mobileconnectors.kinesis.kinesisrecorder.FileManager.newInputStream(FileManager.java:103)  at com.amazonaws.mobileconnectors.kinesis.kinesisrecorder.FileRecordStore$RecordIterator.tryOpenReader(FileRecordStore.java:216)  at com.amazonaws.mobileconnectors.kinesis.kinesisrecorder.FileRecordStore$RecordIterator.hasNext(FileRecordStore.java:244)  at com.amazonaws.mobileconnectors.kinesis.kinesisrecorder.AbstractKinesisRecorder.submitAllRecords(AbstractKinesisRecorder.java:128)  at com.navenio.sdk.aws.kinesis.KinesisStream$sendKinesisWifiStream$1.invokeSuspend(KinesisStream.kt:203)  at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)  at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)  at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:571)  at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750)  at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:678)  at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665) 

[div5yesh]

@hassanctech Please take a look at this.

[meavydev]

We are still seeing this with the AWS SDK v2.73.0 08-25 12:53:31.113 10886 26594 E LoggingKt: [nvsdk] Kinesis submitAll failed 08-25 12:53:31.113 10886 26594 E LoggingKt: com.amazonaws.AmazonClientException: Failed to remove read records 08-25 12:53:31.113 10886 26594 E LoggingKt: at com.navenio.sdk.aws.kinesis.KinesisStream$scheduleSubmitKinesisRecords$1.invokeSuspend(KinesisStream.kt:809) 08-25 12:53:31.113 10886 26594 E LoggingKt: at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) 08-25 12:53:31.113 10886 26594 E LoggingKt: at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:108) 08-25 12:53:31.113 10886 26594 E LoggingKt: at kotlinx.coroutines.internal.LimitedDispatcher$Worker.run(LimitedDispatcher.kt:115) 08-25 12:53:31.113 10886 26594 E LoggingKt: at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:103) 08-25 12:53:31.113 10886 26594 E LoggingKt: at com.amazonaws.mobileconnectors.kinesis.kinesisrecorder.AbstractKinesisRecorder.submitAllRecords(AbstractKinesisRecorder.java:206) 08-25 12:53:31.113 10886 26594 E LoggingKt: at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:584) 08-25 12:53:31.113 10886 26594 E LoggingKt: at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:793) 08-25 12:53:31.113 10886 26594 E LoggingKt: at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:697) 08-25 12:53:31.113 10886 26594 E LoggingKt: at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:684) 08-25 12:53:31.113 10886 26594 E LoggingKt: Caused by: java.io.IOException: Failed to delete read records and persist unread records 08-25 12:53:31.113 10886 26594 E LoggingKt: at com.amazonaws.mobileconnectors.kinesis.kinesisrecorder.FileRecordStore.deleteReadRecords(FileRecordStore.java:184) 08-25 12:53:31.113 10886 26594 E LoggingKt: at com.amazonaws.mobileconnectors.kinesis.kinesisrecorder.FileRecordStore.access$300(FileRecordStore.java:43) 08-25 12:53:31.113 10886 26594 E LoggingKt: at com.amazonaws.mobileconnectors.kinesis.kinesisrecorder.FileRecordStore$RecordIterator.removeReadRecords(FileRecordStore.java:339) 08-25 12:53:31.113 10886 26594 E LoggingKt: at com.amazonaws.mobileconnectors.kinesis.kinesisrecorder.AbstractKinesisRecorder.submitAllRecords(AbstractKinesisRecorder.java:167) 08-25 12:53:31.113 10886 26594 E LoggingKt: ... 9 more