Failure to render image: RecyclableBufferedInputStream$InvalidMarkException

[greyson-signal]

Hi there! Over at Signal a user reported an issue around being unable to render a specific image. Sure enough the problem seems to be completely reproducible with that image.

I made a POC app to verify it wasn't anything Signal was doing: https://github.com/greyson-signal/glide-failure-poc

The POC shows Glide loading a working image and also failing to load the image from the issue.

Thanks!

Glide Version: 4.12.0

Integration libraries: None

Device/Android Version: Android 10 emulator, my Android 11 Pixel 5. In general seems to be a universal issue.

Issue details / Repro steps / Use case background: Image fails to load, but renders in a web browser and can otherwise be opened in desktop image programs.

Glide load line / GlideModule (if any) / list Adapter code (if any): See POC

Layout XML: See POC

Stack trace / LogCat:

2021-03-08 10:39:48.113 18969-18969/com.greysonparrelli.glideimagefailure W/Glide: Load failed for [B@1ea6a87 with size [1080x825]
    class com.bumptech.glide.load.engine.GlideException: Failed to load resource
    There were 6 root causes:
    com.bumptech.glide.load.resource.bitmap.RecyclableBufferedInputStream$InvalidMarkException(Mark has been invalidated, pos: 288768 markLimit: 5242880)
    java.lang.RuntimeException(setDataSourceCallback failed: status = 0x80000000)
    com.bumptech.glide.load.resource.bitmap.RecyclableBufferedInputStream$InvalidMarkException(Mark has been invalidated, pos: 288768 markLimit: 5242880)
    java.lang.RuntimeException(setDataSourceCallback failed: status = 0x80000000)
    com.bumptech.glide.load.resource.bitmap.RecyclableBufferedInputStream$InvalidMarkException(Mark has been invalidated, pos: 288768 markLimit: 5242880)
    com.bumptech.glide.load.resource.bitmap.RecyclableBufferedInputStream$InvalidMarkException(Mark has been invalidated, pos: 2460412 markLimit: 5242880)
     call GlideException#logRootCauses(String) for more detail
      Cause (1 of 2): class com.bumptech.glide.load.engine.GlideException: Failed LoadPath{HeapByteBuffer->Object->Drawable}, LOCAL
    There were 4 root causes:
    com.bumptech.glide.load.resource.bitmap.RecyclableBufferedInputStream$InvalidMarkException(Mark has been invalidated, pos: 288768 markLimit: 5242880)
    java.lang.RuntimeException(setDataSourceCallback failed: status = 0x80000000)
    com.bumptech.glide.load.resource.bitmap.RecyclableBufferedInputStream$InvalidMarkException(Mark has been invalidated, pos: 288768 markLimit: 5242880)
    java.lang.RuntimeException(setDataSourceCallback failed: status = 0x80000000)
     call GlideException#logRootCauses(String) for more detail
        Cause (1 of 3): class com.bumptech.glide.load.engine.GlideException: Failed DecodePath{HeapByteBuffer->GifDrawable->Drawable}
        Cause (2 of 3): class com.bumptech.glide.load.engine.GlideException: Failed DecodePath{HeapByteBuffer->Bitmap->Drawable}
    There were 2 root causes:
    com.bumptech.glide.load.resource.bitmap.RecyclableBufferedInputStream$InvalidMarkException(Mark has been invalidated, pos: 288768 markLimit: 5242880)
    java.lang.RuntimeException(setDataSourceCallback failed: status = 0x80000000)
     call GlideException#logRootCauses(String) for more detail
          Cause (1 of 2): class com.bumptech.glide.load.resource.bitmap.RecyclableBufferedInputStream$InvalidMarkException: Mark has been invalidated, pos: 288768 markLimit: 5242880
          Cause (2 of 2): class java.lang.RuntimeException: setDataSourceCallback failed: status = 0x80000000
        Cause (3 of 3): class com.bumptech.glide.load.engine.GlideException: Failed DecodePath{HeapByteBuffer->BitmapDrawable->Drawable}
    There were 2 root causes:
    com.bumptech.glide.load.resource.bitmap.RecyclableBufferedInputStream$InvalidMarkException(Mark has been invalidated, pos: 288768 markLimit: 5242880)
    java.lang.RuntimeException(setDataSourceCallback failed: status = 0x80000000)
     call GlideException#logRootCauses(String) for more detail
          Cause (1 of 2): class com.bumptech.glide.load.resource.bitmap.RecyclableBufferedInputStream$InvalidMarkException: Mark has been invalidated, pos: 288768 markLimit: 5242880
          Cause (2 of 2): class java.lang.RuntimeException: setDataSourceCallback failed: status = 0x80000000
      Cause (2 of 2): class com.bumptech.glide.load.engine.GlideException: Failed LoadPath{ByteArrayInputStream->Object->Drawable}, LOCAL
    There were 2 root causes:
    com.bumptech.glide.load.resource.bitmap.RecyclableBufferedInputStream$InvalidMarkException(Mark has been invalidated, pos: 288768 markLimit: 5242880)
    com.bumptech.glide.load.resource.bitmap.RecyclableBufferedInputStream$InvalidMarkException(Mark has been invalidated, pos: 2460412 markLimit: 5242880)
     call GlideException#logRootCauses(String) for more detail
        Cause (1 of 3): class com.bumptech.glide.load.engine.GlideException: Failed DecodePath{ByteArrayInputStream->GifDrawable->Drawable}
        Cause (2 of 3): class com.bumptech.glide.load.engine.GlideException: Failed DecodePath{ByteArrayInputStream->Bitmap->Drawable}
    There was 1 root cause:
2021-03-08 10:39:48.113 18969-18969/com.greysonparrelli.glideimagefailure W/Glide: com.bumptech.glide.load.resource.bitmap.RecyclableBufferedInputStream$InvalidMarkException(Mark has been invalidated, pos: 288768 markLimit: 5242880)
     call GlideException#logRootCauses(String) for more detail
          Cause (1 of 1): class com.bumptech.glide.load.resource.bitmap.RecyclableBufferedInputStream$InvalidMarkException: Mark has been invalidated, pos: 288768 markLimit: 5242880
        Cause (3 of 3): class com.bumptech.glide.load.engine.GlideException: Failed DecodePath{ByteArrayInputStream->BitmapDrawable->Drawable}
    There was 1 root cause:
    com.bumptech.glide.load.resource.bitmap.RecyclableBufferedInputStream$InvalidMarkException(Mark has been invalidated, pos: 2460412 markLimit: 5242880)
     call GlideException#logRootCauses(String) for more detail
          Cause (1 of 1): class com.bumptech.glide.load.resource.bitmap.RecyclableBufferedInputStream$InvalidMarkException: Mark has been invalidated, pos: 2460412 markLimit: 5242880

[Tolriq]

This confirms my tests on reported issues that was not fully able to reproduce:

Seems side effect of https://github.com/bumptech/glide/commit/2b61482baa90175e0efdf4ec90f06a5051e1b370#diff-bef6c4ca8f43cb29b8856a5a0adf3e70729ce4e8eb885f2de42031e58dce7683 that is a 4.12 change.

And fix worse than issue.

Ping @sjudd

[greyson-signal]

I do not believe this bug is specific to 4.12. Signal is using 4.11 in production, which also has this issue. I just used 4.12 in the sample project to verify that the bug wasn't fixed in that version :)

[Tolriq]

Arf thanks for info :( I have users that report missing images since I updated but none of them do contact me with logs :( I guess I'll keep praying one of those report do read Play Store answer :(

[sjudd]

@greyson-signal thanks for the detailed report.

@Tolriq if you do get more specific reports and especially any sample images where the issue reproduces, please let me know.

Assuming I'm understanding this correct, this is not caused by https://github.com/bumptech/glide/commit/2b61482baa90175e0efdf4ec90f06a5051e1b370#diff-bef6c4ca8f43cb29b8856a5a0adf3e70729ce4e8eb885f2de42031e58dce7683, but that fix was related to the same category of memory/buffer issues.

Here's the confluence of issues that causes bugs like this one:

1. Glide sets a fixed buffer size to use when decoding InputStreams, currently 5mb: https://github.com/bumptech/glide/commit/2b61482baa90175e0efdf4ec90f06a5051e1b370#diff-bef6c4ca8f43cb29b8856a5a0adf3e70729ce4e8eb885f2de42031e58dce7683
2. If either BitmapFactory or ImageHeaderParser read an InputStream past the 5mb limit, then the load will fail because we no longer have the entire image in memory to decode. At best we'd decode a partial image, which may or may not work at all and a partial image is certainly not what the user wants.
3. There isn't a trivial way to re-open a generic InputStream and for some specific cases (ie HttpUrlConnection) doing so could be very expensive.
4. Byte arrays and ByteBuffers get shoved into the InputStream path in Downsampler, even though they're entirely in memory already and the buffer size shouldn't be relevant.

#1 is necessary. We don't want to read arbitrarily large files into memory if BitmapFactory and/or ExifInterface decide to read the whole thing. #2 is also necessary, we don't want to allow partially decoded images. #3 is not necessary. Even if it's not totally trivial to re-open the stream, it would be better to incur the extra expense than fail the image load. There's room for improvement here. Unfortunately it's hard right now to handle this generically given the way Glide is architected for data types, like InputStream, that cannot trivially be re-opened. We do actually re-open FileDescriptors already, so I expect this bug would not occur when decoding Files, Uris, or remote images that are present in Glide's disk cache. #4 is not necessary either. The ImageReader interface used in Downsampler already could pretty easily be re-used for byte[] and ByteBuffer, which would skip the whole buffering issue.

I believe the specific issue in this bug could be fixed by #4. I don't encourage buffering entire files into memory, but if you're going to do so anyway (or use a ByteBuffer), then there's no reason to add an extra buffer and an artificial limitation.

While investigating this, I also realized that while we support simple FileDescriptors, we do not support AssetFileDescriptors because AssetFileDescriptors have start and len values, not just a FileDescriptor. Downsampler would need to take those values into account when decoding. BitmapFactory isn't much help here because it only supports raw FileDescriptor, so we'd be back to trying to wrap the AssetFileDescriptor into an InputStream so that we read the correct place in the assets file. I need to split this out into a separate bug.

TL;DR:

1. The simplest fix is to avoid using buffers for byte[] and ByteBuffer (not totally trivial to implement but not hard)
2. The ideal fix would be to generically allow for re-opening of data, not just rewinding. This is not trivial to implement without substantial changes to Glide that will probably break API compatibility in some way
3. There's an unrelated issue where we pretend that we can decode AssetFileDescriptors, but can't.

[sarbyn]

Any update about it? We got the same issue and the only way to load images is revert to glide 4.10.0 :( It seems that happens only with ExifInterface 1.3.3 (from another library), with ExifInterface 1.3.2 and glide 4.12.0 all works as expected

[krisdb]

I'm getting the same issue with 4.13.0:

Load failed for https://lexfridman.com/wordpress/wp-content/uploads/powerpress/artwork_3000-230.png with size [300x300]
class com.bumptech.glide.load.engine.GlideException: Failed to load resource
There was 1 root cause:
com.bumptech.glide.load.resource.bitmap.RecyclableBufferedInputStream$InvalidMarkException(Mark has been invalidated, pos: 2570697 markLimit: 5242880)
 call GlideException#logRootCauses(String) for more detail
  Cause (1 of 1): class com.bumptech.glide.load.engine.GlideException: Failed LoadPath{ContentLengthInputStream->Bitmap->Bitmap}, REMOTE
There was 1 root cause:
com.bumptech.glide.load.resource.bitmap.RecyclableBufferedInputStream$InvalidMarkException(Mark has been invalidated, pos: 2570697 markLimit: 5242880)
 call GlideException#logRootCauses(String) for more detail
    Cause (1 of 1): class com.bumptech.glide.load.engine.GlideException: Failed DecodePath{ContentLengthInputStream->Bitmap->Bitmap}
There was 1 root cause:
com.bumptech.glide.load.resource.bitmap.RecyclableBufferedInputStream$InvalidMarkException(Mark has been invalidated, pos: 2570697 markLimit: 5242880)
 call GlideException#logRootCauses(String) for more detail
      Cause (1 of 1): class com.bumptech.glide.load.resource.bitmap.RecyclableBufferedInputStream$InvalidMarkException: Mark has been invalidated, pos: 2570697 markLimit: 5242880

Reverting to 4.11.0 fixed the issue.