DecodeImage: Failed to decode image when using direct ByteBuffer (Android)

[owainc]

As per title, I'm following the object_detection Android sample but I've found that if I use a direct buffer for the input tensor I get the following exception:

Error in /Users/runner/work/1/s/operators/vision/decode_image.cc line 27: [DecodeImage] Invalid input. Failed to decode image.
ORTObjectDetection      ai....ntime.example.objectdetection  E  Exception caught when perform ObjectDetection
   ai.onnxruntime.OrtException: Error code - ORT_RUNTIME_EXCEPTION - message: Encountered unknown exception in Run()

My aim is to use a direct buffer to minimise copying and have the JPEG input generated dynamically, however the issue can be demonstrated with a contrived example by modifying the sample linked above:

val directBuffer = ByteBuffer.allocateDirect(rawImageBytes.size).order(nativeOrder())
rawImageBytes.copyInto(directBuffer.array(), 0, 0, rawImageBytes.size)

val inputTensor = OnnxTensor.createTensor(
            ortEnv,
            directBuffer,
            shape,
            OnnxJavaType.UINT8
        )

Note that using direct buffers seems to be fine for other operations (they're used in the classification example for instance).

I've uploaded a branch of the examples that demonstrates the issue here: https://github.com/owainc/onnxruntime-inference-examples/tree/direct-buffer-issue

I'm relatively new to Android/Kotlin so at a loss to what the issue could be or how to solve it, any input would be greatly appreciated! Thanks!

[Craigacp]

directBuffer.array() is a partial method, so it doesn't work on direct byte buffers. I'm not sure how that code doesn't crash before it gets into ORT for that reason. You should do directBuffer.put(rawImageBytes); directBuffer.rewind().

[owainc]

Well that's annoying - hasArray() was returning true so thought I was good to go. Looking into it a bit more now I can see that .array() returns the directBuffer content prefixed by 4 zero bytes and with 3 trailing zero bytes, which explains the ORT exception. Suppose I'll chalk it up to an ART oddity and move on.

Thanks @Craigacp for solving the mystery 🙏.