search

deepjavalibrary / djl

An Engine-Agnostic Deep Learning Framework in Java
https://djl.ai
Apache License 2.0
4.13k stars 655 forks source link

Nuisance spurious debug error message when loading .onnx models #3461

Closed demq closed 1 month ago

demq commented 1 month ago

When loading a ONNX model, the getSessionOptions function in ai/djl/onnxruntime/engine/OrtModel.java attempts to invoke "customOpLibrary" even when it is not specified in options: https://github.com/deepjavalibrary/djl/blob/bf4ae898f0b9a725ccdc831cca1ad58258ad3c38/engines/onnxruntime/onnxruntime-engine/src/main/java/ai/djl/onnxruntime/engine/OrtModel.java#L200

This in turn calls getOrtxLibraryPath() https://github.com/deepjavalibrary/djl/blob/bf4ae898f0b9a725ccdc831cca1ad58258ad3c38/engines/onnxruntime/onnxruntime-engine/src/main/java/ai/djl/onnxruntime/engine/OrtModel.java#L237

This function assumes that the external packagemicrosoft.onnxruntime.extensions should be on path, and it should contain library files for "onnxruntime-extensions" for the given os-architecture (osx-aarch64 is not supported). Otherwise, the corresponding exceptions will result in a debug message in line https://github.com/deepjavalibrary/djl/blob/bf4ae898f0b9a725ccdc831cca1ad58258ad3c38/engines/onnxruntime/onnxruntime-engine/src/main/java/ai/djl/onnxruntime/engine/OrtModel.java#L244

This is a nuisance message that can trick developers to think that something is wrong with the model/code, while in reality nothing is wrong. Moreover, "onnxruntime-extensions" is not even part of the djl dependencies. Perhaps the message should only appear when an onnxruntime-extension is explicitly specified.

frankfliu commented 1 month ago
  1. I created PR to change the debug log message
  2. osx-aarch64 has to be fixed by onnx. Please create a ticket on onnxruntime repo if you need that
frankfliu commented 1 month ago

@demq

We try to automate the onnxruntime-extensions without user explicitly specify it. We assume that user added the library in the classpath which means they want to use it.

demq commented 1 month ago

@demq

We try to automate the onnxruntime-extensions without user explicitly specify it. We assume that user added the library in the classpath which means they want to use it.

Thanks @frankfliu - this solves the issue, especially that now a stack trace is not logged to debug. I will follow up with onnx on adding osx-aarch64 libraries.