search

onnx / onnx-mlir

Representation and Reference Lowering of ONNX Models in MLIR Compiler Infrastructure
Apache License 2.0
749 stars 316 forks source link

Query regarding debugging #2866

Closed Abhishek-TyRnT closed 2 months ago

Abhishek-TyRnT commented 3 months ago

I was passing a LLM through the compiler. But I get the following error onnx-mlir: /llvm_build/llvm-project/llvm/include/llvm/ADT/SmallVector.h:304: reference llvm::SmallVectorTemplateCommon<onnx_mlir::IndexExpr>::operator[](size_type) [T = onnx_mlir::IndexExpr]: Assertion `idx < size()' failed. Aborted

This error is thrown when I try to emit MLIR , but works fine when I emit ONNXIR.

Is there a command or option that I can use to debug problems like these?

AlexandreEichenberger commented 3 months ago

When emitting ONNX IR, there is only onnx to onnx optimizations. When generating MLIR, we lower the ONNX operations to loops/arithmetic ops. One good first step would be to know which pass this crashes in. You can run the onnx-mlir with these options (be prepared for a very long output log): -mlir-print-ir-after-all --mlir-elide-elementsattrs-if-larger=20 --mlir-elide-resource-strings-if-larger=20

This will list the IR after all passes, and cut down the very large constants from the log. Next would be to detect which operation is causing the error. Probably need to use lldb and if you are lucky it may tell you the op. Once you know the pass, there are also pass specific debug flags that can be turned on.