search

llvm / llvm-project

The LLVM Project is a collection of modular and reusable compiler and toolchain technologies.
http://llvm.org
Other
29.24k stars 12.07k forks source link

[mlir] -one-shot-bufferize crashes #109650

Open axeabc opened 2 months ago

axeabc commented 2 months ago

git version: c3d3cef8d5837

system: Ubuntu 18.04.6 LTS

reproduce with: mlir-opt -one-shot-bufferize a.mlir

a.mlir:

module {
  func.func @test_reshape_3d_same_d2d_auto_empty(%arg0: tensor<3x?x?xi32>) -> tensor<?x?x?xi32> attributes {llvm.emit_c_interface} {
    %collapsed = tensor.collapse_shape %arg0 [[0, 1, 2]] : tensor<3x?x?xi32> into tensor<?xi32>
    %0 = llvm.mlir.constant(0 : index) : i64
    %1 = builtin.unrealized_conversion_cast %0 : i64 to index
    %dim = tensor.dim %collapsed, %1 : tensor<?xi32>
    %2 = builtin.unrealized_conversion_cast %dim : index to i64
    %3 = llvm.mlir.constant(9 : index) : i64
    %4 = llvm.udiv %2, %3  : i64
    %5 = builtin.unrealized_conversion_cast %4 : i64 to index
    %expanded = tensor.expand_shape %collapsed [[0, 1, 2]] output_shape [%5, 3, %dim] : tensor<?xi32> into tensor<?x3x?xi32>
    %cast = tensor.cast %expanded : tensor<?x3x?xi32> to tensor<?x?x?xi32>
    return %cast : tensor<?x?x?xi32>
  }
}

stack trace:

mlir-opt: /data/szy/MLIR/llvm-release/llvm-project/mlir/lib/Dialect/MemRef/IR/MemRefOps.cpp:2298: static void mlir::memref::ExpandShapeOp::build(OpBuilder &, OperationState &, Type, Value, ArrayRef<ReassociationIndices>): Assertion `succeeded(outputShape) && "unable to infer output shape"' failed.
PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace.
Stack dump:
0.  Program arguments: /data/szy/MLIR/llvm-release/llvm-project/build/bin/mlir-opt -one-shot-bufferize a.mlir
 #0 0x000055e031e51458 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) (/data/szy/MLIR/llvm-release/llvm-project/build/bin/mlir-opt+0x1100458)
 #1 0x000055e031e4ef6e llvm::sys::RunSignalHandlers() (/data/szy/MLIR/llvm-release/llvm-project/build/bin/mlir-opt+0x10fdf6e)
 #2 0x000055e031e51ded SignalHandler(int) Signals.cpp:0:0
 #3 0x00007f876cf5d420 __restore_rt (/lib/x86_64-linux-gnu/libpthread.so.0+0x14420)
 #4 0x00007f876c59a00b raise /build/glibc-LcI20x/glibc-2.31/signal/../sysdeps/unix/sysv/linux/raise.c:51:1
 #5 0x00007f876c579859 abort /build/glibc-LcI20x/glibc-2.31/stdlib/abort.c:81:7
 #6 0x00007f876c579729 get_sysdep_segment_value /build/glibc-LcI20x/glibc-2.31/intl/loadmsgcat.c:509:8
 #7 0x00007f876c579729 _nl_load_domain /build/glibc-LcI20x/glibc-2.31/intl/loadmsgcat.c:970:34
 #8 0x00007f876c58afd6 (/lib/x86_64-linux-gnu/libc.so.6+0x33fd6)
 #9 0x000055e0335829f1 (/data/szy/MLIR/llvm-release/llvm-project/build/bin/mlir-opt+0x28319f1)
#10 0x000055e0341dc85b mlir::memref::ExpandShapeOp mlir::OpBuilder::create<mlir::memref::ExpandShapeOp, llvm::ArrayRef<long>, mlir::Value&, llvm::SmallVector<llvm::SmallVector<long, 2u>, 4u>>(mlir::Location, llvm::ArrayRef<long>&&, mlir::Value&, llvm::SmallVector<llvm::SmallVector<long, 2u>, 4u>&&) (/data/szy/MLIR/llvm-release/llvm-project/build/bin/mlir-opt+0x348b85b)
#11 0x000055e0341dbfda mlir::bufferization::detail::BufferizableOpInterfaceInterfaceTraits::FallbackModel<mlir::tensor::(anonymous namespace)::ExpandShapeOpInterface>::bufferize(mlir::bufferization::detail::BufferizableOpInterfaceInterfaceTraits::Concept const*, mlir::Operation*, mlir::RewriterBase&, mlir::bufferization::BufferizationOptions const&) BufferizableOpInterfaceImpl.cpp:0:0
#12 0x000055e032382840 mlir::bufferization::bufferizeOp(mlir::Operation*, mlir::bufferization::BufferizationOptions const&, mlir::bufferization::BufferizationStatistics*) (/data/szy/MLIR/llvm-release/llvm-project/build/bin/mlir-opt+0x1631840)
#13 0x000055e0323877ec (anonymous namespace)::OneShotBufferizePass::runOnOperation() Bufferize.cpp:0:0
#14 0x000055e034fe18e6 mlir::detail::OpToOpPassAdaptor::run(mlir::Pass*, mlir::Operation*, mlir::AnalysisManager, bool, unsigned int) (/data/szy/MLIR/llvm-release/llvm-project/build/bin/mlir-opt+0x42908e6)
#15 0x000055e034fe2250 mlir::detail::OpToOpPassAdaptor::runPipeline(mlir::OpPassManager&, mlir::Operation*, mlir::AnalysisManager, bool, unsigned int, mlir::PassInstrumentor*, mlir::PassInstrumentation::PipelineParentInfo const*) (/data/szy/MLIR/llvm-release/llvm-project/build/bin/mlir-opt+0x4291250)
#16 0x000055e034fe47f2 mlir::PassManager::run(mlir::Operation*) (/data/szy/MLIR/llvm-release/llvm-project/build/bin/mlir-opt+0x42937f2)
#17 0x000055e034fdd0d1 performActions(llvm::raw_ostream&, std::shared_ptr<llvm::SourceMgr> const&, mlir::MLIRContext*, mlir::MlirOptMainConfig const&) MlirOptMain.cpp:0:0
#18 0x000055e034fdcd2b llvm::LogicalResult llvm::function_ref<llvm::LogicalResult (std::unique_ptr<llvm::MemoryBuffer, std::default_delete<llvm::MemoryBuffer>>, llvm::raw_ostream&)>::callback_fn<mlir::MlirOptMain(llvm::raw_ostream&, std::unique_ptr<llvm::MemoryBuffer, std::default_delete<llvm::MemoryBuffer>>, mlir::DialectRegistry&, mlir::MlirOptMainConfig const&)::$_0>(long, std::unique_ptr<llvm::MemoryBuffer, std::default_delete<llvm::MemoryBuffer>>, llvm::raw_ostream&) MlirOptMain.cpp:0:0
#19 0x000055e0350884e5 mlir::splitAndProcessBuffer(std::unique_ptr<llvm::MemoryBuffer, std::default_delete<llvm::MemoryBuffer>>, llvm::function_ref<llvm::LogicalResult (std::unique_ptr<llvm::MemoryBuffer, std::default_delete<llvm::MemoryBuffer>>, llvm::raw_ostream&)>, llvm::raw_ostream&, llvm::StringRef, llvm::StringRef) (/data/szy/MLIR/llvm-release/llvm-project/build/bin/mlir-opt+0x43374e5)
#20 0x000055e034fd8155 mlir::MlirOptMain(llvm::raw_ostream&, std::unique_ptr<llvm::MemoryBuffer, std::default_delete<llvm::MemoryBuffer>>, mlir::DialectRegistry&, mlir::MlirOptMainConfig const&) (/data/szy/MLIR/llvm-release/llvm-project/build/bin/mlir-opt+0x4287155)
#21 0x000055e034fd83ff mlir::MlirOptMain(int, char**, llvm::StringRef, llvm::StringRef, mlir::DialectRegistry&) (/data/szy/MLIR/llvm-release/llvm-project/build/bin/mlir-opt+0x42873ff)
#22 0x000055e034fd872e mlir::MlirOptMain(int, char**, llvm::StringRef, mlir::DialectRegistry&) (/data/szy/MLIR/llvm-release/llvm-project/build/bin/mlir-opt+0x428772e)
#23 0x000055e031e31fe7 main (/data/szy/MLIR/llvm-release/llvm-project/build/bin/mlir-opt+0x10e0fe7)
#24 0x00007f876c57b083 __libc_start_main /build/glibc-LcI20x/glibc-2.31/csu/../csu/libc-start.c:342:3
#25 0x000055e031e31b6e _start (/data/szy/MLIR/llvm-release/llvm-project/build/bin/mlir-opt+0x10e0b6e)
DarshanRamakant commented 1 month ago

This is happening because the IR contains multile dynamic dims for the "tensor.expand_shape"

DynamicShape

0th and the 2nd dimension both are dynamic for the same re-association group. So the transform doesnt understand how to distribute the data. I also have my ownd doubt about the semantic validity of this IR. For example, if the input shape is 3x4x2, then the collapsed shape will be 24. Then you are dividing it by 9, which will give you the quotient as 2. Then you are trying to expand it as <2x3x24>, which will exceed the original number of elements.

axeabc commented 1 month ago

This is happening because the IR contains multile dynamic dims for the "tensor.expand_shape" DynamicShape

0th and the 2nd dimension both are dynamic for the same re-association group. So the transform doesnt understand how to distribute the data. I also have my ownd doubt about the semantic validity of this IR. For example, if the input shape is 3x4x2, then the collapsed shape will be 24. Then you are dividing it by 9, which will give you the quotient as 2. Then you are trying to expand it as <2x3x24>, which will exceed the original number of elements.

Thank you for your explanation. This IR is randomly generated via certain fuzzing technique, so it might have a semantic problem. It's a better practice to report an error rather than crash for mlir-opt?

CoTinker commented 1 month ago

https://github.com/llvm/llvm-project/blob/8e0daabe97cf5e73402bcb4c3e54b3583199ba8f/mlir/lib/Dialect/MemRef/IR/MemRefOps.cpp#L2306-L2318 Can the assertion be changed to an error here?