ExecutionEngine produces expected result but also error Failed to create MemoryBuffer for: libmlir_c_runner_utils.so

[bixia1]

Bugzilla Link	49237
Resolution	DUPLICATE
Resolved on	Feb 18, 2021 07:59
Version	unspecified
OS	All
CC	@jpienaar,@River707

Extended Description

I first set up LD_LIBRARY_PATH so that dlopen can find libmlir_c_runner_utils.so. Then I ran "mlir-cpu-runner -shared-libs=libmlir_c_runner_utils.so ". The command ran fine and produced the expected result. However, I also got this bogus error message:

Failed to create MemoryBuffer for: libmlir_c_runner_utils.so Error: No such file or directory

It is a bogus error message for my case, but could be a real problem for other situations.

The error message is from this code block, which doesn't consider LD_LIBRARY_PATH when trying to open the shared library: https://github.com/llvm/llvm-project/blob/48d2e04152366890e0b219a5f7c6f5b4905af480/mlir/lib/ExecutionEngine/ExecutionEngine.cpp#L260-L265

On the other hand, the shared library was correctly loaded from this code block, as it takes LD_LIBRARY_PATH into consideration -- which is the reason that my command works: https://github.com/llvm/llvm-project/blob/2628e9146120507b2cf025f5c4ccc857cc3724c4/mlir/lib/ExecutionEngine/JitRunner.cpp#L175

You can use mlir/integration_test in the current LLVM tree to demonstrate this:

first set up PATH for mlir-opt and mlir-cpu-runner, LD_LIBRARY_PATH for libmlir_c_runner_utils.so

mlir-opt mlir/integration_test/Sparse/CPU/sparse_sum.mlir \ --test-sparsification="lower" --convert-linalg-to-loops --func-bufferize --tensor-constant-bufferize --tensor-bufferize --finalizing-bufferize --convert-scf-to-std --convert-vector-to-llvm --convert-std-to-llvm | \ TENSOR0="mlir/integration_test/data/test.mtx" \ -cpu-runner -e entry -entry-point-result=void -shared-libs=libmlir_c_runner_utils.so

Output from the command: Failed to create MemoryBuffer for: libmlir_c_runner_utils.so Error: No such file or directory 28.2

If you unset LD_LIBRARY_PATH and run the same command again, you should get different errors and no output from the mlir program being jit: Failed to create MemoryBuffer for: libmlir_c_runner_utils.so Error: No such file or directory JIT session error: Symbols not found: [ delSparseTensor, sparseValsF64, getTensorFilename, sparsePtrsI64, newSparseTensor, printF64, printNewline, sparseIndxsI64 ] Error: Failed to materialize symbols: { (main, { _mlir_entry, entry, kernel_sum_reduce, _mlir_kernel_sum_reduce }) }

[bixia1]

This bug has been marked as a duplicate of bug llvm/llvm-project#48580

[bixia1]

assigned to @joker-eph