microsoft / superbenchmark

A validation and profiling tool for AI infrastructure
https://aka.ms/superbench
MIT License
270 stars 59 forks source link

pytorch cannot find libopen-orted-mpir.so #514

Closed whzyf951620 closed 1 year ago

whzyf951620 commented 1 year ago

What's the issue, what's expected?: pytorch cannot find libopen-orted-mpir.so

Log message or shapshot?:

ERROR: libopen-orted-mpir.so: cannot open shared object file: No such file or directory Traceback (most recent call last): File "/usr/local/lib/python3.8/dist-packages/knack/cli.py", line 233, in invoke cmd_result = self.invocation.execute(args) File "/usr/local/lib/python3.8/dist-packages/knack/invocation.py", line 224, in execute cmd_result = parsed_args.func(params) File "/usr/local/lib/python3.8/dist-packages/knack/commands.py", line 146, in call return self.handler(*args, kwargs) File "/usr/local/lib/python3.8/dist-packages/knack/commands.py", line 253, in _command_handler result = op(client, command_args) if client else op(**command_args) File "/usr/local/lib/python3.8/dist-packages/superbench/cli/_handler.py", line 208, in exec_command_handler executor.exec() File "/usr/local/lib/python3.8/dist-packages/superbench/executor/executor.py", line 247, in exec context = BenchmarkRegistry.create_benchmark_context( File "/usr/local/lib/python3.8/dist-packages/superbench/common/utils/lazy_import.py", line 42, in getattr self._import() File "/usr/local/lib/python3.8/dist-packages/superbench/common/utils/lazy_import.py", line 31, in _import self._callback() File "/usr/local/lib/python3.8/dist-packages/superbench/benchmarks/init.py", line 15, in 'superbench.benchmarks.registry', 'BenchmarkRegistry', lambda: list( File "/usr/lib/python3.8/importlib/init.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "", line 1014, in _gcd_import File "", line 991, in _find_and_load File "", line 975, in _find_and_load_unlocked File "", line 671, in _load_unlocked File "", line 848, in exec_module File "", line 219, in _call_with_frames_removed File "/usr/local/lib/python3.8/dist-packages/superbench/benchmarks/model_benchmarks/init.py", line 7, in from superbench.benchmarks.model_benchmarks.pytorch_bert import PytorchBERT File "/usr/local/lib/python3.8/dist-packages/superbench/benchmarks/model_benchmarks/pytorch_bert.py", line 6, in import torch File "/usr/local/lib/python3.8/dist-packages/torch/init.py", line 191, in _load_global_deps() File "/usr/local/lib/python3.8/dist-packages/torch/init.py", line 153, in _load_global_deps ctypes.CDLL(lib_path, mode=ctypes.RTLD_GLOBAL) File "/usr/lib/python3.8/ctypes/init.py", line 373, in init self._handle = _dlopen(self._name, mode) OSError: libopen-orted-mpir.so: cannot open shared object file: No such file or directorynon-zero return code

additional information Ubuntu 20.04, python3.8;OpenMPI 4.04; I have libopen-orted-mpir.so in /usr/lib/x86_64-linux-gnu/openmpi/lib/libopen-orted-mpir.so and I have written the path in ~/.bashrc; I have tried to ask chatgpt4, and he cannot solve this issue.

guoshzhao commented 1 year ago

Can't reproduce. Are you still suffering from this issue?

cp5555 commented 1 year ago

Close this issue due to no response.