[BUG] ModuleNotFoundError: No module named 'absl'

byounghak36 commented 3 years ago

Describe the bug

A clear and concise description of what the bug is.

local에서 loadgen build 중 absl model not found error 발생

Screenshots or Logs

If applicable, add screenshots to help explain your problem.

tanssg@ltech-gpu10:~/mlperf-inference/loadgen (test)
$ CFLAGS="-std=c++14 -O3" python setup.py develop
Traceback (most recent call last):
  File "setup.py", line 29, in <module>
    from version_generator import generate_loadgen_version_definitions
  File "/home/tanssg/mlperf-inference/loadgen/version_generator.py", line 25, in <module>
    from absl import app
ModuleNotFoundError: No module named 'absl'

Expected behavior

A clear and concise description of what you expected to happen.

Possible Solution

absl install
```
pip install absl-py
```

Additional context

Add any other context about the problem here.

byounghak36 commented 3 years ago

absl install 후 아래와 같이 에러발생

tanssg@ltech-gpu10:~/mlperf-inference/loadgen (test)
$ pip install absl-py
Defaulting to user installation because normal site-packages is not writeable
Collecting absl-py
  Downloading absl_py-0.13.0-py3-none-any.whl (132 kB)
     |████████████████████████████████| 132 kB 1.1 MB/s
Requirement already satisfied: six in /usr/lib/python3/dist-packages (from absl-py) (1.14.0)
Installing collected packages: absl-py
Successfully installed absl-py-0.13.0
tanssg@ltech-gpu10:~/mlperf-inference/loadgen (test)
$ CFLAGS="-std=c++14 -O3" python setup.py develop
running develop
error: can't create or remove files in install directory

The following error occurred while trying to add or remove files in the
installation directory:

    [Errno 13] Permission denied: '/usr/local/lib/python3.8/dist-packages/test-easy-install-2628022.write-test'

The installation directory you specified (via --install-dir, --prefix, or
the distutils default setting) was:

    /usr/local/lib/python3.8/dist-packages/

Perhaps your account does not have write access to this directory?  If the
installation directory is a system-owned directory, you may need to sign in
as the administrator or "root" account.  If you do not have administrative
access to this machine, you may wish to choose a different installation
directory, preferably one that is listed in your PYTHONPATH environment
variable.

For information on other options, you may wish to consult the
documentation at:

  https://setuptools.readthedocs.io/en/latest/easy_install.html

Please make the appropriate changes for your system and try again.

local에서 빌드하려하여 권한문제가 생긴듯함, docker 내부에서 loadgen build 진행 예정

byounghak36 commented 3 years ago

Screenshots or Logs

docker에서 실행시 아래와 같이 에러발생

root@07a61c5e9ab8:/mlperf-inference/loadgen# CFLAGS="-std=c++14 -O3" python setup.py develop
running develop
running egg_info
creating mlperf_loadgen.egg-info
writing mlperf_loadgen.egg-info/PKG-INFO
writing dependency_links to mlperf_loadgen.egg-info/dependency_links.txt
writing top-level names to mlperf_loadgen.egg-info/top_level.txt
writing manifest file 'mlperf_loadgen.egg-info/SOURCES.txt'
reading manifest file 'mlperf_loadgen.egg-info/SOURCES.txt'
writing manifest file 'mlperf_loadgen.egg-info/SOURCES.txt'
running build_ext
building 'mlperf_loadgen' extension
creating build
creating build/temp.linux-x86_64-3.7
creating build/temp.linux-x86_64-3.7/bindings
creating build/temp.linux-x86_64-3.7/generated
gcc -pthread -B /opt/anaconda3/compiler_compat -Wl,--sysroot=/ -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -std=c++14 -O3 -fPIC -DMAJOR_VERSION=0 -DMINOR_VERSION=5 -I. -I../third_party/pybind/include -I/opt/anaconda3/include/python3.7m -c issue_query_controller.cc -o build/temp.linux-x86_64-3.7/issue_query_controller.o
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
gcc -pthread -B /opt/anaconda3/compiler_compat -Wl,--sysroot=/ -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -std=c++14 -O3 -fPIC -DMAJOR_VERSION=0 -DMINOR_VERSION=5 -I. -I../third_party/pybind/include -I/opt/anaconda3/include/python3.7m -c loadgen.cc -o build/temp.linux-x86_64-3.7/loadgen.o
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
gcc -pthread -B /opt/anaconda3/compiler_compat -Wl,--sysroot=/ -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -std=c++14 -O3 -fPIC -DMAJOR_VERSION=0 -DMINOR_VERSION=5 -I. -I../third_party/pybind/include -I/opt/anaconda3/include/python3.7m -c logging.cc -o build/temp.linux-x86_64-3.7/logging.o
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
gcc -pthread -B /opt/anaconda3/compiler_compat -Wl,--sysroot=/ -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -std=c++14 -O3 -fPIC -DMAJOR_VERSION=0 -DMINOR_VERSION=5 -I. -I../third_party/pybind/include -I/opt/anaconda3/include/python3.7m -c test_settings_internal.cc -o build/temp.linux-x86_64-3.7/test_settings_internal.o
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
gcc -pthread -B /opt/anaconda3/compiler_compat -Wl,--sysroot=/ -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -std=c++14 -O3 -fPIC -DMAJOR_VERSION=0 -DMINOR_VERSION=5 -I. -I../third_party/pybind/include -I/opt/anaconda3/include/python3.7m -c utils.cc -o build/temp.linux-x86_64-3.7/utils.o
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
gcc -pthread -B /opt/anaconda3/compiler_compat -Wl,--sysroot=/ -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -std=c++14 -O3 -fPIC -DMAJOR_VERSION=0 -DMINOR_VERSION=5 -I. -I../third_party/pybind/include -I/opt/anaconda3/include/python3.7m -c version.cc -o build/temp.linux-x86_64-3.7/version.o
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
gcc -pthread -B /opt/anaconda3/compiler_compat -Wl,--sysroot=/ -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -std=c++14 -O3 -fPIC -DMAJOR_VERSION=0 -DMINOR_VERSION=5 -I. -I../third_party/pybind/include -I/opt/anaconda3/include/python3.7m -c bindings/python_api.cc -o build/temp.linux-x86_64-3.7/bindings/python_api.o
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
bindings/python_api.cc:26:33: fatal error: pybind11/functional.h: No such file or directory
compilation terminated.
error: command 'gcc' failed with exit status 1

gcc는 설치되어 있음

root@07a61c5e9ab8:/mlperf-inference/loadgen# gcc --version
gcc (Ubuntu 5.4.0-6ubuntu1~16.04.12) 5.4.0 20160609
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Possible Solution

pybind11 module이 설치가 안되어있어서 문제가 생기는 듯함, 설치 진행

root@07a61c5e9ab8:/mlperf-inference/loadgen# pip install pybind11
Requirement already satisfied: pybind11 in /opt/anaconda3/lib/python3.7/site-packages (2.6.2)
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

이미 설치되어 있는것으로 확인됨... 관련 이슈 구글링
- https://github.com/mlcommons/inference/issues/230
- https://github.com/pybind/pybind11/issues/698

byounghak36 commented 3 years ago

git clone --recurse-submodules https://github.com/ltechkorea/mlperf-inference.git ./mlperf-inference-submodule
power-dev 서브모듈이 없어서 발생한 문제, 서브모듈을 clone 받으면서 해결

ltechkorea / mlperf-inference

[BUG] ModuleNotFoundError: No module named 'absl' #62

Describe the bug

Screenshots or Logs

Expected behavior

Possible Solution

Additional context

Screenshots or Logs

Possible Solution