openvinotoolkit / open_model_zoo

Pre-trained Deep Learning models and demos (high quality and extremely fast)
https://docs.openvino.ai/latest/model_zoo.html
Apache License 2.0
4.09k stars 1.37k forks source link

"Failed building wheel for onnx" Error while install_prerequisites.bat #82

Closed Bahramudin closed 5 years ago

Bahramudin commented 5 years ago

I trying to install OpenVINO Toolkit version 2019R1 which the latest version of today (2019-04-11). But during installation I got this error:

Failed building wheel for onnx

I installed everything as the documentation before I was using version 2018R5 and it was installed successfully and working well, but now I want to install the new version, but it gives me the above error.

It is the complete stack trace:

Building wheels for collected packages: onnx Building wheel for onnx (setup.py) ... error Complete output from command c:\users\bahra\appdata\local\programs\python\python37\python.exe -u -c "import setuptools, tokenize;file='C:\Users\bahra\AppData\Local\Temp\pip-install-7w3v8o85\onnx\setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" bdist_wheel -d C:\Users\bahra\AppData\Local\Temp\pip-wheel-zqjeth4c --python-tag cp37: fatal: not a git repository (or any of the parent directories): .git running bdist_wheel running build running build_py running create_version running cmake_build -- Building for: Visual Studio 15 2017 -- Build type not set - defaulting to Release -- Selecting Windows SDK version 10.0.17763.0 to target Windows 10.0.17134. -- The C compiler identification is MSVC 19.16.27030.1 -- The CXX compiler identification is MSVC 19.16.27030.1 -- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2017/Enterprise/VC/Tools/MSVC/14.16.27023/bin/Hostx86/x64/cl.exe -- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2017/Enterprise/VC/Tools/MSVC/14.16.27023/bin/Hostx86/x64/cl.exe -- works -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Detecting C compile features -- Detecting C compile features - done -- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2017/Enterprise/VC/Tools/MSVC/14.16.27023/bin/Hostx86/x64/cl.exe -- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2017/Enterprise/VC/Tools/MSVC/14.16.27023/bin/Hostx86/x64/cl.exe -- works -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Detecting CXX compile features -- Detecting CXX compile features - done CMake Error at CMakeLists.txt:217 (message): Protobuf compiler not found Call Stack (most recent call first): CMakeLists.txt:248 (relative_protobuf_generate_cpp)

-- Configuring incomplete, errors occurred! See also "C:/Users/bahra/AppData/Local/Temp/pip-install-7w3v8o85/onnx/.setuptools-cmake-build/CMakeFiles/CMakeOutput.log". Traceback (most recent call last): File "", line 1, in File "C:\Users\bahra\AppData\Local\Temp\pip-install-7w3v8o85\onnx\setup.py", line 328, in 'backend-test-tools = onnx.backend.test.cmd_tools:main', File "c:\users\bahra\appdata\local\programs\python\python37\lib\site-packages\setuptools__init__.py", line 145, in setup return distutils.core.setup(**attrs) File "c:\users\bahra\appdata\local\programs\python\python37\lib\distutils\core.py", line 148, in setup dist.run_commands() File "c:\users\bahra\appdata\local\programs\python\python37\lib\distutils\dist.py", line 966, in run_commands self.run_command(cmd) File "c:\users\bahra\appdata\local\programs\python\python37\lib\distutils\dist.py", line 985, in run_command cmd_obj.run() File "C:\Users\bahra\AppData\Roaming\Python\Python37\site-packages\wheel\bdist_wheel.py", line 192, in run self.run_command('build') File "c:\users\bahra\appdata\local\programs\python\python37\lib\distutils\cmd.py", line 313, in run_command self.distribution.run_command(command) File "c:\users\bahra\appdata\local\programs\python\python37\lib\distutils\dist.py", line 985, in run_command cmd_obj.run() File "c:\users\bahra\appdata\local\programs\python\python37\lib\distutils\command\build.py", line 135, in run self.run_command(cmd_name) File "c:\users\bahra\appdata\local\programs\python\python37\lib\distutils\cmd.py", line 313, in run_command self.distribution.run_command(command) File "c:\users\bahra\appdata\local\programs\python\python37\lib\distutils\dist.py", line 985, in run_command cmd_obj.run() File "C:\Users\bahra\AppData\Local\Temp\pip-install-7w3v8o85\onnx\setup.py", line 203, in run self.run_command('cmake_build') File "c:\users\bahra\appdata\local\programs\python\python37\lib\distutils\cmd.py", line 313, in run_command self.distribution.run_command(command) File "c:\users\bahra\appdata\local\programs\python\python37\lib\distutils\dist.py", line 985, in run_command cmd_obj.run() File "C:\Users\bahra\AppData\Local\Temp\pip-install-7w3v8o85\onnx\setup.py", line 190, in run subprocess.check_call(cmake_args) File "c:\users\bahra\appdata\local\programs\python\python37\lib\subprocess.py", line 347, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['C:\Program Files\CMake\bin\cmake.exe', '-DPYTHON_INCLUDE_DIR=c:\users\bahra\appdata\local\programs\python\python37\include', '-DPYTHON_EXECUTABLE=c:\users\bahra\appdata\local\programs\python\python37\python.exe', '-DBUILD_ONNX_PYTHON=ON', '-DCMAKE_EXPORT_COMPILE_COMMANDS=ON', '-DONNX_NAMESPACE=onnx', '-DPY_EXT_SUFFIX=.cp37-win_amd64.pyd', '-DPY_VERSION=3.7', '-DONNX_USE_MSVC_STATIC_RUNTIME=ON', '-DCMAKE_GENERATOR_PLATFORM=x64', 'C:\Users\bahra\AppData\Local\Temp\pip-install-7w3v8o85\onnx']' returned non-zero exit status 1.


Failed building wheel for onnx Running setup.py clean for onnx Failed to build onnx Installing collected packages: onnx, test-generator, defusedxml Running setup.py install for onnx ... error Complete output from command c:\users\bahra\appdata\local\programs\python\python37\python.exe -u -c "import setuptools, tokenize;file='C:\Users\bahra\AppData\Local\Temp\pip-install-7w3v8o85\onnx\setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" install --record C:\Users\bahra\AppData\Local\Temp\pip-record-aj9pyef5\install-record.txt --single-version-externally-managed --compile --user --prefix=: fatal: not a git repository (or any of the parent directories): .git running install running build running build_py running create_version running cmake_build -- Selecting Windows SDK version 10.0.17763.0 to target Windows 10.0.17134. CMake Error at CMakeLists.txt:217 (message): Protobuf compiler not found Call Stack (most recent call first): CMakeLists.txt:248 (relative_protobuf_generate_cpp)

-- Configuring incomplete, errors occurred!
See also "C:/Users/bahra/AppData/Local/Temp/pip-install-7w3v8o85/onnx/.setuptools-cmake-build/CMakeFiles/CMakeOutput.log".
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Users\bahra\AppData\Local\Temp\pip-install-7w3v8o85\onnx\setup.py", line 328, in <module>
    'backend-test-tools = onnx.backend.test.cmd_tools:main',
  File "c:\users\bahra\appdata\local\programs\python\python37\lib\site-packages\setuptools\__init__.py", line 145, in setup
    return distutils.core.setup(**attrs)
  File "c:\users\bahra\appdata\local\programs\python\python37\lib\distutils\core.py", line 148, in setup
    dist.run_commands()
  File "c:\users\bahra\appdata\local\programs\python\python37\lib\distutils\dist.py", line 966, in run_commands
    self.run_command(cmd)
  File "c:\users\bahra\appdata\local\programs\python\python37\lib\distutils\dist.py", line 985, in run_command
    cmd_obj.run()
  File "c:\users\bahra\appdata\local\programs\python\python37\lib\site-packages\setuptools\command\install.py", line 61, in run
    return orig.install.run(self)
  File "c:\users\bahra\appdata\local\programs\python\python37\lib\distutils\command\install.py", line 545, in run
    self.run_command('build')
  File "c:\users\bahra\appdata\local\programs\python\python37\lib\distutils\cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "c:\users\bahra\appdata\local\programs\python\python37\lib\distutils\dist.py", line 985, in run_command
    cmd_obj.run()
  File "c:\users\bahra\appdata\local\programs\python\python37\lib\distutils\command\build.py", line 135, in run
    self.run_command(cmd_name)
  File "c:\users\bahra\appdata\local\programs\python\python37\lib\distutils\cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "c:\users\bahra\appdata\local\programs\python\python37\lib\distutils\dist.py", line 985, in run_command
    cmd_obj.run()
  File "C:\Users\bahra\AppData\Local\Temp\pip-install-7w3v8o85\onnx\setup.py", line 203, in run
    self.run_command('cmake_build')
  File "c:\users\bahra\appdata\local\programs\python\python37\lib\distutils\cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "c:\users\bahra\appdata\local\programs\python\python37\lib\distutils\dist.py", line 985, in run_command
    cmd_obj.run()
  File "C:\Users\bahra\AppData\Local\Temp\pip-install-7w3v8o85\onnx\setup.py", line 190, in run
    subprocess.check_call(cmake_args)
  File "c:\users\bahra\appdata\local\programs\python\python37\lib\subprocess.py", line 347, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['C:\\Program Files\\CMake\\bin\\cmake.exe', '-DPYTHON_INCLUDE_DIR=c:\\users\\bahra\\appdata\\local\\programs\\python\\python37\\include', '-DPYTHON_EXECUTABLE=c:\\users\\bahra\\appdata\\local\\programs\\python\\python37\\python.exe', '-DBUILD_ONNX_PYTHON=ON', '-DCMAKE_EXPORT_COMPILE_COMMANDS=ON', '-DONNX_NAMESPACE=onnx', '-DPY_EXT_SUFFIX=.cp37-win_amd64.pyd', '-DPY_VERSION=3.7', '-DONNX_USE_MSVC_STATIC_RUNTIME=ON', '-DCMAKE_GENERATOR_PLATFORM=x64', 'C:\\Users\\bahra\\AppData\\Local\\Temp\\pip-install-7w3v8o85\\onnx']' returned non-zero exit status 1.

----------------------------------------

Command "c:\users\bahra\appdata\local\programs\python\python37\python.exe -u -c "import setuptools, tokenize;file='C:\Users\bahra\AppData\Local\Temp\pip-install-7w3v8o85\onnx\setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" install --record C:\Users\bahra\AppData\Local\Temp\pip-record-aj9pyef5\install-record.txt --single-version-externally-managed --compile --user --prefix=" failed with error code 1 in C:\Users\bahra\AppData\Local\Temp\pip-install-7w3v8o85\onnx\


Warning: please expect that Model Optimizer conversion might be slow. You can boost conversion speed by installing protobuf-*.egg located in the "model-optimizer\install_prerequisites" folder or building protobuf library from sources. For more information please refer to Model Optimizer FAQ, question #80.

snosov1 commented 5 years ago

The root cause seems to be the missing Protobuf (see the line Protobuf compiler not found). Not sure where it should be fixed.

@vmatveic Slava, do you think it's an issue of the installer or the environment?

Bahramudin commented 5 years ago

@snosov1 Thanks for the reply! But when I want to install the Protobuf, gives me this:

pip3 install Protobuf Requirement already satisfied: Protobuf in c:\users\bahra\appdata\roaming\python\python37\site-packages (3.6.1) Requirement already satisfied: six>=1.9 in c:\users\bahra\appdata\roaming\python\python37\site-packages (from Protobuf) (1.12.0) Requirement already satisfied: setuptools in c:\users\bahra\appdata\local\programs\python\python37\lib\site-packages (from Protobuf) (40.8.0)

alalek commented 5 years ago

pip3 install Protobuf

pip3 install onnx is your target command.

protobuf python is not the same as protobuf C++ which is in mentioned error log.

Building wheels

Building of Python packages from sources usually doesn't work well on Windows platform.

Try to downgrade Python or downgrade onnx version - to ensure that it is loaded by pip via wheels (prebuilt wheels may not be available for "latest" configurations).


This issue can't be fixed by patching code in this repository.

Bahramudin commented 5 years ago

@alalek Yes, the problem was with Python version. I have one another question, but I don't know to ask it here or open a new issue, OK I will prefer to ask it here.

That is: In the latest version the pre-trained models (intel_models) are not bundled into the download package, so now the intel_models there is only the description of the model.

So I found a script which is used to download the models. But here I found some problem which some of the models cannot be downloaded, the connection timed out or the connection cannot be established, the host not responding and so on problems, so it makes hard to download the models, the problem may be due to some restriction to some addresses it not possible to have access to some of the links.

It is the result of my download, after many tries

###############|| Post processing ||###############

========= Changing input dimensions in vgg19.prototxt ========= ========= Changing input dimensions in vgg16.prototxt ========= ========= Changing input dimensions in googlenet-v1.prototxt ========= ========= Changing input dimensions in googlenet-v2.prototxt ========= ========= Moving to new Caffe layer presentation googlenet-v2.prototxt ========= ========= Changing input dimensions in alexnet.prototxt ========= FAILED: Sphereface googlenet-v4 inception-resnet-v2 mobilenet-ssd mobilenet-v2-1.4-224 mtcnn-o mtcnn-p mtcnn-r resnet-101 resnet-152 resnet-50 se-inception se-resnet-101 se-resnet-152 se-resnet-50 se-resnext-101 se-resnext-50 squeezenet1.0 squeezenet1.1 ssd300 ssd512

One error example:

Error Connecting: HTTPSConnectionPool(host='docs.google.com', port=443): Max retries exceeded with url: /uc?export=download&id=0B_geeR2lTMegb2F6dmlmOXhWaVk (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x00000202B81BA160>: Failed to establish a new connection:[WinError 10060]

Most occurred error hosts are:

HTTPSConnectionPool(host='docs.google.com', port=443) HTTPSConnectionPool(host='onedrive.live.com', port=443)

So is it not possible to bundle all the models into a single package and download at once? I think the better way is to provide three download ways:

So now everyone will be able to download whatever models they want.


One small question: What does the value exactly mean in after of the each model's names, e.g: instance-segmentation-security-0033 instance-segmentation-security-0049 AND face-detection-adas-0001 face-detection-retail-0004

Thanks in advance!!

Update: One another problem with download script is: when running it for the second time, then it will download the already downloaded models again, so it will be good to check for the model existing first, if already exist, skip and go for the next one. So if some models fail to download and when want to run the second time you cannot run it again, it is a big waste of time and of bandwidth for both sides Server and the Client.

alalek commented 5 years ago

numbers

It is like a version (of used model's scheme, training processes, including used datasets). May be faster and/or more accurate. There is no "gold" solution, so models are always evolving. Some interesting variants goes public.

face-detection-adas-0001 face-detection-retail-0004

Perhaps they are trained on different datasets. And/or based on different scheme of models. Or have different purposes. Refer to attached description/readme files of these models.

Evaluate all applicable variants on your data and pick the best based on these metrics:

snosov1 commented 5 years ago

On the value interpretation - @alalek is correct. It's simply an ID, so we can distinguish between different models.

On the download issue - @atsyplen can you comment? In the meantime, @Bahramudin can you try model_downloader from this repo?

atsyplen commented 5 years ago

@Bahramudin regarding HTTPS errors, first of all please check that the correct date and time settings are set. Secondly make sure that you use a correct https proxy settings, if you need it for your network. Regarding requested features - you can find them in the latest version of model downloader from this repo.

Bahramudin commented 5 years ago

@snosov1 I am just using the one you mentioned to download the models. I think it is the reason which some of the links is not accessible may be restricted, anyway, I have tried many ways such as using VPN, but also was not working well, but VPN is not very stable after a while it is interrupted.

And also this script has a problem which when run it for the second time, then it will start downloading the already downloaded models, it will be good to check if the file already exists.

So I think it will be good to provide three ways of download which I mentioned in the above comment.

About the version, I checked the lower the heavier bin file, so the heavier may be more accurate and also works slow.

snosov1 commented 5 years ago

@Bahramudin please, reopen if it's still relevant