search

dlshriver / dnnv

A Framework for Deep Neural Network Verification
https://docs.dnnv.org
MIT License
56 stars 18 forks source link

dnnv_manage fails while installing verifiers #108

Open yusiyoh opened 1 year ago

yusiyoh commented 1 year ago

Hello and thank you for this work.

I wanted to use this framework however, dnnv_manage fails with all verifiers. I have built dnnv in linux and have dependencies.

Error is: dnnv._manage.errors.InstallError: Failed to install dependency: <verifier_name>

Thanks in advance

dlshriver commented 1 year ago

I cannot reproduce the issue. What is the exact output that you are getting?

Also, what is your OS, CPU architecture, and Python version? These may help diagnose the issue.

yusiyoh commented 1 year ago

Example error (I have same error in the end for other verifiers too): 

 dnnv_manage install eran
INFO     2023-06-07 10:59:12,753 (dnnv_manage.install) installing eran
+ cd /home/ycs/.local/share/dnnv/verifier_virtualenvs
+ rm -rf eran
+ python -m venv eran
+ . eran/bin/activate
+ deactivate nondestructive
+ [ -n  ]
+ [ -n  ]
+ [ -n  -o -n  ]
+ [ -n  ]
+ unset VIRTUAL_ENV
+ [ ! nondestructive = nondestructive ]
+ VIRTUAL_ENV=/home/ycs/.local/share/dnnv/verifier_virtualenvs/eran
+ export VIRTUAL_ENV
+ _OLD_VIRTUAL_PATH=/home/ycs/.local/share/dnnv/opt/gurobi912/linux64/bin:/home/ycs/.cache/dnnv/m4-1.4.1/bin:/home/ycs/.cache/dnnv/gmp-6.1.2/bin:/home/ycs/.cache/dnnv/mpfr-4.1.0/bin:/home/ycs/.cache/dnnv/cddlib-0.94m/bin:/home/ycs/.local/bin:/home/ycs/miniconda3/envs/dnnv/bin:/home/ycs/miniconda3/condabin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/usr/lib/wsl/lib:/mnt/c/gurobi1001/win64/bin:/mnt/c/Program Files (x86)/Common Files/Oracle/Java/javapath:/mnt/c/Program Files/Eclipse Adoptium/jdk-11.0.18.10-hotspot/bin:/mnt/c/WINDOWS/system32:/mnt/c/WINDOWS:/mnt/c/WINDOWS/System32/Wbem:/mnt/c/WINDOWS/System32/WindowsPowerShell/v1.0:/mnt/c/WINDOWS/System32/OpenSSH:/mnt/c/Program Files/dotnet:/mnt/c/Program Files/TortoiseSVN/bin:/mnt/c/Users/yusuf.simsek/AppData/Local/Programs/Python/Python310:/mnt/c/Program Files (x86)/Graphviz2.38/bin:/mnt/c/Program Files/Docker/Docker/resources/bin:/mnt/c/Users/yusuf.simsek/AppData/Local/Programs/Python/Python37/Scripts:/mnt/c/Users/yusuf.simsek/AppData/Local/Programs/Python/Python37:/mnt/c/Users/yusuf.simsek/AppData/Local/Programs/Python/Launcher:/mnt/c/Users/yusuf.simsek/AppData/Local/Microsoft/WindowsApps:/mnt/c/Users/yusuf.simsek/AppData/Local/Programs/Microsoft VS Code/bin:/mnt/c/Users/yusuf.simsek/CMake/bin:/mnt/c/Program Files/CMake/bin:/snap/bin
+ PATH=/home/ycs/.local/share/dnnv/verifier_virtualenvs/eran/bin:/home/ycs/.local/share/dnnv/opt/gurobi912/linux64/bin:/home/ycs/.cache/dnnv/m4-1.4.1/bin:/home/ycs/.cache/dnnv/gmp-6.1.2/bin:/home/ycs/.cache/dnnv/mpfr-4.1.0/bin:/home/ycs/.cache/dnnv/cddlib-0.94m/bin:/home/ycs/.local/bin:/home/ycs/miniconda3/envs/dnnv/bin:/home/ycs/miniconda3/condabin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/usr/lib/wsl/lib:/mnt/c/gurobi1001/win64/bin:/mnt/c/Program Files (x86)/Common Files/Oracle/Java/javapath:/mnt/c/Program Files/Eclipse Adoptium/jdk-11.0.18.10-hotspot/bin:/mnt/c/WINDOWS/system32:/mnt/c/WINDOWS:/mnt/c/WINDOWS/System32/Wbem:/mnt/c/WINDOWS/System32/WindowsPowerShell/v1.0:/mnt/c/WINDOWS/System32/OpenSSH:/mnt/c/Program Files/dotnet:/mnt/c/Program Files/TortoiseSVN/bin:/mnt/c/Users/yusuf.simsek/AppData/Local/Programs/Python/Python310:/mnt/c/Program Files (x86)/Graphviz2.38/bin:/mnt/c/Program Files/Docker/Docker/resources/bin:/mnt/c/Users/yusuf.simsek/AppData/Local/Programs/Python/Python37/Scripts:/mnt/c/Users/yusuf.simsek/AppData/Local/Programs/Python/Python37:/mnt/c/Users/yusuf.simsek/AppData/Local/Programs/Python/Launcher:/mnt/c/Users/yusuf.simsek/AppData/Local/Microsoft/WindowsApps:/mnt/c/Users/yusuf.simsek/AppData/Local/Programs/Microsoft VS Code/bin:/mnt/c/Users/yusuf.simsek/CMake/bin:/mnt/c/Program Files/CMake/bin:/snap/bin
+ export PATH
+ [ -n  ]
+ [ -z  ]
+ _OLD_VIRTUAL_PS1=$
+ [ x(eran)  != x ]
+ PS1=(eran) $
+ export PS1
+ [ -n  -o -n  ]
+ pip install --upgrade pip
Requirement already satisfied: pip in ./eran/lib/python3.7/site-packages (22.0.4)
Collecting pip
  Using cached pip-23.1.2-py3-none-any.whl (2.1 MB)
Installing collected packages: pip
  Attempting uninstall: pip
    Found existing installation: pip 22.0.4
    Uninstalling pip-22.0.4:
      Successfully uninstalled pip-22.0.4
Successfully installed pip-23.1.2
+ pip install numpy>=1.19,<1.22 tensorflow>=2.4,<2.8 onnx>=1.8,<1.11 onnxruntime>=1.7,<1.11 torch>=1.8,<1.11 torchvision>=0.9,<0.12 mpmath>=1.2,<1.3 pillow>=8.1 protobuf<=3.20
Collecting numpy<1.22,>=1.19
  Using cached numpy-1.21.6-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (15.7 MB)
Collecting tensorflow<2.8,>=2.4
  Using cached tensorflow-2.7.4-cp37-cp37m-manylinux2010_x86_64.whl (495.5 MB)
Collecting onnx<1.11,>=1.8
  Using cached onnx-1.10.2-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (12.7 MB)
Collecting onnxruntime<1.11,>=1.7
  Using cached onnxruntime-1.10.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.9 MB)
Collecting torch<1.11,>=1.8
  Using cached torch-1.10.2-cp37-cp37m-manylinux1_x86_64.whl (881.9 MB)
Collecting torchvision<0.12,>=0.9
  Using cached torchvision-0.11.3-cp37-cp37m-manylinux1_x86_64.whl (23.2 MB)
Collecting mpmath<1.3,>=1.2
  Using cached mpmath-1.2.1-py3-none-any.whl (532 kB)
Collecting pillow>=8.1
  Using cached Pillow-9.5.0-cp37-cp37m-manylinux_2_28_x86_64.whl (3.4 MB)
Collecting protobuf<=3.20
  Using cached protobuf-3.20.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.whl (1.0 MB)
Collecting absl-py>=0.4.0 (from tensorflow<2.8,>=2.4)
  Using cached absl_py-1.4.0-py3-none-any.whl (126 kB)
Collecting astunparse>=1.6.0 (from tensorflow<2.8,>=2.4)
  Using cached astunparse-1.6.3-py2.py3-none-any.whl (12 kB)
Collecting libclang>=9.0.1 (from tensorflow<2.8,>=2.4)
  Using cached libclang-16.0.0-py2.py3-none-manylinux2010_x86_64.whl (22.9 MB)
Collecting flatbuffers<3.0,>=1.12 (from tensorflow<2.8,>=2.4)
  Using cached flatbuffers-2.0.7-py2.py3-none-any.whl (26 kB)
Collecting google-pasta>=0.1.1 (from tensorflow<2.8,>=2.4)
  Using cached google_pasta-0.2.0-py3-none-any.whl (57 kB)
Collecting h5py>=2.9.0 (from tensorflow<2.8,>=2.4)
  Using cached h5py-3.8.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.3 MB)
Collecting keras-preprocessing>=1.1.1 (from tensorflow<2.8,>=2.4)
  Using cached Keras_Preprocessing-1.1.2-py2.py3-none-any.whl (42 kB)
Collecting opt-einsum>=2.3.2 (from tensorflow<2.8,>=2.4)
  Using cached opt_einsum-3.3.0-py3-none-any.whl (65 kB)
Collecting protobuf<=3.20
  Using cached protobuf-3.19.6-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.1 MB)
Collecting six>=1.12.0 (from tensorflow<2.8,>=2.4)
  Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting termcolor>=1.1.0 (from tensorflow<2.8,>=2.4)
  Using cached termcolor-2.3.0-py3-none-any.whl (6.9 kB)
Collecting typing-extensions>=3.6.6 (from tensorflow<2.8,>=2.4)
  Downloading typing_extensions-4.6.3-py3-none-any.whl (31 kB)
Collecting wheel<1.0,>=0.32.0 (from tensorflow<2.8,>=2.4)
  Using cached wheel-0.40.0-py3-none-any.whl (64 kB)
Collecting wrapt>=1.11.0 (from tensorflow<2.8,>=2.4)
  Using cached wrapt-1.15.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
Collecting gast<0.5.0,>=0.2.1 (from tensorflow<2.8,>=2.4)
  Using cached gast-0.4.0-py3-none-any.whl (9.8 kB)
Collecting tensorboard~=2.6 (from tensorflow<2.8,>=2.4)
  Using cached tensorboard-2.11.2-py3-none-any.whl (6.0 MB)
Collecting tensorflow-estimator<2.8,~=2.7.0rc0 (from tensorflow<2.8,>=2.4)
  Using cached tensorflow_estimator-2.7.0-py2.py3-none-any.whl (463 kB)
Collecting keras<2.8,>=2.7.0rc0 (from tensorflow<2.8,>=2.4)
  Using cached keras-2.7.0-py2.py3-none-any.whl (1.3 MB)
Collecting tensorflow-io-gcs-filesystem>=0.21.0 (from tensorflow<2.8,>=2.4)
  Using cached tensorflow_io_gcs_filesystem-0.32.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (2.4 MB)
Collecting grpcio<2.0,>=1.24.3 (from tensorflow<2.8,>=2.4)
  Downloading grpcio-1.54.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.1 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.1/5.1 MB 7.0 MB/s eta 0:00:00
Collecting google-auth<3,>=1.6.3 (from tensorboard~=2.6->tensorflow<2.8,>=2.4)
  Downloading google_auth-2.19.1-py2.py3-none-any.whl (181 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 181.3/181.3 kB 6.3 MB/s eta 0:00:00
Collecting google-auth-oauthlib<0.5,>=0.4.1 (from tensorboard~=2.6->tensorflow<2.8,>=2.4)
  Using cached google_auth_oauthlib-0.4.6-py2.py3-none-any.whl (18 kB)
Collecting markdown>=2.6.8 (from tensorboard~=2.6->tensorflow<2.8,>=2.4)
  Using cached Markdown-3.4.3-py3-none-any.whl (93 kB)
Collecting requests<3,>=2.21.0 (from tensorboard~=2.6->tensorflow<2.8,>=2.4)
  Downloading requests-2.31.0-py3-none-any.whl (62 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 62.6/62.6 kB 8.4 MB/s eta 0:00:00
Requirement already satisfied: setuptools>=41.0.0 in ./eran/lib/python3.7/site-packages (from tensorboard~=2.6->tensorflow<2.8,>=2.4) (47.1.0)
Collecting tensorboard-data-server<0.7.0,>=0.6.0 (from tensorboard~=2.6->tensorflow<2.8,>=2.4)
  Using cached tensorboard_data_server-0.6.1-py3-none-manylinux2010_x86_64.whl (4.9 MB)
Collecting tensorboard-plugin-wit>=1.6.0 (from tensorboard~=2.6->tensorflow<2.8,>=2.4)
  Using cached tensorboard_plugin_wit-1.8.1-py3-none-any.whl (781 kB)
Collecting werkzeug>=1.0.1 (from tensorboard~=2.6->tensorflow<2.8,>=2.4)
  Using cached Werkzeug-2.2.3-py3-none-any.whl (233 kB)
Collecting cachetools<6.0,>=2.0.0 (from google-auth<3,>=1.6.3->tensorboard~=2.6->tensorflow<2.8,>=2.4)
  Downloading cachetools-5.3.1-py3-none-any.whl (9.3 kB)
Collecting pyasn1-modules>=0.2.1 (from google-auth<3,>=1.6.3->tensorboard~=2.6->tensorflow<2.8,>=2.4)
  Using cached pyasn1_modules-0.3.0-py2.py3-none-any.whl (181 kB)
Collecting rsa<5,>=3.1.4 (from google-auth<3,>=1.6.3->tensorboard~=2.6->tensorflow<2.8,>=2.4)
  Using cached rsa-4.9-py3-none-any.whl (34 kB)
Collecting urllib3<2.0 (from google-auth<3,>=1.6.3->tensorboard~=2.6->tensorflow<2.8,>=2.4)
  Downloading urllib3-1.26.16-py2.py3-none-any.whl (143 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 143.1/143.1 kB 6.9 MB/s eta 0:00:00
Collecting requests-oauthlib>=0.7.0 (from google-auth-oauthlib<0.5,>=0.4.1->tensorboard~=2.6->tensorflow<2.8,>=2.4)
  Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting importlib-metadata>=4.4 (from markdown>=2.6.8->tensorboard~=2.6->tensorflow<2.8,>=2.4)
  Using cached importlib_metadata-6.6.0-py3-none-any.whl (22 kB)
Collecting charset-normalizer<4,>=2 (from requests<3,>=2.21.0->tensorboard~=2.6->tensorflow<2.8,>=2.4)
  Using cached charset_normalizer-3.1.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (171 kB)
Collecting idna<4,>=2.5 (from requests<3,>=2.21.0->tensorboard~=2.6->tensorflow<2.8,>=2.4)
  Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17 (from requests<3,>=2.21.0->tensorboard~=2.6->tensorflow<2.8,>=2.4)
  Using cached certifi-2023.5.7-py3-none-any.whl (156 kB)
Collecting MarkupSafe>=2.1.1 (from werkzeug>=1.0.1->tensorboard~=2.6->tensorflow<2.8,>=2.4)
  Downloading MarkupSafe-2.1.3-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (25 kB)
Collecting zipp>=0.5 (from importlib-metadata>=4.4->markdown>=2.6.8->tensorboard~=2.6->tensorflow<2.8,>=2.4)
  Using cached zipp-3.15.0-py3-none-any.whl (6.8 kB)
Collecting pyasn1<0.6.0,>=0.4.6 (from pyasn1-modules>=0.2.1->google-auth<3,>=1.6.3->tensorboard~=2.6->tensorflow<2.8,>=2.4)
  Using cached pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
Collecting oauthlib>=3.0.0 (from requests-oauthlib>=0.7.0->google-auth-oauthlib<0.5,>=0.4.1->tensorboard~=2.6->tensorflow<2.8,>=2.4)
  Using cached oauthlib-3.2.2-py3-none-any.whl (151 kB)
Installing collected packages: tensorflow-estimator, tensorboard-plugin-wit, mpmath, libclang, keras, flatbuffers, zipp, wrapt, wheel, urllib3, typing-extensions, termcolor, tensorflow-io-gcs-filesystem, tensorboard-data-server, six, pyasn1, protobuf, pillow, oauthlib, numpy, MarkupSafe, idna, grpcio, gast, charset-normalizer, certifi, cachetools, absl-py, werkzeug, torch, rsa, requests, pyasn1-modules, opt-einsum, onnxruntime, onnx, keras-preprocessing, importlib-metadata, h5py, google-pasta, astunparse, torchvision, requests-oauthlib, markdown, google-auth, google-auth-oauthlib, tensorboard, tensorflow
Successfully installed MarkupSafe-2.1.3 absl-py-1.4.0 astunparse-1.6.3 cachetools-5.3.1 certifi-2023.5.7 charset-normalizer-3.1.0 flatbuffers-2.0.7 gast-0.4.0 google-auth-2.19.1 google-auth-oauthlib-0.4.6 google-pasta-0.2.0 grpcio-1.54.2 h5py-3.8.0 idna-3.4 importlib-metadata-6.6.0 keras-2.7.0 keras-preprocessing-1.1.2 libclang-16.0.0 markdown-3.4.3 mpmath-1.2.1 numpy-1.21.6 oauthlib-3.2.2 onnx-1.10.2 onnxruntime-1.10.0 opt-einsum-3.3.0 pillow-9.5.0 protobuf-3.19.6 pyasn1-0.5.0 pyasn1-modules-0.3.0 requests-2.31.0 requests-oauthlib-1.3.1 rsa-4.9 six-1.16.0 tensorboard-2.11.2 tensorboard-data-server-0.6.1 tensorboard-plugin-wit-1.8.1 tensorflow-2.7.4 tensorflow-estimator-2.7.0 tensorflow-io-gcs-filesystem-0.32.0 termcolor-2.3.0 torch-1.10.2 torchvision-0.11.3 typing-extensions-4.6.3 urllib3-1.26.16 werkzeug-2.2.3 wheel-0.40.0 wrapt-1.15.0 zipp-3.15.0
+ cd /home/ycs/.local/share/dnnv/opt/gurobi912/linux64
+ python setup.py install
running install
running build
running build_py
creating build
creating build/lib
creating build/lib/gurobipy
copying lib/python3.7_utf32/gurobipy/__init__.py -> build/lib/gurobipy
copying lib/python3.7_utf32/gurobipy/gurobipy.so -> build/lib/gurobipy
running install_lib
creating /home/ycs/.local/share/dnnv/verifier_virtualenvs/eran/lib/python3.7/site-packages/gurobipy
copying build/lib/gurobipy/gurobipy.so -> /home/ycs/.local/share/dnnv/verifier_virtualenvs/eran/lib/python3.7/site-packages/gurobipy
copying build/lib/gurobipy/__init__.py -> /home/ycs/.local/share/dnnv/verifier_virtualenvs/eran/lib/python3.7/site-packages/gurobipy
byte-compiling /home/ycs/.local/share/dnnv/verifier_virtualenvs/eran/lib/python3.7/site-packages/gurobipy/__init__.py to __init__.cpython-37.pyc
running install_egg_info
Writing /home/ycs/.local/share/dnnv/verifier_virtualenvs/eran/lib/python3.7/site-packages/gurobipy-9.1.2-py3.7.egg-info
removing /home/ycs/.local/share/dnnv/opt/gurobi912/linux64/build
+ cd /home/ycs/.cache/dnnv/eran-8771d3158b2c64a360d5bdfd4433490863257dd6
+ [ ! -e eran ]
+ cd eran
+ cp -r tf_verify/__main__.py tf_verify/ai_milp.py tf_verify/analyzer.py tf_verify/config.py tf_verify/constraint_utils.py tf_verify/deeppoly_nodes.py tf_verify/deepzono_nodes.py tf_verify/eran.py tf_verify/eranlayers.py tf_verify/krelu.py tf_verify/onnx_translator.py tf_verify/optimizer.py tf_verify/read_net_file.py tf_verify/read_zonotope_file.py tf_verify/refine_activation.py tf_verify/refine_gpupoly.py tf_verify/spatial tf_verify/tensorflow_translator.py tf_verify/utils.py /home/ycs/.local/share/dnnv/verifier_virtualenvs/eran/lib/python3.7/site-packages/
Traceback (most recent call last):
  File "/home/ycs/miniconda3/envs/dnnv/bin/dnnv_manage", line 8, in <module>
    sys.exit(_main())
  File "/home/ycs/miniconda3/envs/dnnv/lib/python3.7/site-packages/dnnv/_manage/__main__.py", line 95, in _main
    return args.command(**params)
  File "/home/ycs/miniconda3/envs/dnnv/lib/python3.7/site-packages/dnnv/_manage/linux/__init__.py", line 18, in install
    import_verifier_module(verifier).install(environment)
  File "/home/ycs/miniconda3/envs/dnnv/lib/python3.7/site-packages/dnnv/_manage/linux/verifiers/eran.py", line 332, in install
    ProgramDependency("grbgetkey", installer=gurobi_installer),
  File "/home/ycs/miniconda3/envs/dnnv/lib/python3.7/site-packages/dnnv/_manage/linux/environment/base.py", line 94, in ensure_dependencies
    self.ensure_dependency(dependency)
  File "/home/ycs/miniconda3/envs/dnnv/lib/python3.7/site-packages/dnnv/_manage/linux/environment/base.py", line 89, in ensure_dependency
    raise InstallError(f"Failed to install dependency: {dependency.name}")
dnnv._manage.errors.InstallError: Failed to install dependency: eran

I am using conda 4.12.0 on:

Distributor ID: Ubuntu
Description:    Ubuntu 22.04.1 LTS
Release:        22.04
Codename:       jammy

and my environment has the attached packages.dnnv_conda.txt

My CPU: 

 *-cpu
      product: 11th Gen Intel(R) Core(TM) i9-11950H @ 2.60GHz
      vendor: Intel Corp.
      physical id: 1
      bus info: cpu@0
      version: 6.141.1
      width: 64 bits

Python ==> python=3.7.16

Thanks again.

dlshriver commented 1 year ago

I think the issue is probably the use of conda. It uses its own virtual environment setup that is not compatible with the verifier installation scripts.