search

DIVA-DIA / DeepDIVA

⛔️ DEPRECATED <Python Framework for Reproducible Deep Learning Experiments>
https://diva-dia.github.io/DeepDIVAweb
GNU Lesser General Public License v3.0
32 stars 31 forks source link

Failing setup and mnist installation when following 'Getting started' #10

Closed guspih closed 5 years ago

guspih commented 5 years ago

Gets errors when running 'bash setup_environment.sh' which presumably causes further errors when attempting to Download mnist ('python util/data/get_a_dataset.py --dataset mnist --output-folder toy_dataset') when following the 'Getting started' instructions.

Printout from command line for Getting started process:

Collecting package metadata (repodata.json): done
Solving environment: done
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
Ran pip subprocess with arguments:
['/opt/Anaconda/envs/deepdiva/bin/python', '-m', 'pip', 'install', '-U', '-r', '/nfs/staff/gusgru/Repositories/DeepDIVA/condaenv.hygei685.requirements.txt']
Pip subprocess output:
Requirement already up-to-date: dask==1.2.2 in /opt/Anaconda/envs/deepdiva/lib/python3.7/site-packages (from -r /nfs/staff/gusgru/Repositories/DeepDIVA/condaenv.hygei685.requirements.txt (line 1)) (1.2.2)
Collecting pydensecrf==1.0rc2 (from -r /nfs/staff/gusgru/Repositories/DeepDIVA/condaenv.hygei685.requirements.txt (line 2))
  Using cached https://files.pythonhosted.org/packages/dd/11/7a79e817078323893f8071bb3fda1f45e34e443ec4a0790f8542158bbced/pydensecrf-1.0rc2.tar.gz
Collecting rarfile==3.0 (from -r /nfs/staff/gusgru/Repositories/DeepDIVA/condaenv.hygei685.requirements.txt (line 3))
Requirement already up-to-date: torch==1.0.1.post2 in /opt/Anaconda/envs/deepdiva/lib/python3.7/site-packages (from -r /nfs/staff/gusgru/Repositories/DeepDIVA/condaenv.hygei685.requirements.txt (line 4)) (1.0.1.post2)
Requirement already up-to-date: wget==3.2 in /opt/Anaconda/envs/deepdiva/lib/python3.7/site-packages (from -r /nfs/staff/gusgru/Repositories/DeepDIVA/condaenv.hygei685.requirements.txt (line 5)) (3.2)
Building wheels for collected packages: pydensecrf
  Building wheel for pydensecrf (setup.py): started
  Building wheel for pydensecrf (setup.py): finished with status 'error'
  Complete output from command /opt/Anaconda/envs/deepdiva/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-xor3fcig/pydensecrf/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/pip-wheel-w0gp68c6 --python-tag cp37:
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.linux-x86_64-3.7
  creating build/lib.linux-x86_64-3.7/pydensecrf
  copying pydensecrf/test.py -> build/lib.linux-x86_64-3.7/pydensecrf
  copying pydensecrf/utils.py -> build/lib.linux-x86_64-3.7/pydensecrf
  copying pydensecrf/__init__.py -> build/lib.linux-x86_64-3.7/pydensecrf
  copying pydensecrf/test_eigen.py -> build/lib.linux-x86_64-3.7/pydensecrf
  running build_ext
  building 'pydensecrf.eigen' extension
  creating build/temp.linux-x86_64-3.7
  creating build/temp.linux-x86_64-3.7/pydensecrf
  gcc -pthread -B /opt/Anaconda/envs/deepdiva/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -Ipydensecrf/densecrf/include -Ipydensecrf -I/opt/Anaconda/envs/deepdiva/include/python3.7m -c pydensecrf/eigen.cpp -o build/temp.linux-x86_64-3.7/pydensecrf/eigen.o
  cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
  In file included from pydensecrf/densecrf/include/Eigen/Core:248:0,
                   from pydensecrf/densecrf/include/Eigen/Dense:1,
                   from pydensecrf/eigen.cpp:299:
  pydensecrf/densecrf/include/Eigen/src/Core/util/Memory.h: In function ‘Index Eigen::internal::first_aligned(const Scalar*, Index)’:
  pydensecrf/densecrf/include/Eigen/src/Core/util/Memory.h:454:48: warning: typedef ‘Packet’ locally defined but not used [-Wunused-local-typedefs]
     typedef typename packet_traits<Scalar>::type Packet;
                                                  ^
  In file included from pydensecrf/densecrf/include/Eigen/Core:324:0,
                   from pydensecrf/densecrf/include/Eigen/Dense:1,
                   from pydensecrf/eigen.cpp:299:
  pydensecrf/densecrf/include/Eigen/src/Core/products/SelfadjointMatrixVector.h: In static member function ‘static void Eigen::internal::selfadjoint_matrix_vector_product<Scalar, Index, StorageOrder, UpLo, ConjugateLhs, ConjugateRhs, Version>::run(Index, const Scalar*, Index, const Scalar*, Index, Scalar*, Scalar)’:
  pydensecrf/densecrf/include/Eigen/src/Core/products/SelfadjointMatrixVector.h:38:44: warning: typedef ‘RealScalar’ locally defined but not used [-Wunused-local-typedefs]
     typedef typename NumTraits<Scalar>::Real RealScalar;
                                              ^
  In file included from pydensecrf/densecrf/include/Eigen/QR:26:0,
                   from pydensecrf/densecrf/include/Eigen/Dense:4,
                   from pydensecrf/eigen.cpp:299:
  pydensecrf/densecrf/include/Eigen/src/QR/HouseholderQR.h: In function ‘void Eigen::internal::householder_qr_inplace_blocked(MatrixQR&, HCoeffs&, typename MatrixQR::Index, typename MatrixQR::Scalar*)’:
  pydensecrf/densecrf/include/Eigen/src/QR/HouseholderQR.h:235:41: warning: typedef ‘RealScalar’ locally defined but not used [-Wunused-local-typedefs]
     typedef typename MatrixQR::RealScalar RealScalar;
                                           ^
  In file included from pydensecrf/densecrf/include/Eigen/Eigenvalues:28:0,
                   from pydensecrf/densecrf/include/Eigen/Dense:7,
                   from pydensecrf/eigen.cpp:299:
  pydensecrf/densecrf/include/Eigen/src/Eigenvalues/Tridiagonalization.h: In function ‘void Eigen::internal::tridiagonalization_inplace(MatrixType&, DiagonalType&, SubDiagonalType&, bool)’:
  pydensecrf/densecrf/include/Eigen/src/Eigenvalues/Tridiagonalization.h:428:38: warning: typedef ‘Index’ locally defined but not used [-Wunused-local-typedefs]
     typedef typename MatrixType::Index Index;
                                        ^
  In file included from pydensecrf/densecrf/include/Eigen/Eigenvalues:34:0,
                   from pydensecrf/densecrf/include/Eigen/Dense:7,
                   from pydensecrf/eigen.cpp:299:
  pydensecrf/densecrf/include/Eigen/src/Eigenvalues/ComplexSchur.h: In static member function ‘static void Eigen::internal::complex_schur_reduce_to_hessenberg<MatrixType, false>::run(Eigen::ComplexSchur<MatrixType>&, const MatrixType&, bool)’:
  pydensecrf/densecrf/include/Eigen/src/Eigenvalues/ComplexSchur.h:312:66: warning: typedef ‘ComplexMatrixType’ locally defined but not used [-Wunused-local-typedefs]
       typedef typename ComplexSchur<MatrixType>::ComplexMatrixType ComplexMatrixType;
                                                                    ^
  pydensecrf/eigen.cpp: In function ‘void __Pyx__ExceptionSave(PyThreadState*, PyObject**, PyObject**, PyObject**)’:
  pydensecrf/eigen.cpp:17032:21: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_type’
       *type = tstate->exc_type;
                       ^
  pydensecrf/eigen.cpp:17033:22: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_value’
       *value = tstate->exc_value;
                        ^
  pydensecrf/eigen.cpp:17034:19: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_traceback’
       *tb = tstate->exc_traceback;
                     ^
  pydensecrf/eigen.cpp: In function ‘void __Pyx__ExceptionReset(PyThreadState*, PyObject*, PyObject*, PyObject*)’:
  pydensecrf/eigen.cpp:17041:24: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_type’
       tmp_type = tstate->exc_type;
                          ^
  pydensecrf/eigen.cpp:17042:25: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_value’
       tmp_value = tstate->exc_value;
                           ^
  pydensecrf/eigen.cpp:17043:22: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_traceback’
       tmp_tb = tstate->exc_traceback;
                        ^
  pydensecrf/eigen.cpp:17044:13: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_type’
       tstate->exc_type = type;
               ^
  pydensecrf/eigen.cpp:17045:13: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_value’
       tstate->exc_value = value;
               ^
  pydensecrf/eigen.cpp:17046:13: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_traceback’
       tstate->exc_traceback = tb;
               ^
  pydensecrf/eigen.cpp: In function ‘int __Pyx__GetException(PyThreadState*, PyObject**, PyObject**, PyObject**)’:
  pydensecrf/eigen.cpp:17101:24: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_type’
       tmp_type = tstate->exc_type;
                          ^
  pydensecrf/eigen.cpp:17102:25: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_value’
       tmp_value = tstate->exc_value;
                           ^
  pydensecrf/eigen.cpp:17103:22: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_traceback’
       tmp_tb = tstate->exc_traceback;
                        ^
  pydensecrf/eigen.cpp:17104:13: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_type’
       tstate->exc_type = local_type;
               ^
  pydensecrf/eigen.cpp:17105:13: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_value’
       tstate->exc_value = local_value;
               ^
  pydensecrf/eigen.cpp:17106:13: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_traceback’
       tstate->exc_traceback = local_tb;
               ^
  pydensecrf/eigen.cpp: In function ‘void __Pyx__ExceptionSwap(PyThreadState*, PyObject**, PyObject**, PyObject**)’:
  pydensecrf/eigen.cpp:17128:24: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_type’
       tmp_type = tstate->exc_type;
                          ^
  pydensecrf/eigen.cpp:17129:25: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_value’
       tmp_value = tstate->exc_value;
                           ^
  pydensecrf/eigen.cpp:17130:22: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_traceback’
       tmp_tb = tstate->exc_traceback;
                        ^
  pydensecrf/eigen.cpp:17131:13: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_type’
       tstate->exc_type = *type;
               ^
  pydensecrf/eigen.cpp:17132:13: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_value’
       tstate->exc_value = *value;
               ^
  pydensecrf/eigen.cpp:17133:13: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_traceback’
       tstate->exc_traceback = *tb;
               ^
  In file included from pydensecrf/eigen.cpp:300:0:
  pydensecrf/eigen_impl.cpp: At global scope:
  pydensecrf/eigen_impl.cpp:13:13: warning: ‘void vecf2buf(const VectorXf&, float*)’ defined but not used [-Wunused-function]
   static void vecf2buf(const Eigen::VectorXf& vec, float *mem)
               ^
  pydensecrf/eigen_impl.cpp:27:13: warning: ‘void matf2buf(const MatrixXf&, float*)’ defined but not used [-Wunused-function]
   static void matf2buf(const Eigen::MatrixXf& mat, float *mem)
               ^
  error: command 'gcc' failed with exit status 1

  ----------------------------------------
  Running setup.py clean for pydensecrf
Failed to build pydensecrf
Installing collected packages: pydensecrf, rarfile
  Running setup.py install for pydensecrf: started
    Running setup.py install for pydensecrf: finished with status 'error'
    Complete output from command /opt/Anaconda/envs/deepdiva/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-xor3fcig/pydensecrf/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-6a_kt9ca/install-record.txt --single-version-externally-managed --compile:
    running install
    running build
    running build_py
    creating build
    creating build/lib.linux-x86_64-3.7
    creating build/lib.linux-x86_64-3.7/pydensecrf
    copying pydensecrf/test.py -> build/lib.linux-x86_64-3.7/pydensecrf
    copying pydensecrf/utils.py -> build/lib.linux-x86_64-3.7/pydensecrf
    copying pydensecrf/__init__.py -> build/lib.linux-x86_64-3.7/pydensecrf
    copying pydensecrf/test_eigen.py -> build/lib.linux-x86_64-3.7/pydensecrf
    running build_ext
    building 'pydensecrf.eigen' extension
    creating build/temp.linux-x86_64-3.7
    creating build/temp.linux-x86_64-3.7/pydensecrf
    gcc -pthread -B /opt/Anaconda/envs/deepdiva/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -Ipydensecrf/densecrf/include -Ipydensecrf -I/opt/Anaconda/envs/deepdiva/include/python3.7m -c pydensecrf/eigen.cpp -o build/temp.linux-x86_64-3.7/pydensecrf/eigen.o
    cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
    In file included from pydensecrf/densecrf/include/Eigen/Core:248:0,
                     from pydensecrf/densecrf/include/Eigen/Dense:1,
                     from pydensecrf/eigen.cpp:299:
    pydensecrf/densecrf/include/Eigen/src/Core/util/Memory.h: In function ‘Index Eigen::internal::first_aligned(const Scalar*, Index)’:
    pydensecrf/densecrf/include/Eigen/src/Core/util/Memory.h:454:48: warning: typedef ‘Packet’ locally defined but not used [-Wunused-local-typedefs]
       typedef typename packet_traits<Scalar>::type Packet;
                                                    ^
    In file included from pydensecrf/densecrf/include/Eigen/Core:324:0,
                     from pydensecrf/densecrf/include/Eigen/Dense:1,
                     from pydensecrf/eigen.cpp:299:
    pydensecrf/densecrf/include/Eigen/src/Core/products/SelfadjointMatrixVector.h: In static member function ‘static void Eigen::internal::selfadjoint_matrix_vector_product<Scalar, Index, StorageOrder, UpLo, ConjugateLhs, ConjugateRhs, Version>::run(Index, const Scalar*, Index, const Scalar*, Index, Scalar*, Scalar)’:
    pydensecrf/densecrf/include/Eigen/src/Core/products/SelfadjointMatrixVector.h:38:44: warning: typedef ‘RealScalar’ locally defined but not used [-Wunused-local-typedefs]
       typedef typename NumTraits<Scalar>::Real RealScalar;
                                                ^
    In file included from pydensecrf/densecrf/include/Eigen/QR:26:0,
                     from pydensecrf/densecrf/include/Eigen/Dense:4,
                     from pydensecrf/eigen.cpp:299:
    pydensecrf/densecrf/include/Eigen/src/QR/HouseholderQR.h: In function ‘void Eigen::internal::householder_qr_inplace_blocked(MatrixQR&, HCoeffs&, typename MatrixQR::Index, typename MatrixQR::Scalar*)’:
    pydensecrf/densecrf/include/Eigen/src/QR/HouseholderQR.h:235:41: warning: typedef ‘RealScalar’ locally defined but not used [-Wunused-local-typedefs]
       typedef typename MatrixQR::RealScalar RealScalar;
                                             ^
    In file included from pydensecrf/densecrf/include/Eigen/Eigenvalues:28:0,
                     from pydensecrf/densecrf/include/Eigen/Dense:7,
                     from pydensecrf/eigen.cpp:299:
    pydensecrf/densecrf/include/Eigen/src/Eigenvalues/Tridiagonalization.h: In function ‘void Eigen::internal::tridiagonalization_inplace(MatrixType&, DiagonalType&, SubDiagonalType&, bool)’:
    pydensecrf/densecrf/include/Eigen/src/Eigenvalues/Tridiagonalization.h:428:38: warning: typedef ‘Index’ locally defined but not used [-Wunused-local-typedefs]
       typedef typename MatrixType::Index Index;
                                          ^
    In file included from pydensecrf/densecrf/include/Eigen/Eigenvalues:34:0,
                     from pydensecrf/densecrf/include/Eigen/Dense:7,
                     from pydensecrf/eigen.cpp:299:
    pydensecrf/densecrf/include/Eigen/src/Eigenvalues/ComplexSchur.h: In static member function ‘static void Eigen::internal::complex_schur_reduce_to_hessenberg<MatrixType, false>::run(Eigen::ComplexSchur<MatrixType>&, const MatrixType&, bool)’:
    pydensecrf/densecrf/include/Eigen/src/Eigenvalues/ComplexSchur.h:312:66: warning: typedef ‘ComplexMatrixType’ locally defined but not used [-Wunused-local-typedefs]
         typedef typename ComplexSchur<MatrixType>::ComplexMatrixType ComplexMatrixType;
                                                                      ^
    pydensecrf/eigen.cpp: In function ‘void __Pyx__ExceptionSave(PyThreadState*, PyObject**, PyObject**, PyObject**)’:
    pydensecrf/eigen.cpp:17032:21: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_type’
         *type = tstate->exc_type;
                         ^
    pydensecrf/eigen.cpp:17033:22: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_value’
         *value = tstate->exc_value;
                          ^
    pydensecrf/eigen.cpp:17034:19: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_traceback’
         *tb = tstate->exc_traceback;
                       ^
    pydensecrf/eigen.cpp: In function ‘void __Pyx__ExceptionReset(PyThreadState*, PyObject*, PyObject*, PyObject*)’:
    pydensecrf/eigen.cpp:17041:24: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_type’
         tmp_type = tstate->exc_type;
                            ^
    pydensecrf/eigen.cpp:17042:25: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_value’
         tmp_value = tstate->exc_value;
                             ^
    pydensecrf/eigen.cpp:17043:22: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_traceback’
         tmp_tb = tstate->exc_traceback;
                          ^
    pydensecrf/eigen.cpp:17044:13: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_type’
         tstate->exc_type = type;
                 ^
    pydensecrf/eigen.cpp:17045:13: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_value’
         tstate->exc_value = value;
                 ^
    pydensecrf/eigen.cpp:17046:13: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_traceback’
         tstate->exc_traceback = tb;
                 ^
    pydensecrf/eigen.cpp: In function ‘int __Pyx__GetException(PyThreadState*, PyObject**, PyObject**, PyObject**)’:
    pydensecrf/eigen.cpp:17101:24: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_type’
         tmp_type = tstate->exc_type;
                            ^
    pydensecrf/eigen.cpp:17102:25: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_value’
         tmp_value = tstate->exc_value;
                             ^
    pydensecrf/eigen.cpp:17103:22: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_traceback’
         tmp_tb = tstate->exc_traceback;
                          ^
    pydensecrf/eigen.cpp:17104:13: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_type’
         tstate->exc_type = local_type;
Setup completed!
Please run 'source ~/.bashrc' to refresh your environment
You can activate the deepdiva environment with 'source activate deepdiva'
(base) gusgru@B4375:~/Repositories/DeepDIVA$ source ~/.bashrc
(base) gusgru@B4375:~/Repositories/DeepDIVA$ source activate deepdiva
(deepdiva) gusgru@B4375:~/Repositories/DeepDIVA$ python util/data/get_a_dataset.py --dataset mnist --output-folder toy_dataset
Traceback (most recent call last):
  File "util/data/get_a_dataset.py", line 19, in <module>
    import rarfile
ModuleNotFoundError: No module named 'rarfile'
LindaSt commented 5 years ago

We will look into it. In the meanwhile, you can run pip install rarfile (and any other missing packages) to install missing packages and still use DeepDIVA.

Renthal commented 5 years ago

I merged the PR can you see if the problem persists?

guspih commented 5 years ago

The setup works now. Thanks