search

bjodah / pycvodes

Python wrapper around cvodes (from the sundials library)
BSD 2-Clause "Simplified" License
35 stars 5 forks source link

pip compilation issues #131

Open jeremyagray opened 3 years ago

jeremyagray commented 3 years ago

I keep having a multiple definition error of several functions when using sundials with lapack and pycvodes trying to use headers in anyode. I created a branch to illustrate how I finally got it to compile and install (no errors/warnings) by passing in the appropriate enviroment variable (PYCVODES_USE_LAPACK='1'), but I wonder if there is a problem elsewhere with one of the libraries or something else. I didn't dive deeply enough into the source to see if this turned off all lapack use, but the tests in pycvodes/tests passed. I was intending to work on or at least document pip installation of chempy and got hung on pycvodes.

Hopefully all the relevant setup details are below; errors follow.

versions (current debian/bullseye):

requirements.txt:

Build Errors: python setup.py bdist_wheel

Compiling pycvodes/_cvodes.pyx because it changed.
[1/1] Cythonizing pycvodes/_cvodes.pyx
running bdist_wheel
running build
running build_py
running egg_info
writing pycvodes.egg-info/PKG-INFO
writing dependency_links to pycvodes.egg-info/dependency_links.txt
writing requirements to pycvodes.egg-info/requires.txt
writing top-level names to pycvodes.egg-info/top_level.txt
reading manifest file 'pycvodes.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'pycvodes.egg-info/SOURCES.txt'
copying pycvodes/_cvodes.cpp -> build/lib.linux-x86_64-3.8/pycvodes
running build_ext
building 'pycvodes._cvodes' extension
x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DPYCVODES_NO_KLU=0 -DPYCVODES_NO_LAPACK=0 -DANYODE_NO_LAPACK=0 -I/home/gray/.virtualenvs/pycvodes/lib/python3.8/site-packages/numpy/core/include -Ipycvodes/include -Iexternal/anyode/include -I/home/gray/.virtualenvs/pycvodes/include -I/usr/include/python3.8 -c pycvodes/_cvodes.cpp -o build/temp.linux-x86_64-3.8/pycvodes/_cvodes.o -DVERSION_INFO="0.13.1" --std=c++11
In file included from /home/gray/.virtualenvs/pycvodes/lib/python3.8/site-packages/numpy/core/include/numpy/ndarraytypes.h:1822,
                 from /home/gray/.virtualenvs/pycvodes/lib/python3.8/site-packages/numpy/core/include/numpy/ndarrayobject.h:12,
                 from /home/gray/.virtualenvs/pycvodes/lib/python3.8/site-packages/numpy/core/include/numpy/arrayobject.h:4,
                 from pycvodes/_cvodes.cpp:667:
/home/gray/.virtualenvs/pycvodes/lib/python3.8/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:17:2: warning: #warning "Using deprecated NumPy API, disable it with " "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
   17 | #warning "Using deprecated NumPy API, disable it with " \
      |  ^~~~~~~
In file included from pycvodes/include/sundials_cxx.hpp:4,
                 from pycvodes/include/cvodes_cxx.hpp:37,
                 from pycvodes/_cvodes.cpp:669:
/home/gray/.virtualenvs/chempy/include/sundials/sundials_lapack.h:38:43: error: conflicting declaration of C function ‘void dgemv_(const char*, sunindextype*, sunindextype*, const double*, const double*, sunindextype*, const double*, sunindextype*, const double*, double*, sunindextype*)’
   38 | #define dgemv_f77       SUNDIALS_F77_FUNC(dgemv, DGEMV)
      |                                           ^~~~~
/home/gray/.virtualenvs/chempy/include/sundials/sundials_lapack.h:38:25: note: in expansion of macro ‘SUNDIALS_F77_FUNC’
   38 | #define dgemv_f77       SUNDIALS_F77_FUNC(dgemv, DGEMV)
      |                         ^~~~~~~~~~~~~~~~~
/home/gray/.virtualenvs/chempy/include/sundials/sundials_lapack.h:122:13: note: in expansion of macro ‘dgemv_f77’
  122 | extern void dgemv_f77(const char *trans, sunindextype *m, sunindextype *n,
      |             ^~~~~~~~~
In file included from external/anyode/include/anyode/anyode_matrix.hpp:14,
                 from external/anyode/include/anyode/anyode_iterative.hpp:7,
                 from external/anyode/include/anyode/anyode_numpy.hpp:5,
                 from pycvodes/_cvodes.cpp:668:
external/anyode/include/anyode/anyode_blas_lapack.hpp:2:17: note: previous declaration ‘void dgemv_(const char*, int*, int*, const double*, const double*, int*, const double*, int*, const double*, double*, int*)’
    2 | extern "C" void dgemv_(const char* trans, int* m, int* n, const double* alpha, const double* a, int* lda,
      |                 ^~~~~~
In file included from pycvodes/include/sundials_cxx.hpp:4,
                 from pycvodes/include/cvodes_cxx.hpp:37,
                 from pycvodes/_cvodes.cpp:669:
/home/gray/.virtualenvs/chempy/include/sundials/sundials_lapack.h:54:43: error: conflicting declaration of C function ‘void sgemv_(const char*, sunindextype*, sunindextype*, const float*, const float*, sunindextype*, const float*, sunindextype*, const float*, float*, sunindextype*)’
   54 | #define sgemv_f77       SUNDIALS_F77_FUNC(sgemv, SGEMV)
      |                                           ^~~~~
/home/gray/.virtualenvs/chempy/include/sundials/sundials_config.h:38:38: note: in definition of macro ‘SUNDIALS_F77_FUNC’
   38 | #define SUNDIALS_F77_FUNC(name,NAME) name ## _
      |                                      ^~~~
/home/gray/.virtualenvs/chempy/include/sundials/sundials_lapack.h:132:13: note: in expansion of macro ‘sgemv_f77’
  132 | extern void sgemv_f77(const char *trans, sunindextype *m, sunindextype *n,
      |             ^~~~~~~~~
In file included from external/anyode/include/anyode/anyode_matrix.hpp:14,
                 from external/anyode/include/anyode/anyode_iterative.hpp:7,
                 from external/anyode/include/anyode/anyode_numpy.hpp:5,
                 from pycvodes/_cvodes.cpp:668:
external/anyode/include/anyode/anyode_blas_lapack.hpp:4:17: note: previous declaration ‘void sgemv_(const char*, int*, int*, const float*, const float*, int*, const float*, int*, const float*, float*, int*)’
    4 | extern "C" void sgemv_(const char* trans, int* m, int* n, const float* alpha, const float* a, int* lda,
      |                 ^~~~~~
In file included from pycvodes/include/sundials_cxx.hpp:4,
                 from pycvodes/include/cvodes_cxx.hpp:37,
                 from pycvodes/_cvodes.cpp:669:
/home/gray/.virtualenvs/chempy/include/sundials/sundials_lapack.h:42:43: error: conflicting declaration of C function ‘void dgbtrf_(const sunindextype*, const sunindextype*, const sunindextype*, const sunindextype*, double*, sunindextype*, sunindextype*, sunindextype*)’
   42 | #define dgbtrf_f77      SUNDIALS_F77_FUNC(dgbtrf, DGBTRF)
      |                                           ^~~~~~
/home/gray/.virtualenvs/chempy/include/sundials/sundials_config.h:38:38: note: in definition of macro ‘SUNDIALS_F77_FUNC’
   38 | #define SUNDIALS_F77_FUNC(name,NAME) name ## _
      |                                      ^~~~
/home/gray/.virtualenvs/chempy/include/sundials/sundials_lapack.h:158:13: note: in expansion of macro ‘dgbtrf_f77’
  158 | extern void dgbtrf_f77(const sunindextype *m, const sunindextype *n,
      |             ^~~~~~~~~~
In file included from external/anyode/include/anyode/anyode_matrix.hpp:14,
                 from external/anyode/include/anyode/anyode_iterative.hpp:7,
                 from external/anyode/include/anyode/anyode_numpy.hpp:5,
                 from pycvodes/_cvodes.cpp:668:
external/anyode/include/anyode/anyode_blas_lapack.hpp:25:17: note: previous declaration ‘void dgbtrf_(const int*, const int*, const int*, const int*, double*, int*, int*, int*)’
   25 | extern "C" void dgbtrf_(const int* dim1, const int* dim2, const int* kl, const int* ku, double* a, int* lda, int* ipiv, int* info);
      |                 ^~~~~~~
In file included from pycvodes/include/sundials_cxx.hpp:4,
                 from pycvodes/include/cvodes_cxx.hpp:37,
                 from pycvodes/_cvodes.cpp:669:
/home/gray/.virtualenvs/chempy/include/sundials/sundials_lapack.h:43:43: error: conflicting declaration of C function ‘void dgbtrs_(const char*, const sunindextype*, const sunindextype*, const sunindextype*, const sunindextype*, double*, const sunindextype*, sunindextype*, double*, const sunindextype*, sunindextype*)’
   43 | #define dgbtrs_f77      SUNDIALS_F77_FUNC(dgbtrs, DGBTRS)
      |                                           ^~~~~~
/home/gray/.virtualenvs/chempy/include/sundials/sundials_config.h:38:38: note: in definition of macro ‘SUNDIALS_F77_FUNC’
   38 | #define SUNDIALS_F77_FUNC(name,NAME) name ## _
      |                                      ^~~~
/home/gray/.virtualenvs/chempy/include/sundials/sundials_lapack.h:163:13: note: in expansion of macro ‘dgbtrs_f77’
  163 | extern void dgbtrs_f77(const char *trans, const sunindextype *n,
      |             ^~~~~~~~~~
In file included from external/anyode/include/anyode/anyode_matrix.hpp:14,
                 from external/anyode/include/anyode/anyode_iterative.hpp:7,
                 from external/anyode/include/anyode/anyode_numpy.hpp:5,
                 from pycvodes/_cvodes.cpp:668:
external/anyode/include/anyode/anyode_blas_lapack.hpp:28:17: note: previous declaration ‘void dgbtrs_(const char*, const int*, const int*, const int*, const int*, double*, const int*, int*, double*, const int*, int*)’
   28 | extern "C" void dgbtrs_(const char* trans, const int* n, const int* kl, const int* ku, const int* nrhs, double* a,
      |                 ^~~~~~~
In file included from pycvodes/include/sundials_cxx.hpp:4,
                 from pycvodes/include/cvodes_cxx.hpp:37,
                 from pycvodes/_cvodes.cpp:669:
/home/gray/.virtualenvs/chempy/include/sundials/sundials_lapack.h:44:43: error: conflicting declaration of C function ‘void dgetrf_(const sunindextype*, const sunindextype*, double*, sunindextype*, sunindextype*, sunindextype*)’
   44 | #define dgetrf_f77      SUNDIALS_F77_FUNC(dgetrf, DGETRF)
      |                                           ^~~~~~
/home/gray/.virtualenvs/chempy/include/sundials/sundials_config.h:38:38: note: in definition of macro ‘SUNDIALS_F77_FUNC’
   38 | #define SUNDIALS_F77_FUNC(name,NAME) name ## _
      |                                      ^~~~
/home/gray/.virtualenvs/chempy/include/sundials/sundials_lapack.h:179:13: note: in expansion of macro ‘dgetrf_f77’
  179 | extern void dgetrf_f77(const sunindextype *m, const sunindextype *n, double *a,
      |             ^~~~~~~~~~
In file included from external/anyode/include/anyode/anyode_matrix.hpp:14,
                 from external/anyode/include/anyode/anyode_iterative.hpp:7,
                 from external/anyode/include/anyode/anyode_numpy.hpp:5,
                 from pycvodes/_cvodes.cpp:668:
external/anyode/include/anyode/anyode_blas_lapack.hpp:14:17: note: previous declaration ‘void dgetrf_(const int*, const int*, double*, int*, int*, int*)’
   14 | extern "C" void dgetrf_(const int* dim1, const int* dim2, double* a, int* lda, int* ipiv, int* info);
      |                 ^~~~~~~
In file included from pycvodes/include/sundials_cxx.hpp:4,
                 from pycvodes/include/cvodes_cxx.hpp:37,
                 from pycvodes/_cvodes.cpp:669:
/home/gray/.virtualenvs/chempy/include/sundials/sundials_lapack.h:45:43: error: conflicting declaration of C function ‘void dgetrs_(const char*, const sunindextype*, const sunindextype*, double*, const sunindextype*, sunindextype*, double*, const sunindextype*, sunindextype*)’
   45 | #define dgetrs_f77      SUNDIALS_F77_FUNC(dgetrs, DGETRS)
      |                                           ^~~~~~
/home/gray/.virtualenvs/chempy/include/sundials/sundials_config.h:38:38: note: in definition of macro ‘SUNDIALS_F77_FUNC’
   38 | #define SUNDIALS_F77_FUNC(name,NAME) name ## _
      |                                      ^~~~
/home/gray/.virtualenvs/chempy/include/sundials/sundials_lapack.h:183:13: note: in expansion of macro ‘dgetrs_f77’
  183 | extern void dgetrs_f77(const char *trans, const sunindextype *n,
      |             ^~~~~~~~~~
In file included from external/anyode/include/anyode/anyode_matrix.hpp:14,
                 from external/anyode/include/anyode/anyode_iterative.hpp:7,
                 from external/anyode/include/anyode/anyode_numpy.hpp:5,
                 from pycvodes/_cvodes.cpp:668:
external/anyode/include/anyode/anyode_blas_lapack.hpp:17:17: note: previous declaration ‘void dgetrs_(const char*, const int*, const int*, double*, const int*, int*, double*, const int*, int*)’
   17 | extern "C" void dgetrs_(const char* trans, const int* n, const int* nrhs, double* a, const int* lda, int* ipiv, double * b, const int* ldb, int* info);
      |                 ^~~~~~~
In file included from pycvodes/include/sundials_cxx.hpp:4,
                 from pycvodes/include/cvodes_cxx.hpp:37,
                 from pycvodes/_cvodes.cpp:669:
/home/gray/.virtualenvs/chempy/include/sundials/sundials_lapack.h:58:43: error: conflicting declaration of C function ‘void sgbtrf_(const sunindextype*, const sunindextype*, const sunindextype*, const sunindextype*, float*, sunindextype*, sunindextype*, sunindextype*)’
   58 | #define sgbtrf_f77      SUNDIALS_F77_FUNC(sgbtrf, SGBTRF)
      |                                           ^~~~~~
/home/gray/.virtualenvs/chempy/include/sundials/sundials_config.h:38:38: note: in definition of macro ‘SUNDIALS_F77_FUNC’
   38 | #define SUNDIALS_F77_FUNC(name,NAME) name ## _
      |                                      ^~~~
/home/gray/.virtualenvs/chempy/include/sundials/sundials_lapack.h:205:13: note: in expansion of macro ‘sgbtrf_f77’
  205 | extern void sgbtrf_f77(const sunindextype *m, const sunindextype *n,
      |             ^~~~~~~~~~
In file included from external/anyode/include/anyode/anyode_matrix.hpp:14,
                 from external/anyode/include/anyode/anyode_iterative.hpp:7,
                 from external/anyode/include/anyode/anyode_numpy.hpp:5,
                 from pycvodes/_cvodes.cpp:668:
external/anyode/include/anyode/anyode_blas_lapack.hpp:26:17: note: previous declaration ‘void sgbtrf_(const int*, const int*, const int*, const int*, float*, int*, int*, int*)’
   26 | extern "C" void sgbtrf_(const int* dim1, const int* dim2, const int* kl, const int* ku, float* a, int* lda, int* ipiv, int* info);
      |                 ^~~~~~~
In file included from pycvodes/include/sundials_cxx.hpp:4,
                 from pycvodes/include/cvodes_cxx.hpp:37,
                 from pycvodes/_cvodes.cpp:669:
/home/gray/.virtualenvs/chempy/include/sundials/sundials_lapack.h:59:43: error: conflicting declaration of C function ‘void sgbtrs_(const char*, const sunindextype*, const sunindextype*, const sunindextype*, const sunindextype*, float*, const sunindextype*, sunindextype*, float*, const sunindextype*, sunindextype*)’
   59 | #define sgbtrs_f77      SUNDIALS_F77_FUNC(sgbtrs, SGBTRS)
      |                                           ^~~~~~
/home/gray/.virtualenvs/chempy/include/sundials/sundials_config.h:38:38: note: in definition of macro ‘SUNDIALS_F77_FUNC’
   38 | #define SUNDIALS_F77_FUNC(name,NAME) name ## _
      |                                      ^~~~
/home/gray/.virtualenvs/chempy/include/sundials/sundials_lapack.h:210:13: note: in expansion of macro ‘sgbtrs_f77’
  210 | extern void sgbtrs_f77(const char *trans, const sunindextype *n,
      |             ^~~~~~~~~~
In file included from external/anyode/include/anyode/anyode_matrix.hpp:14,
                 from external/anyode/include/anyode/anyode_iterative.hpp:7,
                 from external/anyode/include/anyode/anyode_numpy.hpp:5,
                 from pycvodes/_cvodes.cpp:668:
external/anyode/include/anyode/anyode_blas_lapack.hpp:30:17: note: previous declaration ‘void sgbtrs_(const char*, const int*, const int*, const int*, const int*, float*, const int*, int*, float*, const int*, int*)’
   30 | extern "C" void sgbtrs_(const char* trans, const int* n, const int* kl, const int* ku, const int* nrhs, float* a,
      |                 ^~~~~~~
In file included from pycvodes/include/sundials_cxx.hpp:4,
                 from pycvodes/include/cvodes_cxx.hpp:37,
                 from pycvodes/_cvodes.cpp:669:
/home/gray/.virtualenvs/chempy/include/sundials/sundials_lapack.h:60:43: error: conflicting declaration of C function ‘void sgetrf_(const sunindextype*, const sunindextype*, float*, sunindextype*, sunindextype*, sunindextype*)’
   60 | #define sgetrf_f77      SUNDIALS_F77_FUNC(sgetrf, SGETRF)
      |                                           ^~~~~~
/home/gray/.virtualenvs/chempy/include/sundials/sundials_config.h:38:38: note: in definition of macro ‘SUNDIALS_F77_FUNC’
   38 | #define SUNDIALS_F77_FUNC(name,NAME) name ## _
      |                                      ^~~~
/home/gray/.virtualenvs/chempy/include/sundials/sundials_lapack.h:226:13: note: in expansion of macro ‘sgetrf_f77’
  226 | extern void sgetrf_f77(const sunindextype *m, const sunindextype *n, float *a,
      |             ^~~~~~~~~~
In file included from external/anyode/include/anyode/anyode_matrix.hpp:14,
                 from external/anyode/include/anyode/anyode_iterative.hpp:7,
                 from external/anyode/include/anyode/anyode_numpy.hpp:5,
                 from pycvodes/_cvodes.cpp:668:
external/anyode/include/anyode/anyode_blas_lapack.hpp:15:17: note: previous declaration ‘void sgetrf_(const int*, const int*, float*, int*, int*, int*)’
   15 | extern "C" void sgetrf_(const int* dim1, const int* dim2, float* a, int* lda, int* ipiv, int* info);
      |                 ^~~~~~~
In file included from pycvodes/include/sundials_cxx.hpp:4,
                 from pycvodes/include/cvodes_cxx.hpp:37,
                 from pycvodes/_cvodes.cpp:669:
/home/gray/.virtualenvs/chempy/include/sundials/sundials_lapack.h:61:43: error: conflicting declaration of C function ‘void sgetrs_(const char*, const sunindextype*, const sunindextype*, float*, const sunindextype*, sunindextype*, float*, const sunindextype*, sunindextype*)’
   61 | #define sgetrs_f77      SUNDIALS_F77_FUNC(sgetrs, SGETRS)
      |                                           ^~~~~~
/home/gray/.virtualenvs/chempy/include/sundials/sundials_config.h:38:38: note: in definition of macro ‘SUNDIALS_F77_FUNC’
   38 | #define SUNDIALS_F77_FUNC(name,NAME) name ## _
      |                                      ^~~~
/home/gray/.virtualenvs/chempy/include/sundials/sundials_lapack.h:230:13: note: in expansion of macro ‘sgetrs_f77’
  230 | extern void sgetrs_f77(const char *trans, const sunindextype *n,
      |             ^~~~~~~~~~
In file included from external/anyode/include/anyode/anyode_matrix.hpp:14,
                 from external/anyode/include/anyode/anyode_iterative.hpp:7,
                 from external/anyode/include/anyode/anyode_numpy.hpp:5,
                 from pycvodes/_cvodes.cpp:668:
external/anyode/include/anyode/anyode_blas_lapack.hpp:18:17: note: previous declaration ‘void sgetrs_(const char*, const int*, const int*, float*, const int*, int*, float*, const int*, int*)’
   18 | extern "C" void sgetrs_(const char* trans, const int* n, const int* nrhs, float* a, const int* lda, int* ipiv, float * b, const int* ldb, int* info);
      |                 ^~~~~~~
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
bjodah commented 3 years ago

It's a mess, I know--believe me. Thank you for trying to document this.

Some things that come to mind here:

bjodah commented 3 years ago

What if you insert this line:

#ifndef _SUNDIALS_LAPACK_H

above this one? https://github.com/bjodah/anyode/blob/ce9db8c91adf036657219a3cdf3c251483a3f0f7/include/anyode/anyode_blas_lapack.hpp#L2

(and insert a #endif after the relevant extern declarations?)

jeremyagray commented 3 years ago

I still get the same errors (no new ones). Compilations fails with PYCVODES_USE_LAPACK='0' or unset and compiles with it set to '1'.

On Sat, Oct 31, 2020 at 6:09 AM Bjorn notifications@github.com wrote:

What if you insert this line:

ifndef _SUNDIALS_LAPACK_H

above this one?

https://github.com/bjodah/anyode/blob/ce9db8c91adf036657219a3cdf3c251483a3f0f7/include/anyode/anyode_blas_lapack.hpp#L2

(and insert a #endif after the relevant extern declarations?)

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/bjodah/pycvodes/issues/131#issuecomment-719918956, or unsubscribe https://github.com/notifications/unsubscribe-auth/AOQCHS332HKUD5FPHY5UNS3SNPV7FANCNFSM4TEID3FQ .

-- Jeremy A. Gray

bjodah commented 3 years ago

sorry, I don't quite follow, did you export environment variables on the command line prior to installing?, i.e.:

export PYCVODES_NO_LAPACK=1 PYCVODES_NO_KLU=1
python3 -m pip install .

regarding requirements.txt, I think it's deprecated since 5 years or so? setuptools & requires arguments are preferred I believe? (way back, I used to maintain a requirements.txt in the repository even...)

jeremyagray commented 3 years ago

I added the #ifndef/#endif, then cleared the environment so that PYCVODES_NO_LAPACK was unset so that it would build as normal with python setup.py build in the pycvodes top directory, and I got the same multiple definition errors. Then, I exported and ran export PYCVODES_NO_LAPACK='1' && python setup.py build in the pycvodes top directory and successfully compiled. I do the same test with chempy (with and without PYCVODES_NO_LAPACK in the environment), and install always fails on pycvodes if PYCVODES_NO_LAPACK='1' is not set in the environment. I have not changed PYCVODES_NO_KLU.

I'm using requirements.txt (in chempy) right now as a list to get a virtualenv set up for testing and development (deployment only, not packaging and installing chempy).

On Sat, Oct 31, 2020 at 10:50 AM Bjorn notifications@github.com wrote:

sorry, I don't quite follow, did you export environment variables on the command line prior to installing?, i.e.:

export PYCVODES_NO_LAPACK=1 PYCVODES_NO_KLU=1 python3 -m pip install .

regarding requirements.txt, I think it's deprecated since 5 years or so? setuptools & requires arguments are preferred I believe? (way back, I used to maintain a requirements.txt in the repository even...)

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/bjodah/pycvodes/issues/131#issuecomment-719951855, or unsubscribe https://github.com/notifications/unsubscribe-auth/AOQCHS53WZ6TW4AXYMR4AJTSNQW6BANCNFSM4TEID3FQ .

-- Jeremy A. Gray

bjodah commented 3 years ago

Alright, hard to say exactly why it is failing. Support for both with & without lapack was added to allow compilation of sundials with long double (which BLAS/LAPACK does not support). There's been a lot of back and forth in this area over the years, simply because each new environment has essentially shown to need its own treatment.

When one has multiple versions of sundials on the same machine, I have found that my sanity is best preserved by directly setting CFLAGS & LDFLAGS:

export CFLAGS="-isystem /opt/sundials-5.2.0-noklu-nolapack-extended-int64/include -DPYCVODES_NO_KLU=1 -DPYCVODES_NO_LAPACK=1"
export LDFLAGS="-Wl,--disable-new-dtags -Wl,-rpath,/opt/sundials-5.2.0-noklu-nolapack-extended-int64/lib -L/opt/sundials-5.2.0-noklu-nolapack-extended-int64/lib"
jeremyagray commented 3 years ago

Let me see if I understand everything. I intend to have just one sundials installation. I have lapack, blas, boost, suitesparse, gsl, and sundials either in the system include/ld paths or in a searched path, so all of them can include and link with each other as necessary (and I can adjust CFLAGS/LDFLAGS if necessary). So if sundials is compiled without klu and lapack, does pycvodes/anyode include/link with suitesparse and lapack directly? Is there any functionality lost that might be useful in pycvodes or chempy?

I think my question and this issue rests on how sundials should be configured before compilation because .drone.yml indicates that sundials should be compiled without klu and lapack. Or alternatively, if sundials is compiled with certain features (klu or lapack), should pycvodes/anyode ignore parts of the sundials interface in deference to the implementation in pycvodes/anyode?

On Sat, Oct 31, 2020 at 11:44 AM Bjorn notifications@github.com wrote:

Alright, hard to say exactly why it is failing. Supporting both with & without lapack done to allow compilation of sundials with long double (which BLAS/LAPACK does not support). There's been a lot of back and forth in this area over the years, simply because each new environment has essentially shown to need its own treatment.

When one has multiple versions of sundials on the same machine, I have found that my sanity is best preserved by directly setting CFLAGS & LDFLAGS:

export CFLAGS="-isystem /opt/sundials-5.2.0-noklu-nolapack-extended-int64/include -DPYCVODES_NO_KLU=1 -DPYCVODES_NO_LAPACK=1" export LDFLAGS="-Wl,--disable-new-dtags -Wl,-rpath,/opt/sundials-5.2.0-noklu-nolapack-extended-int64/lib -L/opt/sundials-5.2.0-noklu-nolapack-extended-int64/lib"

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/bjodah/pycvodes/issues/131#issuecomment-719958396, or unsubscribe https://github.com/notifications/unsubscribe-auth/AOQCHS24DN6Q43SDJEQVILDSNQ5H5ANCNFSM4TEID3FQ .

-- Jeremy A. Gray

bjodah commented 3 years ago

So if sundials is compiled without klu and lapack, does pycvodes/anyode include/link with suitesparse and lapack directly?

It should not.

Is there any functionality lost that might be useful in pycvodes or chempy?

No, after some problem size LAPACK will be faster, but for small problems, I've seen the inverse, i.e. not using LAPACK is significantly faster.

pycvodes can exploit matrix sparseness when KLU is available. ChemPy does not use this however.

Right now I just upgraded the Debian version to Bullseye in the Dockerfile in ChemPy's repository: https://github.com/bjodah/chempy/pull/186

All tests pass in that environment (you need docker installed to run that script on your machine in case you want to compare).

jeremyagray commented 3 years ago

I've finally tracked down the source of the issue, but I don't know enough about C++ to completely understand what is happening. Since we tried an include guard with #ifndef _SUNDIALS_LAPACK_H and it did not work, I tried some other variations that also failed. After some research, I found that the include guards work against multiple inclusion but not multiple definition. Changing all the extern "C" in anyode/include/anyode/anyode_blas_lapack.h to just extern allowed pycvodes to build without error (tests ran successfully). I would assume this has something to do with sundials using extern "C" in sundials_lapack.h.

This is on debian bullseye current, with sundials-5.5.0 compiled with

cmake -DCMAKE_INSTALL_PREFIX=/home/gray/.local -DLAPACK_ENABLE=ON
-DKLU_ENABLE=ON -DKLU_INCLUDE_DIR=/usr/include/suitesparse
-DKLU_LIBRARY_DIR=/usr/lib/x86_64-linux-gnu .. && make && make install

On Tue, Nov 24, 2020 at 3:05 AM Bjorn notifications@github.com wrote:

So if sundials is compiled without klu and lapack, does pycvodes/anyode include/link with suitesparse and lapack directly?

It should not.

Is there any functionality lost that might be useful in pycvodes or chempy?

No, after some problem size LAPACK will be faster, but for small problems, I've seen the inverse, i.e. not using LAPACK is significantly faster.

pycvodes can exploit matrix sparseness when KLU is available. ChemPy does not use this however.

Right now I just upgraded the Debian version to Bullseye in the Dockerfile in ChemPy's repository: bjodah/chempy#186 https://github.com/bjodah/chempy/pull/186

All tests pass in that environment (you need docker installed to run that script on your machine in case you want to compare).

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/bjodah/pycvodes/issues/131#issuecomment-732758125, or unsubscribe https://github.com/notifications/unsubscribe-auth/AOQCHS3KYTYIO75JSVTIERLSRNZMVANCNFSM4TEID3FQ .

-- Jeremy A. Gray

bjodah commented 3 years ago

Oh, that's a tricky one. Often extern "C" { itself is guarded by #ifdef __cplusplus (to allow compilation with C++ compilers as well as C compilers). Sounds like I may have forgotten such a guard somewhere then.