Installation using alternate lapack lib?

[dPys]

Hi guys,

I've started including skggm as an optional dependency for PyNets (see graphestimation module), but I've been struggling to get skggm to compile on HPC where I don't have root privileges and lapack is only offered as part of the MKL suite. Have you all had any success installing skggm with MKL implementations of lapack?

If I install out-of-the-box, the installer can't find lapack: /opt/apps/gcc/7.1.0/lib/gcc/x86_64-pc-linux-gnu/7.1.0/../../../../x86_64-pc-linux-gnu/bin/ld: cannot find -llapack collect2: error: ld returned 1 exit status

A modified install from the MKL libraries would probably be a command that looks something like this:

python3 setup.py config -I$MKLROOT/include -L${MKLROOT}/lib/intel64 -lmkl_intel_lp64 -lmkl_core -lmkl_gnu_thread -lpthread -llmkl_lapack95_lp64 install

Thoughts?

derek;

[jasonlaska]

Hi Derek,

I am unfamiliar with MKL-lapack but if you do figure out how to get it to work, we'd appreciate a PR with some changes to setup.py file (maybe to detect that those versions of the libraries exist and use them instead of https://github.com/skggm/skggm/blob/develop/setup.py#L42). Including it there may make it easier for your package to rely on skggm as a dependency.

You can clone this repo and try building just the cpp binary via

cd inverse_covariance/pyquic
make
make python3  (for python3)

and play around with this version of the setup file: https://github.com/skggm/skggm/blob/develop/inverse_covariance/pyquic/setup.py so as not to rebuild the whole skggm module.

-Jason

[jasonlaska]

Also-- thanks for including us!

[dPys]

You bet. Will try a few things and let you know what I come up with!

[dPys]

Update:

So I worked some magic and got skggm to install using the MKL lapack implementation.

First, I add the following to both setup.py and inverse_covariance/pyquic/setup.py:

    include_dirs = [numpy.get_include(), "/usr/local/include", "/opt/intel/compilers_and_libraries_2017.4.196/linux/mkl/include"]
    extra_compile_args = ['-msse2', '-O2', '-fPIC', '-w']
    extra_link_args = ["-L/opt/intel/compilers_and_libraries_2017.4.196/linux/mkl -lmkl_def -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_sequential -lmkl_core -lmkl_gnu_thread -lmkl_lapack95_lp64"]

and add the following to my setup.cfg:

[mkl]
library_dirs = /opt/intel/compilers_and_libraries_2017.4.196/linux/mkl/lib/intel64
include_dirs = /opt/intel/compilers_and_libraries_2017.4.196/linux/mkl/include
mkl_libs = mkl_def, mkl_intel_lp64 mkl_core, mkl_gnu_thread, mkl_intel_thread, mkl_sequential 
lapack_libs = mkl_lapack95_lp64

Within inverse_covariance/pyquic, I then delete pyquic.cpp, build, and *.so

I then run:

make python3

which completes successfully.

Next, I cd back to skggm's base directory and run:

python3 setup.py install --user

Then everything builds and installs successfully, albeit with this (probably negligible?) warning at the very beginning: _Warning: Extension name 'pyquic.pyquic' does not match fully qualified name 'inverse_covariance.pyquic.pyquic' of 'inversecovariance/pyquic/pyquic.pyx'

But then when I enter the python3 interpreter, and run the following import: from inverse_covariance import AdaptiveGraphLasso, QuicGraphLassoEBIC

It errors with the following: _ImportError: /work/04171/dpisner/stampede2/skggm/inverse_covariance/pyquic/pyquic.cpython-36m-x86_64-linux-gnu.so: undefined symbol: dpotrf__

Maybe the MKL lapack version isn't meshing well with the dpotrf declaration? Thoughts?

derek;

[jasonlaska]

Try making the same edits in the skggm setup.py, clean the .so and try re-running/compiling from the skggm setup.

[dPys]

Thanks for the quick reply!

Unfortunately, no dice. I had already tweaked both setup.py's...

Other thoughts?

[jasonlaska]

Looks like a linking issue with something in lapack, but cursory googling hasn't yielded any good results. Found this page which gives a long list of linking options you can try: https://software.intel.com/en-us/mkl-linux-developer-guide-linking-on-intel-64-architecture-systems

[dPys]

Yeah that's what I figured. And in my experience, this type of thing is usually a roadblock...

Hmm, what version of lapack does skggm use by default? the one from ATLAS? that is, which lapack implementation did you guys develop with?

If unsure, running:

np.__config__.show()

in your python3 interpreter might offer some hints?

Thanks for bearing with me! -derek

[jasonlaska]

I'm on macos 10.11.6:

>>> np.__config__.show()
blas_mkl_info:
  NOT AVAILABLE
blis_info:
  NOT AVAILABLE
openblas_info:
  NOT AVAILABLE
atlas_3_10_blas_threads_info:
  NOT AVAILABLE
atlas_3_10_blas_info:
  NOT AVAILABLE
atlas_blas_threads_info:
  NOT AVAILABLE
atlas_blas_info:
  NOT AVAILABLE
blas_opt_info:
    extra_compile_args = ['-msse3', '-I/System/Library/Frameworks/vecLib.framework/Headers']
    extra_link_args = ['-Wl,-framework', '-Wl,Accelerate']
    define_macros = [('NO_ATLAS_INFO', 3), ('HAVE_CBLAS', None)]
lapack_mkl_info:
  NOT AVAILABLE
openblas_lapack_info:
  NOT AVAILABLE
atlas_3_10_threads_info:
  NOT AVAILABLE
atlas_3_10_info:
  NOT AVAILABLE
atlas_threads_info:
  NOT AVAILABLE
atlas_info:
  NOT AVAILABLE
lapack_opt_info:
    extra_compile_args = ['-msse3']
    extra_link_args = ['-Wl,-framework', '-Wl,Accelerate']
    define_macros = [('NO_ATLAS_INFO', 3), ('HAVE_CBLAS', None)]

[dPys]

Thanks!

I'll ping the HPC admins and see what gives with our lapack implementation...

Stay tuned. -derek

[dPys]

Okay so I think I've isolated the issue, and it is indeed a problem with non-ATLAS versions of lapack not playing nice with QUIC.

Specifically, the ATLAS version of lapack comes with one special c-extension, clapack-dpotrf, which no other lapack implementations (MKL or otherwise) seem to include! This means that anyone who is not on OS X (which doesn't have to worry about lapack for compiling QUIC) and does not have the ability to install ATLAS should be unable to run skggm! Have you had any other users encounter this?

-derek

[jasonlaska]

I'm pretty sure you can install ATLAS on many platforms, available in debian linux and bsd distributions by default (http://math-atlas.sourceforge.net/faq.html). It is also pretty easily installed with apt on ubuntu (see notes here: http://caffe.berkeleyvision.org/install_apt.html).

Is it possible to get your admin to install that for you? What os are you working with?

[mnarayan]

@dpisner453 Thanks for your work on this. Also you can look at our travis.yml https://github.com/skggm/skggm/blob/develop/.travis.yml to get a sense of how we've been able to run skggm on many linux systems.

I have been able to run skggm on two different HPC clusters, so I can double check their specs and get back to you on what works.

An alternative solution: Would it helpful for you to have a docker container to run skggm from instead? This could end up being an easier path if you can do docker->singularity on HPC machines.

[dPys]

Thanks for following up on this guys! I'm working on TACC's stampede2 supercomputer. Although I've pushed for them to install ATLAS in the past (because as @jasonlaska noted, it can be installed on pretty much every platform!), they are insistent that the MKL libs should suffice in place of ATLAS (I strongly disagree with this, even though they are correct that MKL's lapack should theoretically be faster/more superior). I believe the stampede2 OS is redhat, but TACC's systems just have so many users (XSEDE, etc.), that the admins are very rigid about what they will install at the system level. Without sudo or root privileges, I'm pretty much stuck with MKL for now :/

Good idea @mnarayan RE: a docker container. Stampede2 does indeed have singularity... But please don't feel pressured to build a container just for me unless you've been meaning to do this anyway for skggm.

Cheers, Derek

[mnarayan]

Also want to point out, maybe solution could include slightly modifying QUIC.C to include the right libraries.

Some decisions were made to make things Octave compatible.

// It would be preferable to use an include such as lapack.h. Except
// lapack.h is not available from the octave or liblapack-dev packages...

extern "C" {

    void dpotrf_(char* uplo, ptrdiff_t* n, double* A, ptrdiff_t* lda,

ptrdiff_t* info);

    void dpotri_(char* uplo, ptrdiff_t* n, double* A, ptrdiff_t* lda,

ptrdiff_t* info);

}

On Thu, Jan 4, 2018, 15:52 Derek Pisner notifications@github.com wrote:

Thanks for following up on this guys! I'm working on TACC's stampede2 supercomputer. Although I've pushed for them to install ATLAS in the past (because as @jasonlaska https://github.com/jasonlaska noted, it can be installed on pretty much every platform!), they are insistent that the MKL libs should suffice in place of ATLAS (I strongly disagree with this, even though they are correct that MKL's lapack should theoretically be faster/more superior). I believe the stampede2 OS is redhat, but TACC's systems just have so many users (XSEDE, etc.), that the admins are very rigid about what they will install at the system level. Without sudo or root privileges, I'm pretty much stuck with MKL for now :/

Good idea @mnarayan https://github.com/mnarayan RE: a docker container. Stampede2 does indeed have singularity... But please don't feel pressured to build a container just for me unless you've been meaning to do this anyway for skggm.

Cheers, Derek

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/skggm/skggm/issues/106#issuecomment-355395390, or mute the thread https://github.com/notifications/unsubscribe-auth/AAov9_-SlYbRkz2FoNDe_jZnrMpjsEM8ks5tHToYgaJpZM4RQKas .

[jasonlaska]

Here's the dockerfile (https://github.com/neuroquant/jf2016-skggm/blob/master/Dockerfile) from @mnarayan's binder demo (https://github.com/neuroquant/jf2016-skggm), maybe you can use that as boilerplate of what would go into a singularity container.

[mnarayan]

Also, singularity hub automatically builds singularity containers from a Dockerfile.

See https://github.com/TheEtkinLab/CPAC/blob/master/sing_deploy.sh

On Thu, Jan 4, 2018, 16:48 Jason Laska notifications@github.com wrote:

Here's the dockerfile ( https://github.com/neuroquant/jf2016-skggm/blob/master/Dockerfile) from @mnarayan https://github.com/mnarayan's binder demo ( https://github.com/neuroquant/jf2016-skggm), maybe you can use that as boilerplate of what would go into a singularity container.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/skggm/skggm/issues/106#issuecomment-355408334, or mute the thread https://github.com/notifications/unsubscribe-auth/AAov93LDBh7OUw3gQyWcWgkT-g2A9hhAks5tHUcdgaJpZM4RQKas .

[dPys]

Cool. @mnarayan -- dpotri_ appears in lines: 203, 355, and 454 of QUIC.C I have no idea what it does though, so I hesitate to modify it and would probably just break things...

And thanks @jasonlaska -- Will give the Dockerfile a try.

[mnarayan]

With respect to modifying QUIC.C, we might want to try making a conditional — if intel MKL available include lapacke.h otherwise keep the current declaration. Then it should take care of the compilation errors that dpotrf or dportri was not found. These are C interface functions to lapack for fast Cholesky decompositions. @jasonlaska Does this make sense to you?

Shogun has an instance of doing this here

It seems to me that the function API slightly different, needs to be dpotrf for MKL not dpotrf_. This would explain @dpisner453's error

It errors with the following: ImportError: /work/04171/dpisner/stampede2/skggm/inverse_covariance/pyquic/pyquic.cpython-36m-x8664-linux-gnu.so: undefined symbol: dpotrf

Maybe the MKL lapack version isn't meshing well with the dpotrf declaration? Thoughts?

If so, we can look into making this change in pyquic and making the change here as well.

[dPys]

@mnarayan -- good catch. I have a feeling that will work. And for other users who compile with mkl, I think they may see a considerable speedup with many of skggm's functions as well...

In the meantime, I've built a docker / singularity container, which I will be testing on stampede2 this weekend, in case the tweaks to pyquic won't be ready yet!

Thank you so much guys. derek;

[jasonlaska]

Yea if you can make it work with a preprocessor flag #ifdef and just just swap out the call dependening on the def, that would be swell. Just get a PR working on your end and we can try compiling it on our various other environments.

[dPys]

FYI:

Dockerfile:

FROM andrewosh/binder-base

USER root

# Add dependency
RUN apt-get -qq update
RUN conda update libgfortran --force
RUN conda install libgcc --force
RUN apt-get install -y libblas3gf libblas-doc libblas-dev liblapack3gf liblapack-doc liblapack-dev
RUN apt-get install -y software-properties-common python-software-properties
RUN apt-get install -y libxtst6 libgtk2.0-bin libxft2 lib32ncurses5 libXp6 libxpm-dev libxpm4 libxmu-dev
#RUN apt-get install -y libbz2-dev libsqlite3-dev

# Create python3 environment
#RUN wget https://www.python.org/ftp/python/3.6.0/Python-3.6.0.tgz
#RUN tar xvf Python-3.6.0.tgz
#RUN cd Python-3.6.0 && ./configure --enable-optimizations --enable-loadable-sqlite-extensions && make -j8 && make altinstall
#RUN wget https://bootstrap.pypa.io/get-pip.py && python3.6 get-pip.py
#RUN rm -f Python-3.6.0.tgz
RUN apt-get install -y python3-dev python3-pip

RUN python3.4 -m pip install --upgrade pip
RUN python3.4 -m pip install Cython>=0.24 nilearn>=0.2.4 numpy>=1.12.1 scikit-learn>=0.18.2 scipy>=0.19.1 pytest>=2.9.2 seaborn>=0.7.1 nose>=1.3.6 tabulate>=0.7.5 ipython

# Install requirements for Python 3
RUN python3.4 -m pip install skggm --upgrade
RUN python3.4 -m pip install pynets>=0.3.5 --upgrade

RUN chmod 775 -R /usr/local/lib/python3.4
RUN chown root:main -R /usr/local/lib/python3.4

USER main
RUN python3.4 -m pip install pandas --upgrade --user

# Environment variable try to fix lapack issue
ENV PYTHONPATH /usr/local/lib/python3.4/dist-packages:$PYTHONPATH
ENV LD_PRELOAD /usr/lib/x86_64-linux-gnu/libgfortran.so.3

Then:

docker build -t skggm_singularity .
docker run -ti --rm --privileged \
    -v /tmp:/tmp \
    -v /var/tmp:/var/tmp \
    skggm_singularity

and:

docker run -ti --rm \
    --privileged \
    -v /var/run/docker.sock:/var/run/docker.sock \
    -v ${BUILDIR}/singularity_images:/output \
    filo/docker2singularity "skggm_singularity:latest"

[dPys]

It's not running super cleanly yet, but I'm working on it. Really want to get a docker image with all of this set up for python3.6...

@mnarayan -- Any progress on your end with the dpotrf_ swap?

[mnarayan]

Sorry, I'm not going to be able to test out the idea for a few days at least.

On Sat, Jan 6, 2018 at 11:02 PM, Derek Pisner notifications@github.com wrote:

It's not running super cleanly yet, but I'm working on it. Really want to get a docker image with all of this set up for python3.6...

@mnarayan https://github.com/mnarayan -- Any progress on your end with the dpotrf_ swap?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/skggm/skggm/issues/106#issuecomment-355797666, or mute the thread https://github.com/notifications/unsubscribe-auth/AAov99R_PJeZACjGsb1mRXvOleHPY2pMks5tIEHkgaJpZM4RQKas .

-- Manjari Narayan, Ph.D. Postdoctoral Research Fellow School of Medicine, Stanford University

---

[dPys]

No problem!

Grateful for the help. derek

[mnarayan]

Ok, I tested out my idea for using preprocessor flags to define options for MKL libraries, https://github.com/skggm/skggm/blob/feature/pyquic-mkl/inverse_covariance/pyquic/QUIC.C#L55

However since I don't have automatic configuration of flags, you will currently see the same error that @dpisner453 was running into with undefined reference to dpotrf_.

When I force HAVE_MKL to be 1 then it looks like QUIC's calls to dpotrf and dpotri are not compatible with the MKL API

QUIC.C: In function ‘double projLogDet(long unsigned int, const double*, double*, double*, double*)’:
QUIC.C:213:46: error: cannot convert ‘ptrdiff_t* {aka long int*}’ to ‘const int*’ for argument ‘2’ to ‘void dpotrf(const char*, const int*, double*, const int*, int*)’
     dpotrf_((char*) "U", &p0, prW, &p0, &info);
                                              ^
QUIC.C: In function ‘void QUIC(char, uint32_t&, const double*, double*, uint32_t&, const double*, double&, int32_t&, uint32_t&, double*, double*, double*, double*, uint32_t*, double*)’:
QUIC.C:365:45: error: cannot convert ‘ptrdiff_t* {aka long int*}’ to ‘const int*’ for argument ‘2’ to ‘void dpotrf(const char*, const int*, double*, const int*, int*)’
      dpotrf_((char*) "U", &p0, U, &p0, &info);
                                             ^
QUIC.C:417:45: error: cannot convert ‘ptrdiff_t* {aka long int*}’ to ‘const int*’ for argument ‘2’ to ‘void dpotrf(const char*, const int*, double*, const int*, int*)’
      dpotrf_((char*) "U", &p0, W, &p0, &info);
                                             ^
QUIC.C:464:41: error: cannot convert ‘ptrdiff_t* {aka long int*}’ to ‘const int*’ for argument ‘2’ to ‘void dpotri(const char*, const int*, double*, const int*, int*)’
  dpotri_((char*) "U", &p0, W, &p0, &info);

So it looks like the fix will not be as easy as I previously thought.

[jasonlaska]

@mnarayan yea, I would also suggest that we wrap the call sites with #ifdef/#ifndef/#else rather than redefining the function name which is an antipattern or alternatively define new variables DPOTRF / DPOTRI that get used at the call sites (closer to the shogun example). Also, #def variables should be all caps by convention as well.

If you wrap the call sites, you can change the call so that it's compatible with the interface. In this case looks like you'll need to downcast long int * to int * (and since this is an array, you may have to actually cycle through it and create a new array which may not be desirable), this may cause precision issues for us. Alternatively, you could determine whether or not its really required that p be long https://github.com/skggm/skggm/blob/feature/pyquic-mkl/inverse_covariance/pyquic/QUIC.C#L196.

In general, I agree, its less trivial do this swap than we had hoped.

[dPys]

Figured as much. Well, I think we learned at least three things here*:

1. lapack/blas implementations outside of ATLAS (though possibly just limited to MKL) do not play nice with QUIC.

2. Swapping the dpotrf and dpotri references is involved, but for future skggm/QUIC development, may be something to consider, particularly if skggm wishes to widen its user-base further (and since MKL is in a unique position to know about future Intel instruction sets / upcoming processors, it will probably always have the upper-hand/be faster than ATLAS).

3. Using a docker container for skggm can be an excellent way around this problem thanks to @mnarayan 's great suggestion.

Nice to meet you all and thanks for the help! derek;

[mnarayan]

You might want to use a gender neutral term ;-). Not all dudes here.

Its not an issue for OPENBLAS as well which have been available on HPCs I've tested. We should add this to travis too. MKL seems to be the main issue.

On a related note. The QUIC implementation as a part of BIGQUIC does not seem to use lapack libraries at all since it tries not to store the covariance matrix though it does have its own sparse matrix class that uses omp.h and metis.h. Unfortunately the API and functionality of this version does not support the weighted graphical lasso.

On Mon, Jan 8, 2018 at 3:21 PM, Derek Pisner notifications@github.com wrote:

Figured as much. Well, I think we learned at least three things here gentlemen:

1.

lapack/blas implementations outside of ATLAS (though possibly just limited to MKL) do not play nice with QUIC. 2.

Swapping the dpotrf and dpotri references is involved, but for future skggm/QUIC development, may be something to consider, particularly if skggm wishes to widen its user-base further (and since MKL is in a unique position to know about future Intel instruction sets / upcoming processors, it will probably always have the upper-hand/be faster than ATLAS). 3.

Using a docker container for skggm can be an excellent way around this problem thanks to @jasonlaska https://github.com/jasonlaska 's great suggestion.

Nice to meet you all, derek;

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/skggm/skggm/issues/106#issuecomment-356128404, or mute the thread https://github.com/notifications/unsubscribe-auth/AAov912BHfWqPFFNDnYAMHkc6cLL_qaCks5tIqMIgaJpZM4RQKas .

-- Manjari Narayan, Ph.D. Postdoctoral Research Fellow School of Medicine, Stanford University

---

[dPys]

Rightly noted!*

Good suggestion RE: BIGQUIC. Will take a look!

-derek

[mnarayan]

Yeah, we have another issue regarding BigQUIC implementation. Unfortunately it is a lower priority since it requires a lot of work to bring API to support the various estimator classes supported by skggm. I don't recommend it for practical/science usage as you cannot modify tuning parameters as needed.

It is not clear that time on BigQUIC will be time well spent. In fact, scrapping most of their interface and wrapping our existing QUIC.C with metis for clustering to distribute computations might be good enough.

We are already looking into supporting some other fast solvers other than QUIC too. So will keep C library issues in mind as we proceed.

On Tue, Jan 9, 2018 at 2:30 PM, Derek Pisner notifications@github.com wrote:

Rightly noted!*

Good suggestion RE: BIGQUIC. Will take a look!

-derek

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/skggm/skggm/issues/106#issuecomment-356435729, or mute the thread https://github.com/notifications/unsubscribe-auth/AAov9we-L9SsBX6d3-9y1amQHO880vkdks5tI-iVgaJpZM4RQKas .

-- Manjari Narayan, Ph.D. Postdoctoral Research Fellow School of Medicine, Stanford University

---

[dPys]

Makes sense! Well again, thanks for all the help @mnarayan and @jasonlaska . Look forward to working with you all more in the future and seeing what skggm creates next!

derek;

[jasonlaska]

Please do let us know how your project goes. We'd love for more folks to use these methods in their work.

-j

[mnarayan]

@jasonlaska Regarding your comment here. If lapack doesn't declare matrix size needs to be long int, I don't see why we need to be. I don't think we will lose precision here.

If you wrap the call sites, you can change the call so that it's compatible with the interface. In this case looks like you'll need to downcast long int to int (and since this is an array, you may have to actually cycle through it and create a new array which may not be desirable), this may cause precision issues for us. Alternatively, you could determine whether or not its really required that p be long https://github.com/skggm/skggm/blob/feature/pyquic-mkl/inverse_covariance/pyquic/QUIC.C#L196.

[mnarayan]

I also checked with my numerical analyst friend who concurs that down casting long int to int shouldn't matter for us.

[alimoges42]

Hi all, hate to reopen this nearly expired can of worms but I am having an identical problem with the install (receiving the dpotrf_ error as described by @dPys ). Fortunately, I do have admin privileges and installed libatlas-base-dev as suggested, but I am not sure how to get numpy to link to the lapack in ATLAS as opposed to BLAS. Any chance one of you could help out with this? Here is output of my np.config\.show():

blas_mkl_info:
  NOT AVAILABLE

blis_info:
  NOT AVAILABLE

openblas_info:
    libraries = ['openblas', 'openblas']
    library_dirs = ['/usr/local/lib']
    language = c
    define_macros = [('HAVE_CBLAS', None)]

blas_opt_info:
    libraries = ['openblas', 'openblas']
    library_dirs = ['/usr/local/lib']
    language = c
    define_macros = [('HAVE_CBLAS', None)]

lapack_mkl_info:
  NOT AVAILABLE

openblas_lapack_info:
    libraries = ['openblas', 'openblas']
    library_dirs = ['/usr/local/lib']
    language = c
    define_macros = [('HAVE_CBLAS', None)]

lapack_opt_info:
    libraries = ['openblas', 'openblas']
    library_dirs = ['/usr/local/lib']
    language = c
    define_macros = [('HAVE_CBLAS', None)]

[jasonlaska]

Hi @alimoges42 thanks for the comment.

I think you need to re-install numpy and specify a different configuration, does https://stackoverflow.com/questions/21671040/link-atlas-mkl-to-an-installed-numpy help?

Are you on Linux or MacOS?

[alimoges42]

Ok, I had a feeling I would need to recompile. Working on that now. I am using Ubuntu 18.04, and I'll keep you posted on progress. Thank you!

[alimoges42]

Hey @jasonlaska everything seems to be working for the most part. I built numpy against atlas and tried a few different methods of installing skggm just to see how things worked. One thing I noticed, however, is that when I run the pytest line, it tells me that no tests are being run. Not sure if that's a big deal though, as I also ran the estimator_suite.py file which seems to work fine.

Anyway, thanks for the help! Sorry it took so long to get back to you - I wanted to learn how to compile a couple linear algebra libraries from source before rebuilding numpy (still learning my way around Linux). Looking forward to implementing this in my neuro research!

[jasonlaska]

Great @alimoges42 -- To run the tests, are you running python -m pytest inverse_covariance from the repository directory (e.g., skggm)? It should find 79 items.

Please do let us know how you're using the package, we'd love to hear about publications using skggm that we can promote.

[alimoges42]

@jasonlaska Ah, that was the problem. I was executing pytest on the inverse_covariance.py file from inside the inverse_covariance directory. The test ran, although I did get some warnings.

[alimoges42]

Forgot to mention that when running estimator_suite.py, it throws an error due to a matplotlib function change. It crunches the numbers fine, just won't show the correlation matrices. Here: Traceback (most recent call last): File "estimator_suite.py", line 529, in <module> show_results(plot_covs, plot_precs) File "estimator_suite.py", line 94, in show_results multiplot(covs, 'Covariance Estimates') File "estimator_suite.py", line 87, in multiplot ax.set_axis_bgcolor('.7') AttributeError: 'AxesSubplot' object has no attribute 'set_axis_bgcolor'

Looks like this is due to replacement of set_axis_bgcolor by facecolor. https://github.com/scikit-learn/scikit-learn/issues/10762 https://github.com/matplotlib/matplotlib/blob/b541a8a4cefc026d91e705bf1e105c331cd87059/doc/api/api_changes.rst#color-of-axes