search

manodeep / Corrfunc

⚡️⚡️⚡️Blazing fast correlation functions on the CPU.
https://corrfunc.readthedocs.io
MIT License
166 stars 51 forks source link

Cant install on SDSS Jupyter notebook #113

Closed tevslin closed 7 years ago

tevslin commented 7 years ago
  1. got message in make and make install that modules couldn't be found when installing clone:

/usr/bin/ld: cannot find -lm /usr/bin/ld: cannot find -lrt /usr/bin/ld: cannot find -lm /usr/bin/ld: cannot find -lpthread /usr/bin/ld: cannot find -lc

actually these aren't modules at all. they are gcc options.

  1. tried simple pip install. got subset of messages.

    /usr/bin/ld: cannot find -lm /usr/bin/ld: cannot find -lpthread /usr/bin/ld: cannot find -lc

  2. tried installing manodeep version of gcc from github:

/home/idies/anaconda3/bin/.gcc-post-link.sh: line 10: lsb_release: command not found Couldn't locate crtXXX.o in default library search paths. You may not have it at all. It is usually packaged in libc6-dev/glibc-devel packages. We will try to locate crtXXX.o with system installed gcc... There is no /usr/bin/gcc Can't install the gcc package unless your system has crtXXX.o.

  1. tried installing anaconda version. same thing.

Don't have all the details of the system Jupyter is actually running on since it is provided by SkyServer. Pretty sure it's UNIX. Usually works nicely with imported packages.

Thank you/

manodeep commented 7 years ago

The conda error message makes me think that installing Corrfunc is not possible. Generally speaking, those errors occur because default libraries required by the compiler are not found. In any *nix, libm.so, libpthread.so and libc.so should be in the linker path.

My guess is that SkyServer is in a sandboxed environment, specifically designed to prevent most compile/installs.

@lgarrison - do you have any idea how to reproduce/fix this?

lgarrison commented 7 years ago

I made an account on http://compute.sciserver.org to test this, and I was able to reproduce the problem. I managed to get much further by choosing the Image Python + Dev (astro) when creating a container, which comes with gcc and GNU Make, unlike the default image. git clone followed by python setup.py build works without a hitch; however python setup.py install results in a permissions error for writing to easy-install.pth. The workaround is to use a conda env. So, the full solution looks like this:

Trying to install Corrfunc 1.1 via pip install Corrfunc has its own problems, but I don't think it's worth fixing since Corrfunc 2.0 is so close to release.

tevslin commented 7 years ago

Thank you both very much.

Will try this tomorrow. It sounds like a solution.

From: Lehman Garrison [mailto:notifications@github.com] Sent: Sunday, March 26, 2017 8:30 PM To: manodeep/Corrfunc Corrfunc@noreply.github.com Cc: tevslin tom@evslin.com; Author author@noreply.github.com Subject: Re: [manodeep/Corrfunc] Cant install on SDSS Jupyter notebook (#113)

I made an account on http://compute.sciserver.org to test this, and I was able to reproduce the problem. I managed to get much further by choosing the Image Python + Dev (astro) when creating a container, which comes with gcc and GNU Make, unlike the default image. git clone followed by python setup.py build works without a hitch; however python setup.py install results in a permissions error for writing to easy-install.pth. The workaround is to use a conda env https://conda.io/docs/using/envs.html . So, the full solution looks like this:

cd ~ conda create --name myenv python gsl numpy source activate myenv git clone https://github.com/manodeep/Corrfunc.git cd Corrfunc python setup.py install

and that should do it! Test it with

conda install nose nosetests Corrfunc

Trying to install Corrfunc 1.1 via pip install Corrfunc has its own problems, but I don't think it's worth fixing since Corrfunc 2.0 is so close to release.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/manodeep/Corrfunc/issues/113#issuecomment-289329048 , or mute the thread https://github.com/notifications/unsubscribe-auth/AZcghvHzEiK1iXAGkbnYWwriUstThjWCks5rpwMcgaJpZM4MpR9G . https://github.com/notifications/beacon/AZcghluPwHZ4hMK9Zg7LjHMLSX-rybg2ks5rpwMcgaJpZM4MpR9G.gif

manodeep commented 7 years ago

Thanks @lgarrison.

If this is simply a easy-install.pth issue, then python setup.py install --user might also solve the install issue.

lgarrison commented 7 years ago

You're probably right! Don't know why I didn't think to try that. I think I was thrown off by the fact that the anaconda install was already in the user's home directory, despite the user not having write permissions there.

On Sun, Mar 26, 2017 at 8:56 PM, Manodeep Sinha notifications@github.com wrote:

Thanks @lgarrison https://github.com/lgarrison.

If this is simply a easy-install.pth issue, then python setup.py install --user might also solve the install issue.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/manodeep/Corrfunc/issues/113#issuecomment-289330890, or mute the thread https://github.com/notifications/unsubscribe-auth/AFYPy3ARsVkhqgp4mlvuSd2ueCafOSJjks5rpwlGgaJpZM4MpR9G .

manodeep commented 7 years ago

Fingers crossed - something should work :)

@tevslin - please let us know if you get Corrfunc to work.

tevslin commented 7 years ago

Will try again tonight or tomorrow am. Thank you both for the very timely support.

From: Manodeep Sinha [mailto:notifications@github.com] Sent: Sunday, March 26, 2017 11:33 PM To: manodeep/Corrfunc Corrfunc@noreply.github.com Cc: tevslin tom@evslin.com; Mention mention@noreply.github.com Subject: Re: [manodeep/Corrfunc] Cant install on SDSS Jupyter notebook (#113)

Fingers crossed - something should work :)

@tevslin https://github.com/tevslin - please let us know if you get Corrfunc to work.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/manodeep/Corrfunc/issues/113#issuecomment-289346074 , or mute the thread https://github.com/notifications/unsubscribe-auth/AZcghkml0NeC4iJyQPkYl9lbCD4K979Oks5rpy3tgaJpZM4MpR9G . https://github.com/notifications/beacon/AZcghuuhD-eRYTdEN7q3IsaoV8W0AMhQks5rpy3tgaJpZM4MpR9G.gif

tevslin commented 7 years ago

Thanks to both of you it’s installed and tested.

Did it within the Jupyter notebook so I can send to you. It’s attached.

I very much appreciate your help and am ,looking forward to working with Corrfunc (and seeing what’s in 2.0)

From: Manodeep Sinha [mailto:notifications@github.com] Sent: Sunday, March 26, 2017 11:33 PM To: manodeep/Corrfunc Corrfunc@noreply.github.com Cc: tevslin tom@evslin.com; Mention mention@noreply.github.com Subject: Re: [manodeep/Corrfunc] Cant install on SDSS Jupyter notebook (#113)

Fingers crossed - something should work :)

@tevslin https://github.com/tevslin - please let us know if you get Corrfunc to work.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/manodeep/Corrfunc/issues/113#issuecomment-289346074 , or mute the thread https://github.com/notifications/unsubscribe-auth/AZcghkml0NeC4iJyQPkYl9lbCD4K979Oks5rpy3tgaJpZM4MpR9G . https://github.com/notifications/beacon/AZcghuuhD-eRYTdEN7q3IsaoV8W0AMhQks5rpy3tgaJpZM4MpR9G.gif

manodeep commented 7 years ago

Fantastic :). Closing this issue then.

tevslin commented 7 years ago

I should’ve mention that I didn’t make the env, just added –user to the python command as you suggested.

From: Manodeep Sinha [mailto:notifications@github.com] Sent: Tuesday, March 28, 2017 7:22 PM To: manodeep/Corrfunc Corrfunc@noreply.github.com Cc: tevslin tom@evslin.com; Mention mention@noreply.github.com Subject: Re: [manodeep/Corrfunc] Cant install on SDSS Jupyter notebook (#113)

Fantastic :). Closing this issue then.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/manodeep/Corrfunc/issues/113#issuecomment-289934891 , or mute the thread https://github.com/notifications/unsubscribe-auth/AZcghvL6VBeJ0l2tFdD5jBjgdtT_30Tdks5rqZYtgaJpZM4MpR9G . https://github.com/notifications/beacon/AZcghkE8keDIChm2G9PvlbplSKQY1vtfks5rqZYtgaJpZM4MpR9G.gif