Closed ledwar04 closed 8 months ago
MelissaGraham
commented
1 year ago Hi @ledwar04, I asked around a bit and one RSP team member offered a quick guess: LD_LIBRARY_PATH has to be set somewhere in local_setup, not (just) bashrc. Perhaps try that as a first test?
frossie
commented
1 year ago Did @ledwar04 do this test?
ledwar04
commented
1 year ago Hi,
Yes, I did do this test, it was unsuccessful.
l.
From: Frossie @.> Sent: Thursday, May 25, 2023 10:55 AM To: rubin-dp0/Support @.> Cc: Louise Olivia Violet Edwards @.>; Mention @.> Subject: Re: [rubin-dp0/Support] Unable to point to MultiNest package libraries within notebook (Issue #39)
Did @ledwar04https://github.com/ledwar04 do this test?
— Reply to this email directly, view it on GitHubhttps://github.com/rubin-dp0/Support/issues/39#issuecomment-1563292543, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AJCBEBFYKOPVWKBX7WW4GGTXH6MJJANCNFSM6AAAAAAS62TRPA. You are receiving this because you were mentioned.Message ID: @.***>
DouglasLeeTucker
commented
11 months ago Hi Louise,
@leannep just created a very nice notebook to deal with this issue. I reviewed it, going step-by-step, and it works nicely. It can be found here: https://github.com/rubin-dp0/tutorial-notebooks/blob/main/DP02_20_Working_with_user_packages.ipynb
I know I am going to be making use of this method myself!
I hope this helps! Let us know if it doesn't!
Best regards, Douglas
PS: I believe @ryanlauastro has also sent you a direct message about this same notebook.
DouglasLeeTucker
commented
11 months ago Oops -- that link is broken. Try this instead: https://github.com/rubin-dp0/tutorial-notebooks/blob/main/DP02_11_Working_with_user_packages.ipynb
It will soon be moved from main to prod and appear in your https:/data.lsst.cloud/ notebook directory.
If you can't access that info just yet, here is a summary of Leanne's notebook:
Open a terminal in the RSP to execute the following commands:
pip install --user bagpipes mkdir ~/local cd ~/local git clone https://github.com/JohannesBuchner/MultiNest cd MultiNest/build cmake .. make
The LD_LIBRARY_PATH environment variable must now be updated. To test in the RSP terminal window, try:
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:${HOME}/local/MultiNest/lib python -c 'import bagpipes as pipes'
This should work, or only result in a warning about LaTeX issues.
If it does, you can update your .bashrc file and your notebooks/.user_setups file by adding the following line to each:
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:${HOME}/local/MultiNest/lib
You will need to log out and log back in again before you trying using bagpipes in an RSP Jupyter notebook.
Once you do, you can run the following in an RSP notebook:
import os print(os.getenv('LD_LIBRARY_PATH')) import bagpipes as pipes
I tried it myself, and it worked; so give it a try yourself!
Let us know if you continue to have issues.
Thanks!
MelissaGraham
commented
11 months ago Thanks Douglas.
@ledwar04, whenever you get back to this, if the above advice works let us know so we can close this ticket.
ledwar04
commented
11 months ago Oh my gosh!
Wow, this will be a super duper helpful tutorial - my group has been pip installing packages all summer 😉
I have moved on to using Dense Basis (which installed really well) instead of Bagpipes - but I can take a moment to test this, and close the ticket - absolutely! Hopefully okay if it takes me a week or two to get to it - l.
From: Douglas L. Tucker @.> Sent: Monday, July 31, 2023 5:05 PM To: rubin-dp0/Support @.> Cc: Louise Olivia Violet Edwards @.>; Mention @.> Subject: Re: [rubin-dp0/Support] Unable to point to MultiNest package libraries within notebook (Issue #39)
Oops -- that link is broken. Try this instead: https://github.com/rubin-dp0/tutorial-notebooks/blob/main/DP02_11_Working_with_user_packages.ipynb
It will soon be moved from main to prod and appear in your https:/data.lsst.cloud/ notebook directory.
If you can't access that info just yet, here is a summary of Leanne's notebook:
Open a terminal in the RSP to execute the following commands:
pip install --user bagpipes mkdir ~/local cd ~/local git clone https://github.com/JohannesBuchner/MultiNest cd MultiNest/build cmake .. make
The LD_LIBRARY_PATH environment variable must now be updated. To test in the RSP terminal window, try:
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:${HOME}/local/MultiNest/lib python -c 'import bagpipes as pipes'
This should work, or only result in a warning about LaTeX issues.
If it does, you can update your .bashrc file and your notebooks/.user_setups file by adding the following line to each: export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:${HOME}/local/MultiNest/lib
You will need to log out and log back in again before you trying using bagpipes in an RSP Jupyter notebook. Once you do, you can run the following in an RSP notebook:
import os print(os.getenv('LD_LIBRARY_PATH')) import bagpipes as pipes
I tried it myself, and it worked; so give it a try yourself! Let us know if you continue to have issues. Thanks!
— Reply to this email directly, view it on GitHubhttps://github.com/rubin-dp0/Support/issues/39#issuecomment-1659369884, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AJCBEBHOVIXT2SXNL5SZXXLXTBB4DANCNFSM6AAAAAAS62TRPA. You are receiving this because you were mentioned.Message ID: @.***>
ledwar04
commented
11 months ago Hmm, unfortunately this did not work.
I got the following error when I try the command:
python -c 'import bagpipes as pipes'
from part 1.3 of the tutorial.
The error is below:
@.*** ~]$ python -c 'import bagpipes as pipes'
Traceback (most recent call last):
File "
Louise
From: Douglas L. Tucker @.> Sent: Monday, July 31, 2023 5:05 PM To: rubin-dp0/Support @.> Cc: Louise Olivia Violet Edwards @.>; Mention @.> Subject: Re: [rubin-dp0/Support] Unable to point to MultiNest package libraries within notebook (Issue #39)
Oops -- that link is broken. Try this instead: https://github.com/rubin-dp0/tutorial-notebooks/blob/main/DP02_11_Working_with_user_packages.ipynb
It will soon be moved from main to prod and appear in your https:/data.lsst.cloud/ notebook directory.
If you can't access that info just yet, here is a summary of Leanne's notebook:
Open a terminal in the RSP to execute the following commands:
pip install --user bagpipes mkdir ~/local cd ~/local git clone https://github.com/JohannesBuchner/MultiNest cd MultiNest/build cmake .. make
The LD_LIBRARY_PATH environment variable must now be updated. To test in the RSP terminal window, try:
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:${HOME}/local/MultiNest/lib python -c 'import bagpipes as pipes'
This should work, or only result in a warning about LaTeX issues.
If it does, you can update your .bashrc file and your notebooks/.user_setups file by adding the following line to each: export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:${HOME}/local/MultiNest/lib
You will need to log out and log back in again before you trying using bagpipes in an RSP Jupyter notebook. Once you do, you can run the following in an RSP notebook:
import os print(os.getenv('LD_LIBRARY_PATH')) import bagpipes as pipes
I tried it myself, and it worked; so give it a try yourself! Let us know if you continue to have issues. Thanks!
— Reply to this email directly, view it on GitHubhttps://github.com/rubin-dp0/Support/issues/39#issuecomment-1659369884, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AJCBEBHOVIXT2SXNL5SZXXLXTBB4DANCNFSM6AAAAAAS62TRPA. You are receiving this because you were mentioned.Message ID: @.***>
DouglasLeeTucker
commented
11 months ago Dear Louise,
You might need to log out and re-login to the RSP for it to work. If that doesn’t work, we can try to go at an upcoming Stack Club or DP0 meeting, or maybe even an ad hoc Zoom session. Let us know if you still have trouble with it! Thanks!
Best regards, Douglas
From: Louise Edwards @.> Date: Tuesday, August 1, 2023 at 3:03 PM To: rubin-dp0/Support @.> Cc: Douglas L Tucker @.>, Comment @.> Subject: Re: [rubin-dp0/Support] Unable to point to MultiNest package libraries within notebook (Issue #39) Hmm, unfortunately this did not work.
I got the following error when I try the command:
python -c 'import bagpipes as pipes'
from part 1.3 of the tutorial.
The error is below:
@.*** ~]$ python -c 'import bagpipes as pipes'
Traceback (most recent call last):
File "
Louise
From: Douglas L. Tucker @.> Sent: Monday, July 31, 2023 5:05 PM To: rubin-dp0/Support @.> Cc: Louise Olivia Violet Edwards @.>; Mention @.> Subject: Re: [rubin-dp0/Support] Unable to point to MultiNest package libraries within notebook (Issue #39)
Oops -- that link is broken. Try this instead: https://github.com/rubin-dp0/tutorial-notebooks/blob/main/DP02_11_Working_with_user_packages.ipynbhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_rubin-2Ddp0_tutorial-2Dnotebooks_blob_main_DP02-5F11-5FWorking-5Fwith-5Fuser-5Fpackages.ipynb&d=DwQFaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=R-57Mm7OJX1IswbY4ezVTQy0X_wNb9wJndrdxKYoMiA&m=YZ7MNbpYOvk3U9daA7p8P0ZJGIGUlQaMAI8QYuwIdwS183EUqlwJyJw8-I3_J78I&s=bc0YkDD7OIlYz68nNL5PgGgMARz0vvpHFd5WeHm3lPw&e=
It will soon be moved from main to prod and appear in your https:/data.lsst.cloud/https://urldefense.proofpoint.com/v2/url?u=https-3A__data.lsst.cloud_&d=DwQFaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=R-57Mm7OJX1IswbY4ezVTQy0X_wNb9wJndrdxKYoMiA&m=YZ7MNbpYOvk3U9daA7p8P0ZJGIGUlQaMAI8QYuwIdwS183EUqlwJyJw8-I3_J78I&s=FNTGHc2NPGRcGUjBEtIU-oOK8n8CBHXR-pzjcoEVa6c&e= notebook directory.
If you can't access that info just yet, here is a summary of Leanne's notebook:
Open a terminal in the RSP to execute the following commands:
pip install --user bagpipes mkdir ~/local cd ~/local git clone https://github.com/JohannesBuchner/MultiNesthttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_JohannesBuchner_MultiNest&d=DwQFaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=R-57Mm7OJX1IswbY4ezVTQy0X_wNb9wJndrdxKYoMiA&m=YZ7MNbpYOvk3U9daA7p8P0ZJGIGUlQaMAI8QYuwIdwS183EUqlwJyJw8-I3_J78I&s=9buHprjmVDLnB-jdKvx0-3A1Tiy2NrlO6MvkBjWXsNA&e= cd MultiNest/build cmake .. make
The LD_LIBRARY_PATH environment variable must now be updated. To test in the RSP terminal window, try:
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:${HOME}/local/MultiNest/lib python -c 'import bagpipes as pipes'
This should work, or only result in a warning about LaTeX issues.
If it does, you can update your .bashrc file and your notebooks/.user_setups file by adding the following line to each: export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:${HOME}/local/MultiNest/lib
You will need to log out and log back in again before you trying using bagpipes in an RSP Jupyter notebook. Once you do, you can run the following in an RSP notebook:
import os print(os.getenv('LD_LIBRARY_PATH')) import bagpipes as pipes
I tried it myself, and it worked; so give it a try yourself! Let us know if you continue to have issues. Thanks!
— Reply to this email directly, view it on GitHubhttps://github.com/rubin-dp0/Support/issues/39#issuecomment-1659369884https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_rubin-2Ddp0_Support_issues_39-23issuecomment-2D1659369884-253E&d=DwQFaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=R-57Mm7OJX1IswbY4ezVTQy0X_wNb9wJndrdxKYoMiA&m=YZ7MNbpYOvk3U9daA7p8P0ZJGIGUlQaMAI8QYuwIdwS183EUqlwJyJw8-I3_J78I&s=6Y2Y9YrlB2p7PEnOffc0xcZqvlD-QAhvLKE1Zw67Uvo&e=, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AJCBEBHOVIXT2SXNL5SZXXLXTBB4DANCNFSM6AAAAAAS62TRPAhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AJCBEBHOVIXT2SXNL5SZXXLXTBB4DANCNFSM6AAAAAAS62TRPA-253E&d=DwQFaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=R-57Mm7OJX1IswbY4ezVTQy0X_wNb9wJndrdxKYoMiA&m=YZ7MNbpYOvk3U9daA7p8P0ZJGIGUlQaMAI8QYuwIdwS183EUqlwJyJw8-I3_J78I&s=iYY7-FNx1QSeYfcRBJqdq4Ocj-LYAcw_0-EizawWWJY&e=. You are receiving this because you were mentioned.Message ID: @.***>
— Reply to this email directly, view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_rubin-2Ddp0_Support_issues_39-23issuecomment-2D1660999596&d=DwMFaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=R-57Mm7OJX1IswbY4ezVTQy0X_wNb9wJndrdxKYoMiA&m=YZ7MNbpYOvk3U9daA7p8P0ZJGIGUlQaMAI8QYuwIdwS183EUqlwJyJw8-I3_J78I&s=IuYpFbvr-1_pLY55_cyfIV3w_rpVTYGoFrcF_PTN6No&e=, or unsubscribehttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AARAHOJDLH4UYJ6IERC3MZ3XTFOJFANCNFSM6AAAAAAS62TRPA&d=DwMFaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=R-57Mm7OJX1IswbY4ezVTQy0X_wNb9wJndrdxKYoMiA&m=YZ7MNbpYOvk3U9daA7p8P0ZJGIGUlQaMAI8QYuwIdwS183EUqlwJyJw8-I3_J78I&s=3aGG5qqOpFc41FIQjptp02IWL9dZamk6ja3sIOKCJtU&e=. You are receiving this because you commented.Message ID: @.***>
MelissaGraham
commented
8 months ago @ledwar04 and @DouglasLeeTucker, could you confirm that this issue has been resolved and that this ticket can be closed?
ledwar04
commented
8 months ago Yes, I confirm. l.
From: Melissa Graham @.> Sent: Thursday, October 5, 2023 10:58 AM To: rubin-dp0/Support @.> Cc: Louise Olivia Violet Edwards @.>; Mention @.> Subject: Re: [rubin-dp0/Support] Unable to point to MultiNest package libraries within notebook (Issue #39)
@ledwar04https://github.com/ledwar04 and @DouglasLeeTuckerhttps://github.com/DouglasLeeTucker, could you confirm that this issue has been resolved and that this ticket can be closed?
— Reply to this email directly, view it on GitHubhttps://github.com/rubin-dp0/Support/issues/39#issuecomment-1749394524, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AJCBEBE3ECAYLQCXE4YTGQLX53YLTAVCNFSM6AAAAAAS62TRPCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONBZGM4TINJSGQ. You are receiving this because you were mentioned.Message ID: @.***>
MelissaGraham
commented
8 months ago Thank you! Closing issue.
DouglasLeeTucker
commented
8 months ago Thanks, @MelissaGraham ! Thanks, @ledwar04 !
Describe the bug Ultimately, I am trying to use Bagpipes, an SED fitting code. https://bagpipes.readthedocs.io/en/latest/ which installs well with pip. However, it requires MuiltNest (and PyMultiNest) for full fuctionality. After installing several dependancies and brew, I am able to successfully install MultiNest, PyMultiNest and Bagpipes using the command line and point to the required libraries (e.g. libmultinest.so, which I put in /home/ledwar04/local/lib/). However, when I open up a Notebook, I get the following error:
ERROR: Could not load MultiNest library "libmultinest.so" ERROR: You have to build it first, ERROR: and point the LD_LIBRARY_PATH environment variable to it! ERROR: manual: http://johannesbuchner.github.com/PyMultiNest/install.html
ERROR: Could not load MultiNest library: libmultinest.so ERROR: You have to build MultiNest, ERROR: and point the LD_LIBRARY_PATH environment variable to it! ERROR: manual: http://johannesbuchner.github.com/PyMultiNest/install.html
problem: libmultinest.so: cannot open shared object file: No such file or directory Bagpipes: PyMultiNest import failed, fitting will be unavailable.
To Reproduce Steps to reproduce the behavior:
Note: I use make install instead of sudo make install and manually edit them make file to point to /home/ledwar04/local instead of /usr/local, since I don't have permissions to do that. I manually point to the MultiNest libraries by adding the following line to my bash, to point to where I put them: export LD_LIBRARY_PATH=/home/ledwar04/local/lib/:$LD_LIBRARY_PATH
Install PyMultiNest with the following commands: git clone https://github.com/JohannesBuchner/PyMultiNest.git python setup.py install --user export PATH=$PATH:$HOME/.local/bin/
Install Bagpipes following instructions: here https://bagpipes.readthedocs.io/en/latest/: pip install bagpipes
Expected behavior I expected the jupyter notebooks commands to recongnize the MultiNest install (afterall, it successfully recognizes the babpipes install) to install, and find the libraries, since in the terminal I can see they are pointing to the right place by doing and echo command:
(lsst-scipipe-4.1.0) [ledwar04@nb-ledwar04 ~]$ echo $LD_LIBRARY_PATH /home/ledwar04/local/lib/::/usr/local/lib64:/usr/local/lib64
Screenshots This is a screenshot of how the terminal knows where the library files are:
This is a screenshot of the error I get that says bagpipes doesn't know where the libraries are:
This is a screenshot that shows, when I start up the notebook, the libraries aren't included in the path, eventhough I have defined them within the terminal. Interestingly, it also shows that even after I point the to the libraries within the notebook Bagpipes still can't find them:
URL If applicable, cut and paste the URL from your browser at which the issue occurred https://data.lsst.cloud/nb/user/ledwar04/lab/tree/SEDfitting
Additional context Bagpipes itself works well, except for the functionalities that require MultiNest (of course, that's the fun part I need to use to do the science). There were quite a few other packages I needed to install to get MultiNest up and running (e.g. brew, lapack), gcc libraries. I also tried making my own conda environment, and installing there, but with no success. Though, I couldn't figure out how to ensure the notebooks I was starting up were starting up in a particular conda environment. In any case, I'll be wanting to use Bagpipes while also having the rubin-obs packages.