Pyspex is not loaded in Jupyter notebook

[sinancankara]

Hi Jelle,

I am trying to use pyspex with Jupyter notebook. For the installation I followed the AHEAD2020 School documentation. I selected the spex kernel, but I get the error in this entry: https://github.com/spex-xray/spex-help/issues/6#issue-727980389

Kernel is installed according to the comment and pyspex works when I run python.

Also I tried SciServer installation and running the notebook there, but I get the same error.

Thank you!

Best, Sinancan

[jdeplaa]

Hi Sinancan,

The reason for the error may be different for both methods. Let me give some advice for both situations separately:

SPEX on local machine:

• Could you check whether the Jupyter lab that you are running is the one from the spex conda environment? If you execute the command which jupyter, it returns the path to the jupyter program. For SPEX to work in the notebook, it should point to jupyter in /<conda installation>/envs/spex/bin/jupyter. It could be that you have a Jupyter installed elsewhere that gets priority in your path. What happens when you start jupyter like this?

  (spex) user@unix:~> $CONDA_PREFIX/bin/jupyter notebook & 

• Could you check whether SPEX works in a basic python session? Please execute the commands below:

  (spex) user@unix:~> python
  Python 3.X Anaconda
  >>> from pyspex.spex import Session()
  >>> s=Session()

  Does this work in python?

SPEX on Sciserver.org:

• On Sciserver, you need to restart the image sometimes to make it work:
  • So you close the Jupyter window and go back to the page with the list of your images.
  • There you can stop running the image by clicking the red square. Wait until the image is stopped.
  • Then, you can click the green triangle to start the image again.
  • Click on the name of the image to start Jupyter again. In this session, SPEX should work.

I hope these tips work for you. I will also add them to the school webpage.

[sinancankara]

Hi Jelle,

Thank you for the quick reply.

• On local machine:

I forgot to mention but I use WSL Ubuntu enviornment. However, the exact same problem occured when tried on regular Ubuntu. The command which jupyter gave /home/sinancan/.local/bin/jupyter. When I tried $CONDA_PREFIX/bin/jupyter notebook &, it is able to find the pyspex. However this time I get this error and couldn't find a way to resolve it:

---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
Cell In[9], line 1
----> 1 from pyspex.spex import Session
      2 s=Session()

File ~/miniconda3/envs/spex/opt/spex/python/pyspex/__init__.py:3
      1 #!/usr/bin/env python
----> 3 from .spex import Session

File ~/miniconda3/envs/spex/opt/spex/python/pyspex/spex.py:16
     12 from future import standard_library
     14 standard_library.install_aliases()
---> 16 from . import pyspex_f2py
     17 from . import data
     18 from . import model

ImportError: cannot import name 'pyspex_f2py' from partially initialized module 'pyspex' (most likely due to a circular import) (/home/sinancan/miniconda3/envs/spex/opt/spex/python/pyspex/__init__.py)

Also, it does work when I try a basic python session.

• On Sciserver.org:

I tried stopping and running the image but it does not work. I get the same error.

It also works when I try a basic python session on Sciserver.org, the problem is in Jupyter notebook session.

Thank you!

Best, Sinancan

[jdeplaa]

Dear Sinancan,

Thanks for sharing the issues that you encounter. Please try the things below and let me know the outcome.

For your local install:

• This error can also occur when there is another SPEX installation loaded at the same time. How did you install SPEX? Do you have both a regular install and an install through Anaconda? If yes, please make sure that the regularly installed SPEX is not sourced. You can check the output of the command echo $LD_LIBRARY_PATH on Linux or echo $DYLD_LIBRARY_PATH on Mac. If there are more than 1 paths to SPEX, you have multiple SPEX environments loaded (using the spexdist.sh script). If this is the case, check if you source the spexdist.sh file somewhere in your ~/.bashrc or ~/.profile settings. If yes, comment out the line and start again in a new terminal.

For the Sciserver.org install:

• Just to be sure. Did you modify the ~/.local/share/jupyter/kernels/spex/kernel.json file as described here?
• Is the Jupyter lab set to the (spex) environment? You can check this in the top-right corner of the Jupyter lab environment?
• You can try to install the nb_conda_kernels package: conda install nb_conda_kernels, and then restart the heasoft image again.

I hope some of the tips above help.

[sinancankara]

Hi Jelle,

For the Sciserver.org install:

Apologizes, I misinterpreted the modification part as if you have already modified the kernel.json. I did the modification and now it works perfectly.

For the local install:

I had done the regular installation before but the regularly installed SPEX is not sourced. echo $LD_LIBRARY_PATH gives only one enviornment /home/sinancan/miniconda3/envs/spex/opt/spex/lib. This is after executing conda activate spex

Deactivating conda gives this paths:

sinancan@LAPTOP-QLUE2VU1:~$ echo $LD_LIBRARY_PATH
/home/sinancan/heasoft-6.31.1/x86_64-pc-linux-gnu-libc2.35/lib::/home/sinancan/miniconda3/envs/spex/opt/spex/lib:/home/sinancan/miniconda3/envs/spex/opt/spex/lib

Thank you for your help.

[jdeplaa]

Great that the Sciserver installation is running now! :)

Regarding the local install:

• Given that you had a local jupyter install in ~/.local/bin/jupyter, there may be more things it picks up from the wrong environment. Could you have a look at the content of your PATH variable and (temporarily) remove the .local path from it? For example:

  (spex) user@unix:~> echo $PATH
  ~/.local/bin:~/miniconda3/env/spex/bin:~/miniconda3/env/spex/opt/spex/bin
  
  # Below you copy the part of the PATH that you want to keep:
  (spex) user@unix:~> export PATH="~/miniconda3/env/spex/bin:~/miniconda3/env/spex/opt/spex/bin"
  (spex) user@unix:~> jupyter notebook &

  Does this help?

[sinancankara]

Yes, thanks! :)

Unfortunately removing .local path did not solve the issue on local machine.