Errors in running "test_presto_python"

[Adirana27]

Hi There, Following error comes up whenever I run " test_presto_python". I couldnt figure out what the problem is. I'm newbie to presto. Thanks

**adeel@adeel-HP-Pavilion-x360-Convertible-14-dh1xxx:~/pulsar_software/presto/tests$ python3 test_presto_python.py Traceback (most recent call last): File "/home/adeel/pulsar_software/presto/tests/test_presto_python.py", line 4, in from presto import presto ImportError: cannot import name 'presto' from 'presto' (/home/adeel/.local/lib/python3.8/site-packages/presto/init.py)

---

[scottransom]

Sorry for the delay. This is because the python portion of PRESTO is not installed correctly, or that you are using a different python than the one where you installed PRESTO.

Did you install PRESTO from scratch? If so, what is your OS?

Also, please do the following:

1. Make sure that your python environment is the one that you want to use
2. cd $PRESTO

3. pip install . # note the ending "."!

   It should end by saying:

   Successfully built presto
   Installing collected packages: presto
   Attempting uninstall: presto
   Found existing installation: presto 4.0
   Uninstalling presto-4.0:
     Successfully uninstalled presto-4.0
   Successfully installed presto-4.0

[Jiaying-Xu-Zhejiang]

Hi, I have the same problem about the "python tests/test_presto_python.py". I'm sure that I install the PRESTO and no error remind me. I have no idea what's wrong. Thanks.

(base) administrator@administrator-System-Product-Name:~/Software/presto$ pip install .
Processing /home/administrator/Software/presto
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
...........................
Successfully built presto
Installing collected packages: presto
  Attempting uninstall: presto
    Found existing installation: presto 4.0
    Uninstalling presto-4.0:
      Successfully uninstalled presto-4.0
Successfully installed presto-4.0
(base) administrator@administrator-System-Product-Name:~/Software/presto$ python tests/test_presto_python.py 
Traceback (most recent call last):
  File "/home/administrator/Software/presto/tests/test_presto_python.py", line 3, in <module>
    from presto import presto
ImportError: cannot import name 'presto' from 'presto' (/home/administrator/Software/presto/python/presto/__init__.py)

[scottransom]

Did you install everything as administrator? Can you check to see that the presto python modules are installed somewhere in your (as a user, not as administrator) PYTHONPATH? (not necessarily based on the environment variable, but based on where python looks for packages?). Also, make sure that the pip that you used corresponds to the same python that you are using to run the test script.

[Jiaying-Xu-Zhejiang]

I installed all my softwares as administrator (the only account on my Ubuntu20.04). And I put the presto python path into the environment variables PYTHONPATH="/home/administrator/Software/presto/python". Does python go anywhere else to find packages besides environment variables? And I run the pip under a miniconda environment.

[JPGlaser]

Just to confirm here, @scottransom , I am also running into this issue on Link. I have everything installed just fine with the C scripts, but all of the Python scripts break with this importing of presto. pip is showing that it is installed just fine with all of the environment variables set correctly.

I will note that the init.py are all empty and there doesn't look to be .py scripts inside the directories aside from the main one. I am quite confused on what the setup.py script actually does or if there is a make command that we are missing in the install instructions?

~ Joe G.

[scottransom]

@Jiaying-Xu-Zhejiang Did the PRESTO python code actually get installed into that PYTHONPATH directory? Note that you should not need to use PYTHONPATH at all, anymore, and I would recommend unsetting that variable. If you want to install the python modules in your user directory, you should do "pip install --user ." when you install. And note that pip allows you to specify specific install paths if you need/want them (but in those cases, you might need the PYTHONPATH variable set). Note that these are not at all PRESTO-specific issues, but python installation issues, in general.

[scottransom]

@JPGlaser Similar to my last comment, how exactly did you run the pip command? And are in you the python environment you want the users to be in when you did that? If you are in a python virtual environment that others will use (including a conda environment), then simply "pip install ." should install the PRESTO python stuff into the site-packages directory of that virtual environment. And I highly recommend using a virtual environment of some kind!

[scottransom]

Note you can also get more information on the install (including were things are installed and where it is finding dependencies) by doing pip install -v . in $PRESTO. Here is what I see when I look in the proper virtual environment directory after running the install:

❯ ls -altr /home/sransom/python_venvs/py3.10.4/lib/python3.10/site-packages/presto/     11:45:16
.rw-rw-r-- sransom sransom  29 KB Fri Jul  1 11:44:27 2022  Pgplot.py
.rw-rw-r-- sransom sransom 5.8 KB Fri Jul  1 11:44:27 2022  bestprof.py
.rw-rw-r-- sransom sransom  12 KB Fri Jul  1 11:44:27 2022  barycenter.py
.rw-rw-r-- sransom sransom   0 B  Fri Jul  1 11:44:27 2022  __init__.py
.rw-rw-r-- sransom sransom 3.9 KB Fri Jul  1 11:44:27 2022  simple_roots.py
.rw-rw-r-- sransom sransom 7.0 KB Fri Jul  1 11:44:27 2022  sigproc.py
.rw-rw-r-- sransom sransom  53 KB Fri Jul  1 11:44:27 2022  sifting.py
.rw-rw-r-- sransom sransom  16 KB Fri Jul  1 11:44:27 2022  rfifind.py
.rw-rw-r-- sransom sransom 3.6 KB Fri Jul  1 11:44:27 2022  residuals.py
.rw-rw-r-- sransom sransom  16 KB Fri Jul  1 11:44:27 2022  pypsrcat.py
.rw-rw-r-- sransom sransom  27 KB Fri Jul  1 11:44:27 2022  psrfits.py
.rw-rw-r-- sransom sransom  73 KB Fri Jul  1 11:44:27 2022  psr_utils.py
.rw-rw-r-- sransom sransom 1.3 KB Fri Jul  1 11:44:27 2022  psr_constants.py
.rw-rw-r-- sransom sransom  40 KB Fri Jul  1 11:44:27 2022  prepfold.py
.rw-rw-r-- sransom sransom  10 KB Fri Jul  1 11:44:27 2022  polycos.py
.rw-rw-r-- sransom sransom  10 KB Fri Jul  1 11:44:27 2022  parfile.py
.rw-rw-r-- sransom sransom  86 KB Fri Jul  1 11:44:27 2022  mpfit.py
.rw-rw-r-- sransom sransom 4.3 KB Fri Jul  1 11:44:27 2022  kuiper.py
.rw-rw-r-- sransom sransom  48 KB Fri Jul  1 11:44:27 2022  injectpsr.py
.rw-rw-r-- sransom sransom 6.8 KB Fri Jul  1 11:44:27 2022  infodata.py
.rw-rw-r-- sransom sransom  11 KB Fri Jul  1 11:44:27 2022  filterbank.py
.rw-rw-r-- sransom sransom  21 B  Fri Jul  1 11:44:27 2022  fftfit.py
.rw-rw-r-- sransom sransom  18 KB Fri Jul  1 11:44:27 2022  events.py
.rw-rw-r-- sransom sransom 1.9 KB Fri Jul  1 11:44:27 2022  cosine_rand.py
.rw-rw-r-- sransom sransom  20 KB Fri Jul  1 11:44:27 2022  cosine_rand.json
.rw-rw-r-- sransom sransom  10 KB Fri Jul  1 11:44:27 2022  binary_psr.py
.rw-rw-r-- sransom sransom  17 KB Fri Jul  1 11:44:27 2022  waterfaller.py
.rw-rw-r-- sransom sransom  13 KB Fri Jul  1 11:44:27 2022  spectra.py
.rw-rw-r-- sransom sransom 6.3 KB Fri Jul  1 11:44:27 2022  sinc_interp.py
drwxrwxr-x sransom sransom 4.0 KB Fri Jul  1 11:44:27 2022  .
drwxrwxr-x sransom sransom 4.0 KB Fri Jul  1 11:44:27 2022  ppgplot
drwxrwxr-x sransom sransom 4.0 KB Fri Jul  1 11:44:27 2022  presto
drwxrwxr-x sransom sransom 4.0 KB Fri Jul  1 11:44:27 2022  singlepulse
drwxrwxr-x sransom sransom 4.0 KB Fri Jul  1 11:44:27 2022  __pycache__
drwxrwxr-x sransom sransom  20 KB Fri Jul  1 11:44:27 2022  ..

You should see something very similar unless the compilation of the C-code python wrappers failed, which pip install -v . should show you.

[scottransom]

Note that there should also be 3 shared library files located in the top-level site-packages directory:

.rwxrwxr-x sransom sransom 150 KB Fri Jul  1 11:44:27 2022  /home/sransom/python_venvs/py3.10.4/lib/python3.10/site-packages/_ppgplot.cpython-310-x86_64-linux-gnu.so
.rwxrwxr-x sransom sransom 146 KB Fri Jul  1 11:44:27 2022  /home/sransom/python_venvs/py3.10.4/lib/python3.10/site-packages/_fftfit.cpython-310-x86_64-linux-gnu.so
.rwxrwxr-x sransom sransom 760 KB Fri Jul  1 11:44:27 2022  /home/sransom/python_venvs/py3.10.4/lib/python3.10/site-packages/_presto.cpython-310-x86_64-linux-gnu.so

Those come from the C-wrappers for the various libraries that PRESTO uses.

[JPGlaser]

So we are not using venvs, but rather conda environments. That said, it still should be fine with the pip install using the pip that is inside said env, which we are.

(pulsar) [sysadmin@link presto]$ which pip -a
/opt/python/3.9/envs/pulsar/bin/pip

Said sysadmin user has ownership over the /opt/ directory and all sub-directories (including where $PRESTO=/opt/pulsar/presto is set) and it seems from the following log that everything installs correctly (pip even shows that presto=4.0 has been installed in the pip and conda list).

pypresto_install_output.log

~ Joe G.

[JPGlaser]

Just a note, when installing via pip in a conda env, we end up with the following:

(pulsar) [sysadmin@link presto]$ which -a DDplan.py 
/opt/pulsar/presto/bin/DDplan.py
/opt/python/3.9/envs/pulsar/bin/DDplan.py
(pulsar) [sysadmin@link presto]$ /opt/python/3.9/envs/pulsar/bin/DDplan.py
Traceback (most recent call last):
  File "/opt/python/3.9/envs/pulsar/bin/DDplan.py", line 6, in <module>
    from presto.Pgplot import *
  File "/opt/pulsar/presto/python/presto/Pgplot.py", line 22, in <module>
    from presto import ppgplot
ImportError: cannot import name 'ppgplot' from 'presto' (/opt/pulsar/presto/python/presto/__init__.py)
(pulsar) [sysadmin@link presto]$ /opt/pulsar/presto/bin/DDplan.py
Traceback (most recent call last):
  File "/opt/pulsar/presto/bin/DDplan.py", line 6, in <module>
    from presto.Pgplot import *
  File "/opt/pulsar/presto/python/presto/Pgplot.py", line 22, in <module>
    from presto import ppgplot
ImportError: cannot import name 'ppgplot' from 'presto' (/opt/pulsar/presto/python/presto/__init__.py)

It looks like it is trying to use things from the presto/python folder rather than what has been installed into the environment.

(pulsar) [sysadmin@link presto]$ echo $PATH
/opt/pulsar/presto/bin:/opt/pulsar/tempo/bin:/opt/pulsar/bin:/opt/openmpi/4.1.4/bin:/opt/python/3.9/envs/pulsar/bin:/opt/python/3.9/condabin:/home/sysadmin/.local/bin:/home/sysadmin/bin:/usr/share/Modules/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/var/lib/snapd/snap/bin

~ Joe G.

[scottransom]

Just making sure, @JPGlaser ... You don't have PYTHONPATH set, do you? I use conda on my nimrod cluster using this same exact install methodology (including into a "global" conda environment that all of my students etc use). That install log certainly looks like things installed correctly. But a PYTHONPATH can certainly screw that up.

[scottransom]

I used to require that PYTHONPATH contain $PRESTO/python, but that is bad now and will lead to exactly what you see above.

[cherryng]

I am using a condo environment and had the same error with from presto import presto at first but it was indeed due to the fact that I had $PYTHONPATH set to another presto directory and not the one in miniconda where pip installed the files. Once unsetting $PYTHONPATH, I was able to by pass that issue. however, I got another error:

$ python tests/test_presto_python.py Testing FFT stuff... Traceback (most recent call last): File "tests/test_presto_python.py", line 13, in nx = presto.rfft(presto.rfft(x, -1), 1) AttributeError: module 'presto' has no attribute 'rfft'

i am using python3.7 and I just git cloned presto so I assume it is the latest version 4.0. any idea why it can't find rfft?

[paulray]

I assume that in that conda environment you did pip install . to install the presto python codes? Try checking that you are getting the right version. Run python in your conda env and make sure that from presto import presto works. Then try presto.__file__ and see if that shows a path you are expecting. If you are using a current presto then presto.rfft should exist after that import.