search

zoeye859 / presto_v3_singularity

How I installed Presto3.0.1 and create a singularity .simg file (download link included) from this installation.
4 stars 3 forks source link

Error in chkfopen(): No such file or directory path = 'resid2.tmp' #3

Closed yiqishangxuequ closed 4 years ago

yiqishangxuequ commented 4 years ago

Hi, Zoe! I downloaded the container from http://haoyangye.com/PSRSOFT_yhy.simg, but I encounter a problem when running it.

I input "singularity shell PSRSOFT_yhy.simg" to enter the container. Then I input "source presto_simg.sh" to export environment variables. The content of "presto_simg.sh" is below.

#!/bin/bash
export ASTROSOFT=/PSRSOFT

# OSTYPE
export OSTYPE=linux

# PGPLOT
export PGPLOT_DIR=$ASTROSOFT/pgplot
export PGPLOT_DEV=/xw

#TEMPO
export TEMPO=$ASTROSOFT/tempo

#TEMPO2
export TEMPO2=/usr/local/tempo2

# PRESTO
export PRESTO=$ASTROSOFT/presto

# PSRCAT
export PSRCAT_RUNDIR=$ASTROSOFT/psrcat_tar
export PSRCAT_FILE=$ASTROSOFT/psrcat_tar/psrcat.db

# DSPSR
export PATH=/sigproc/bin:$PATH

# LD_LIBRARY_PATH
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/lib:/usr/lib64:$PGPLOT_DIR:$ASTROSOFT/lib:$PRESTO/lib:$ASTROSOFT/fftw-3.3.8/lib:$ASTROSOFT/cfitsio-3.48/lib:
# PATH
# Some Presto executables match sigproc executables so keep separate -
# all other executables are found in $ASTROSOFT/bin
export PATH=$PATH:$ASTROSOFT/bin:$PRESTO/bin:$PGPLOT_DIR

Finally, to get the Average topocentric velocity, I run the command "prepdata -o tmp GBT_Lband_PSR.fil | grep Average" but get an error, the error is below.

Singularity> prepdata -o tmp GBT_Lband_PSR.fil | grep Average

Error in chkfopen(): No such file or directory
Singularity> prepdata -o tmp GBT_Lband_PSR.fil

So, I try "prepdata -o tmp GBT_Lband_PSR.fil", but still get an error, the error is like this:

Singularity> prepdata -o tmp GBT_Lband_PSR.fil               

           Pulsar Data Preparation Routine
    Type conversion, de-dispersion, barycentering.
                 by Scott M. Ransom

Assuming the data are SIGPROC filterbank format...
Reading SIGPROC filterbank data from 1 file:
  'GBT_Lband_PSR.fil'

    Number of files = 1
       Num of polns = 2 (summed)
  Center freq (MHz) = 1400
    Num of channels = 96
    Sample time (s) = 7.2e-05       
     Spectra/subint = 2400
   Total points (N) = 531000
     Total time (s) = 38.232        
     Clipping sigma = 6.000
   Invert the band? = False
          Byteswap? = False
     Remove zeroDM? = False

File  Start Spec   Samples     Padding        Start MJD
----  ----------  ----------  ----------  --------------------
1              0      531000           0  53010.48482638889254

Setting a 'good' output length of 537600 samples
Writing output data to 'tmp.dat'.
Writing information to 'tmp.inf'.

Generating barycentric corrections...

Error in chkfopen(): No such file or directory
   path = 'resid2.tmp'

I try using google to solve it , but I can't get answer. If you can give me some advice, I will be very grateful to you !

Liu

zoeye859 commented 4 years ago

Hi, Liu, thank you for your feedback. Sorry for not replying to you earlier.

Have you tried the two following commands? If not, try these two first. Do they produce errors?


> readfile GBT_Lband_PSR.fil
> rfifind -time 1.0 -o Lband GBT_Lband_PSR.fil

After this, try 

prepdata -nobary -o Lband_topo_DM0.00 -dm 0.0 -mask Lband_rfifind.mask -numout 530000 GBT_Lband_PSR.fil

Does this one work?

If these three work, I think your command 

prepdata -o tmp GBT_Lband_PSR.fil | grep Average

would work too.

If not, let me which order gives you which error. Thank you!

Best, Zoe

yiqishangxuequ commented 4 years ago

Thanks your reply!

I follow your commands, The first three commands work well. But when I run prepdata -o tmp GBT_Lband_PSR.fil | grep Average, It still informs me of Error in chkfopen(): No such file or directory.

When I run prepdata -o tmp GBT_Lband_PSR.fil The content below will come up.

Singularity> prepdata -o tmp GBT_Lband_PSR.fil               

           Pulsar Data Preparation Routine
    Type conversion, de-dispersion, barycentering.
                 by Scott M. Ransom

Assuming the data are SIGPROC filterbank format...
Reading SIGPROC filterbank data from 1 file:
  'GBT_Lband_PSR.fil'

    Number of files = 1
       Num of polns = 2 (summed)
  Center freq (MHz) = 1400
    Num of channels = 96
    Sample time (s) = 7.2e-05       
     Spectra/subint = 2400
   Total points (N) = 531000
     Total time (s) = 38.232        
     Clipping sigma = 6.000
   Invert the band? = False
          Byteswap? = False
     Remove zeroDM? = False

File  Start Spec   Samples     Padding        Start MJD
----  ----------  ----------  ----------  --------------------
1              0      531000           0  53010.48482638889254

Setting a 'good' output length of 537600 samples
Writing output data to 'tmp.dat'.
Writing information to 'tmp.inf'.

Generating barycentric corrections...

Error in chkfopen(): No such file or directory
   path = 'resid2.tmp'

Best, Liu

zoeye859 commented 4 years ago

Hi Liu,

I found something very interesting

This command works:

> prepdata -nobary -o ANYNAME GBT_Lband_PSR.fil

But this one does not:

> prepdata -o ANYNAME GBT_Lband_PSR.fil

Generating barycentric corrections...

Error in chkfopen(): No such file or directory
   path = 'resid2.tmp'

Can you try these two commands and tell me what results you got?

Thank you!

Best, Zoe

yiqishangxuequ commented 4 years ago

Hi Zoe,

I try these two commands, the results are the same as your

when I try the first command, the results is below:

Singularity> prepdata -nobary -o ANYNAME GBT_Lband_PSR.fil 

           Pulsar Data Preparation Routine
    Type conversion, de-dispersion, barycentering.
                 by Scott M. Ransom

Assuming the data are SIGPROC filterbank format...
Reading SIGPROC filterbank data from 1 file:
  'GBT_Lband_PSR.fil'

    Number of files = 1
       Num of polns = 2 (summed)
  Center freq (MHz) = 1400
    Num of channels = 96
    Sample time (s) = 7.2e-05       
     Spectra/subint = 2400
   Total points (N) = 531000
     Total time (s) = 38.232        
     Clipping sigma = 6.000
   Invert the band? = False
          Byteswap? = False
     Remove zeroDM? = False

File  Start Spec   Samples     Padding        Start MJD
----  ----------  ----------  ----------  --------------------
1              0      531000           0  53010.48482638889254

Setting a 'good' output length of 537600 samples
Writing output data to 'ANYNAME.dat'.
Writing information to 'ANYNAME.inf'.

Massaging the data ...

Amount Complete =  99%

Done.

Simple statistics of the output data:
             Data points written:  530400
          Padding points written:  7200
           Maximum value of data:  910.00
           Minimum value of data:  672.00
              Data average value:  785.54
         Data standard deviation:  23.57

Then, I try the second command, that is below:

Singularity> prepdata -o ANYNAME GBT_Lband_PSR.fil 

           Pulsar Data Preparation Routine
    Type conversion, de-dispersion, barycentering.
                 by Scott M. Ransom

Assuming the data are SIGPROC filterbank format...
Reading SIGPROC filterbank data from 1 file:
  'GBT_Lband_PSR.fil'

    Number of files = 1
       Num of polns = 2 (summed)
  Center freq (MHz) = 1400
    Num of channels = 96
    Sample time (s) = 7.2e-05       
     Spectra/subint = 2400
   Total points (N) = 531000
     Total time (s) = 38.232        
     Clipping sigma = 6.000
   Invert the band? = False
          Byteswap? = False
     Remove zeroDM? = False

File  Start Spec   Samples     Padding        Start MJD
----  ----------  ----------  ----------  --------------------
1              0      531000           0  53010.48482638889254

Setting a 'good' output length of 537600 samples
Writing output data to 'ANYNAME.dat'.
Writing information to 'ANYNAME.inf'.

Generating barycentric corrections...

Error in chkfopen(): No such file or directory
   path = 'resid2.tmp'

Thanks for your reply~

Best, Liu

zoeye859 commented 4 years ago

Hi Liu,

Thank you for trying these for me.

For now, maybe try to add '-nobary' to your "prepdata" command. This test data, GBT_Lband_PSR.fil, has already been corrected in terms of barycentric correction, therefore using '-nobary' can save time for you. (You can find whether your data has gone through barycentric correction or not using 'readfile'.

In the meantime, I will try to ask a person who has installed an older version of PRESTO (hence more stable) to test these two commands for me.

I Will let you know.

If you have any questions, just let me know (maybe open a new issue?).

Thanks for your patience.

Best, Zoe

yiqishangxuequ commented 4 years ago

Hi Zoe,

Thank you for your help~

Today, I tried installing PRESTO. When finished, I only input prepdata -o tmp GBT_Lband_PSR.fil | grep Average. the command worked and the response was below:

Singularity> prepdata -o tmp GBT_Lband_PSR.fil | grep Average
   Average topocentric velocity (c) = -5.697334e-05

Then, I only input prepdata -o ANYNAME GBT_Lband_PSR.fil.It also worked and the response was below:

Singularity> prepdata -o ANYNAME GBT_Lband_PSR.fil

           Pulsar Data Preparation Routine
    Type conversion, de-dispersion, barycentering.
                 by Scott M. Ransom

Assuming the data are SIGPROC filterbank format...
Reading SIGPROC filterbank data from 1 file:
  'GBT_Lband_PSR.fil'

    Number of files = 1
       Num of polns = 2 (summed)
  Center freq (MHz) = 1400
    Num of channels = 96
    Sample time (s) = 7.2e-05       
     Spectra/subint = 2400
   Total points (N) = 531000
     Total time (s) = 38.232        
     Clipping sigma = 6.000
   Invert the band? = False
          Byteswap? = False
     Remove zeroDM? = False

File  Start Spec   Samples     Padding        Start MJD
----  ----------  ----------  ----------  --------------------
1              0      531000           0  53010.48482638889254

Setting a 'good' output length of 537600 samples
Writing output data to 'ANYNAME.dat'.
Writing information to 'ANYNAME.inf'.

Generating barycentric corrections...
   Average topocentric velocity (c) = -5.697334e-05
   Maximum topocentric velocity (c) = -5.697064e-05
   Minimum topocentric velocity (c) = -5.697603e-05

Collecting and barycentering GBT_Lband_PSR.fil...

The DM of 0.00 at the barycentric observing freq of 1447.418 MHz
   causes a delay of 0.000000 seconds compared to infinite freq.
   This delay is removed from the barycented times.

Topocentric epoch (at data start) is:
   53010.48482638889

Barycentric epoch (infinite obs freq at data start) is:
   53010.48095741457

Massaging the data...

Amount Complete =  99%

Done.

Simple statistics of the output data:
             Data points written:  530430
          Padding points written:  7170
    Bins added for barycentering:  30
           Maximum value of data:  910.00
           Minimum value of data:  672.00
              Data average value:  785.54
         Data standard deviation:  23.57

It looks nice!

Best, Liu

zoeye859 commented 4 years ago

Ah, great! I am happy that you solved this!

Also, it means that I may need to reinstall PRESTO inside my container.

Can I ask you which version of PRESTO did you install?

Best, Zoe

yiqishangxuequ commented 4 years ago

Actually, I'm still confused in the process of installing, but I will share my attempts. I hole that would be useful for you.

---PRESTO version is in end of the comment---

I roughly follows the installing guidehttps://blog.csdn.net/sinat_34850075/article/details/52434526

I use apt-get *** insteading of yum *** in the guide.

Ubuntu can use the 'pgplot5'. Therefore, I use apt-get install pgplot5 insteading of the counterpart and have 'PGPLOT_DIR=/usr/lib/pgplot5'

And, in the guide, he use cd /home/m/astrosoft/presto/python then use make. But I find that I can't use make in the directory , because there is no makefile in the directory. Anyway, the step does't affect the success of installing .

Finally, I use pip3 install . in the the directory 'home/m/astrosoft/presto' to finish the installing

---PRESTO version--- I use git clone git://github.com/scottransom/presto.git to down the PRESTO. And the version of PRESTO in pip is below:

Singularity> pip3 list 
Package         Version
--------------- -----------
astropy         4.0.1.post1
biopython       1.78
certifi         2020.6.20
cycler          0.10.0
future          0.18.2
kiwisolver      1.2.0
matplotlib      3.3.2
numpy           1.19.2
pandas          1.1.2
Pillow          7.2.0
pip             20.2.3
presto          3.0.1
pyparsing       2.4.7
pySLALIB        1.0.4
python-dateutil 2.8.1
pytz            2020.1
scipy           1.5.2
setuptools      50.3.0
six             1.15.0
wheel           0.35.1

Best, Liu

zoeye859 commented 4 years ago

Thanks to a friend of mine, I reinstalled the tempo2 and the problem got resolved:

> sudo singularity shell --writable ubuntu/
> cd /PSRSOFT/
> rm -rf tempo
> git clone git://git.code.sf.net/p/tempo/tempo
> cd tempo
> ./prepare
> ./configure F77=gfortran --prefix=$ASTROSOFT CFLAGS=-fPIC FFLAGS=-fPIC
> make
> make install
> exit
> sudo singularity build PSRSOFT_yhy1.1.simg ubuntu/

I am uploading the newest PSRSOFT_yhy1.1.simg for users' download.

Best, Zoe

yiqishangxuequ commented 4 years ago

Congratulations!

I will try newest PSRSOFT_yhy1.1.simg.

Best, Liu