dspsr and dspsr-bl missing symbols

gijzelaerr commented 5 years ago

@telegraphic reported here https://github.com/UCBerkeleySETI/dspsr/issues/4 that the dspsr and dspsr-bl packages are missing symbols:

(obs) obs@bls2:~/tiddalik/celery/apps/dspsr$ singularity shell --bind /datax3 dspsr.simg
Singularity: Invoking an interactive shell within container...

Singularity dspsr.simg:~/tiddalik/celery/apps/dspsr> cd /datax3/holding_8bit/
Singularity dspsr.simg:/datax3/holding_8bit> dspsr -A -L 10 spliced_blc000102030405_guppi_57776_55564_18726585_J0835-4510_0001.gpuspec.0001.fil
dspsr: symbol lookup error: dspsr: undefined symbol: _ZN6Pulsar18IntegrationManager6resizeEjb

Does installing psrcat solve the issue?

telegraphic commented 5 years ago

For bl-dspsr, installing psrchive and psrcat does not solve the issue.

For dspsr, the issue is different: I believe the issue is that if psrcat is not installed/found at compile time, an IFDEF in the C code will fail, and it will try and use a really old piece of code (like decades old) called psrinfo instead.

gijzelaerr commented 5 years ago

Can you point me to a small fil file so I can automatically replicate this issue?

Op do 15 nov. 2018 01:59 schreef Danny Price <notifications@github.com:

For bl-dspsr, installing psrchive and psrcat does not solve the issue.

For dspsr, the issue is different: I believe the issue is that if psrcat is not installed/found at compile time, an IFDEF in the C code will fail, and it will try and use a really old piece of code (like decades old) called psrinfo instead.

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/kernsuite/packaging/issues/169#issuecomment-438818688, or mute the thread https://github.com/notifications/unsubscribe-auth/AAT6pBWfvR3axmZvpwE_ikYF6qn6KgTNks5uvIXZgaJpZM4YdoFp .

telegraphic commented 5 years ago

How about this one: https://github.com/ewanbarr/sigpyproc/blob/master/examples/tutorial.fil

gijzelaerr commented 5 years ago

The current DSPSR from kern-dev gives:

$  dspsr -A -L 10  tutorial.fil

Error::stack
        dsp::File::create
Error::InvalidParam
Error::message
        tutorial.fil not a recognized file format
        6 registered Formats: Dummy DADA FITSFile SigProc MultiFile Multiplex

Is it missing a file format?

gijzelaerr commented 5 years ago

i've recompiled the latest dspsr and enabled all plugins that compile but now i get a buffer overflow https://github.com/kernsuite/packaging/issues/172

telegraphic commented 5 years ago

Hi @gijzelaerr -- I decided to try and install dspsr from source in a container, to figure out what various flags in the pulsar software stack were required for a functional version.

Here's my singularity file (it copies in source code for psrcat, tempo, tempo2, psrchive and dspsr (Paul Demorest's branch on github)):

Bootstrap: docker
From: kernsuite/seti:latest

%files
    ./src /src

%post
    docker-apt-install make cmake g++ csh gfortran tcsh vim automake libtool libreadline-dev curl wget libxml2-dev libcfitsio-dev libfftw3-dev bl-sigproc libcairo2-dev swig libgsl-dev libomp-dev

    # Install jupyter lab and ipython
    docker-apt-install python-setuptools python-pip
    pip install --upgrade setuptools wheel
    pip install numpy pandas cython astropy matplotlib
    pip install --only-binary=scipy scipy
    pip install jupyter ipython jupyterlab
    pip install git+https://github.com/ucberkeleyseti/blimpy

    # psrcat install
    if [ ! -d /usr/local/psrcat ]; then
        mkdir /usr/local/psrcat; cp /src/psrcat/*.db /usr/local/psrcat;
    fi
    cd /src/psrcat; ./makeit; cp psrcat /usr/local/bin/;

    # Tempo install
    cd /src/tempo; ./prepare; ./configure; make -j8; make install; hash -r;

    # Tempo2 install
    cd /src/tempo2; ./bootstrap; ./configure; make -j8; make install; make plugins; make plugins-install; hash -r;
    if [ ! -d /usr/local/tempo2 ]; then
        cp -r /src/tempo2/T2runtime /usr/local/tempo2;
    fi
    export TEMPO2=/usr/local/tempo2;

    # PSRCHIVE install
    cd /src/psrchive; ./bootstrap; ./configure --with-psrcat=psrcat --enable-shared;
    make clean; make -j8; make install; hash -r;

    # DSPSR install
    echo "apsr asp bcpm bpsr caspsr cpsr cpsr2 dummy fits kat lbadr lbadr64 lump lwa sigproc ska1" > /src/dspsr-demorest/backends.list;
    cd /src/dspsr-demorest; ./bootstrap; ./configure; make -j8; make install; hash -r;

%environment
    export PSRCAT_FILE=/usr/local/psrcat/psrcat.db
    export PGPLOT_DIR=/usr/local
    export LD_LIBRARY_PATH=/usr/local/lib
    export TEMPO2=/usr/local/tempo2

I think my particular issue was actually that PSRCHIVE needed to be configured with the ./configure --with-psrcat=psrcat flag, otherwise if it doesn't find PSRCAT during installation it will set some IFDEFS to use a piece of software that hasn't existed for 20 years.

PSRCHIVE has python bindings, which are useful, and get generated only if ./configure is run with --enable-shared.

PSRCHIVE also freaks if it doesn't see the TEMPO2 environment variable.

Finally getting to DSPSR, I had trouble getting it to work without making a backends.list file that doesn't include every single option.

Finally, to use the code some environment variables are obligatory as far as I can tell.

gijzelaerr commented 5 years ago

Ok lets try to move all this into the psrchive KERN package. Are we talking now about bl-dspsr or dspsr?

about the psrcat flag: https://github.com/kernsuite/packaging/issues/174
about the python libraries, they are already included in the debian package (python-psrchive)
About environment variables, that should be an issue during compile time but it is annoying run time. We should modify/patch the source to point to the debian locations.
About the backends.list, why can't you supply a sublist? For bl-dspsr i just use the default, for dspsr I enable everything that compiles:

https://github.com/kernsuite-debian/bl-dspsr/blob/master/debian/rules https://github.com/kernsuite-debian/dspsr/blob/master/debian/rules

gijzelaerr commented 5 years ago

Why dont you try to use the TEMPO2 and psrchive KERN packages? If they don't work something is wrong and you should open an issue.

gijzelaerr commented 5 years ago

what is the format of that tutorial file?

λ  ~/Work/bl-dspsr/Signal/Pulsar/dspsr -A -L 1 -N J0835-4510 -k PKS tutorial.fil
dspsr: Source name set to J0835-4510

Error::stack
        dsp::File::create
Error::InvalidParam
Error::message
        tutorial.fil not a recognized file format
        22 registered Formats: Dummy DADA ASP BCPM CPSR CPSR2 GMRT GMRT GMRTFilterbank SMRO LBADR64 Mark4 Mark5 Maxim MWA Pdev PMDAQ PuMa2 S2 SigProc MultiFile Multiplex

gijzelaerr commented 5 years ago

Aarch, I directly saved that link, so it actually was a HTML file...

gijzelaerr commented 5 years ago

@telegraphic Ok I get the same error now (psrinfo: not found). I can't figure out what package should contain psrinfo? Not psrcat. The internet doesn't help much.

gijzelaerr commented 4 years ago

Unfortunately, due to time and budget constraints we are dropping pulsar packages from KERN-6.

kernsuite / packaging

dspsr and dspsr-bl missing symbols #169