package jiveplot

[sklandrausis]

JIVE developed packages j2ms2, tConvert and jplotter could be include. Packages j2ms2 allow JIVE developed correlator SFXC output .cor file to be converted to ms files tConvert provide ms convention to fits that can be loaded into CASA, AIPS.

packages jplotter allow to create inspection plots for VLBI data correlation using ms file created from SFXC .cor file

[gijzelaerr]

Can you supply urls to the projects and installation instructions?

[sklandrausis]

Installation instructions

1. CASACORE git clone http://github.com/casacore/casacore.git cd casacore mkdir build cd build cmake -DUSE_FFTW3=ON -DCMAKE_INSTALL_PREFIX=/path/to/eee -DUSE_OPENMP=ON USE_HDDF5=ON -DBUILD_PYTHON=ON -DUSE_THREADS=ON .. make make install
2. Python casacore git clone https://github.com/casacore/python-casacore cd python-casacore sudo pip install --upgrade pip sudo pip install setuptools unset LC_CTYPE sudo python setup.py build_ext -I/path/to/eee/include -L/path/to/eee/lib install mkdir -p /path/to/eee/share/casacore/data echo '#!/bin/sh

This is the directory where the ephemerides will

be stored

measdir=/path/to/eee/share/casacore/data

attempt to move to that directory

echo cd "${measdir}" cd "${measdir}" if [ $? -ne 0 ]; then echo "$0: Failed to cd into ${measdir}" exit 1 fi

remove old data

rm -f WSRT_Measures.ztar

ftp over the new stuff

wget ftp://ftp.astron.nl/outgoing/Measures/WSRT_Measures.ztar 2>&1 if [ $? -ne 0 ]; then echo "$0: Failed to retrieve latest measures data from WSRT" exit 1 fi

unzip + untar

tar zvxf WSRT_Measures.ztar if [ $? -ne 0 ]; then echo "$0: Failed to unbzip2/extract ephemeris data!" exit 1 fi ' > PREFIX/eee/bin/update_measure_data.sh chmod +x /path/to/eee/bin/update_measure_data.sh crontab -l | { cat; echo "0 0 sun /home/root2/postCorrelation/update_measure_data.sh | mail -s 'ephemeris update results' maintainer@your.mail.domain"; } | crontab -

3. VEX parser wget http://www.jive.eu/~verkout/myvex.tar.gz tar zxvf myvex.tar.gz cd myvex make

4. j2ms2 + tConvert wget http://www.jive.eu/~verkout/jive-casa-snapshot.tar.gz tar zxvf jive-casa-snapshot.tar.gz export ROOT=/path/to/eee/src/code/jive cd code/jive ln -s implement jive vim ${ROOT}/system/locations.mak -VEXROOT := /pcint2/src/myvex-cvs +VEXROOT := /path/to/eee/src/myvex

   -FFTW3ROOT = /pcint2
   +FFTW3ROOT = /usr
   
   -CASAROOT = /pcint2/src/casacore-2.0.1-root
   +CASAROOT = /path/to/eee
   
   -FFTWROOT = /pcint2
   +FFTWROOT = /usr
   
   -CFITSIOROOT = /pcint2
   +CFITSIOROOT = /usr

   cd implement make -f Makefile B2B=64 cd ../apps/ make -f Makefile B2B=64

5. Python PGPLOT git clone https://github.com/haavee/ppgplot cd ppgplot sudo python setup.py build

6. jplotter git clone https://github.com/haavee/jiveplot.git Create ~/.pythonrc + add PYTHONSTARTUP to ~/.profile echo '# startup script for python to enable saving of interpreter history and

   enabling name completion

import needed modules

import atexit import os import readline import rlcompleter

where is history saved

historyPath = os.path.expanduser("~/.pyhistory")

handler for saving history

def save_history(historyPath=historyPath): import readline readline.write_history_file(historyPath)

read history, if it exists

if os.path.exists(historyPath): readline.read_history_file(historyPath)

register saving handler

atexit.register(save_history)

enable completion

readline.parse_and_bind('tab: complete')

cleanup

del os, atexit, readline, rlcompleter, save_history, historyPath' > ~/.pythonrc

Final step: make Python read the startup file at startup

$ echo "export PYTHONSTARTUP=${HOME}/.pythonrc" >> ~/.profile

[gijzelaerr]

all files on http://www.jive.eu/~verkout/ give a 403 error.

[sklandrausis]

i did no know that Joint Institute for VLBI in Europe web page jive.eu currenty is under construction maybe after some time theirs page will be fixed, but i can send source to email.

[sklandrausis]

I checked today all the links work.

[gijzelaerr]

@Athanaseus do you want to have a look?

[Athanaseus]

Roger that @gijzelaerr

[Athanaseus]

Hi @sklandrausis, this also implies that a VEXparser packgage should be independently build and added to KERN?

[sklandrausis]

Yes

[Athanaseus]

@sklandrausis

4. j2ms2 + tConvert

After I untar, the resulting directory code has only two directories, jive & vlbi.

According to the instructions, it seems to me only the jive is being used, let me know if I'm missing something.

My approach was to perhaps split it into two say: j2ms2-jive and j2ms2-vlbi.

[sklandrausis]

Yes to install only jive directory is used I am no sure, but probotly probable some files from VLBI is include to files in jive directory

[gijzelaerr]

I had a little chat with Harro Verkouter, which explained j2ms2 and tConvert are not publicly available yet, and those snapshots are outdated.

Only https://github.com/haavee/jiveplot is available, so let's see if we can quickly package that one. If j2ms2 and tConvert come available in the future we can maybe package them around that time.

[haavee]

Please contact me (Harro Verkouter/JIVE) about installing/packaging questions in the future.

We are the owners and original authors and would be happy to update you in great detail about dependencies and installation instructions and answer questions, if and when the software will be publicly available.

[Athanaseus]

Thank you so much @haavee.

Is there any public release that I can start working on?

[haavee]

Unfortunately no. The code was reorganized, put under git (local repo) control and the build system was rewritten to use CMake so any attempt at packaging the above linked-to version would be a wast of anyone's time. (Especially since I've already made a recipe for building a container image from the source code ...) Eventually the new code will be made publicly available, either through my personal github or via https://github.com/jive-vlbi (my money is on the latter).

[Athanaseus]

Thanks @haavee, I'll await the public announcement of that release then.

[gijzelaerr]

Hi @haavee , I think @Athanaseus is refering to jiveplot. It is available and seems to work, but there is no release made on github. We prefer to package software that has a version, so there are no ambiguities about bug tracking and issue resolving.

[haavee]

Oh sorry! I did not realize that, my bad! Up until now there have not been any releases; I can make one because it is the proper way to go.

Tada: https://github.com/haavee/jiveplot/releases

[gijzelaerr]

@Athanaseus did you make progress adding jiveplot?

[Athanaseus]

I'm working on this right now. It seems like the code is not fully ported to python3 yet? https://github.com/haavee/jiveplot/search?q=print

[gijzelaerr]

could be, @haavee what is the py3 status of jiveplot?

[haavee]

I have a Py3 branch that I've tested internally and seems to be working. However, with Python you're never quite sure you've tested all code paths, but smoke tests were passed. I don't think I've pushed that branch but certainly could give that a go.

Actually - the branch is on the public github already: https://github.com/haavee/jiveplot/tree/python23compat

[haavee]

I checked today all the links work.

We've had some issues with the webserver configuration lately, sorry for the inconvenience. The pages themselves were never at risk.

[Athanaseus]

Thanks @haavee, I'm testing the Py3 patch. Can I ask: Why jiveplot has no setup.py file? Is ppgplot a dependency or can we still use giza?

$ ./jplotter 
Traceback (most recent call last):
  File "./jplotter", line 51, in <module>
    import command, jplotter
  File "jplotter.py", line 398, in <module>
    import copy, re, math, operator, itertools, plotiterator, ppgplot, datetime, os, subprocess, numpy, parsers, imp, time
ModuleNotFoundError: No module named 'ppgplot

Solved by: pip install git+https://github.com/haavee/ppgplot.git

$ ./jplotter 
parsers.py:2655: SyntaxWarning: "is not" with a literal. Did you mean "!="?
  if tok(s).type is not 'comma':
+++++++++++++++++++++ Welcome to cli +++++++++++++++++++
$Id: command.py,v 1.16 2015-11-04 13:30:10 jive_cc Exp $
  'exit' exits, 'list' lists, 'help' helps 
jcli>

[haavee]

Hi @Athanaseus, jiveplot does not have a setup.py because I never intended it as a package, but as a standalone command line program, e.g. installed in /usr/local/bin. The structure of the project therefore never followed the proper model.

If the dependency issue(s) can be fixed with a setup.py, that would be an option, although I'm not sure how easy it would be to backport the proper project structure into the project.

As to that syntax warning: I have not seen that one but that is probably due to not having access to a modern Python environment (the latest Py3 I have access to is 3.6.5)

[Athanaseus]

@haavee I think packaging can still be possible even if it's a command-line program. It will be easy to restructure from looking at the source code. Also, this will surely resolve the dependency issues. It's not that complicated, so if you don't mind I can set it up for you.

[haavee]

Oh that would be generous if you could help with that!

[Athanaseus]

Cool. I using Py23 branch to set it up. Can you please add me to the project and I will push the branch for you to review.

[haavee]

Can't you fork-and-make-a-PR?

[Athanaseus]

Good idea. Here check it out https://github.com/haavee/jiveplot/pull/17