realfast

Realfast is the name of a project and software package related to radio interferometric data analysis at the Very Large Array. For more information, see realfast.io or visit the VLA web site.

This repo includes a Python 3 application that integrates a transient search pipeline with the real-time environment at the VLA. This requires running asynchronous processes in a cluster environment and using VLA services for distributing data, metadata, and archiving. This application:

• monitors multicast messages with evla_mcast,
• catches visibility data via the vys protocol,
• defines a fast transient search pipeline with rfpipe,
• submits search pipeline to a compute cluster with distributed,
• indexes candidate transients and other metadata for the search interface, and
• writes and archives new visibility files for candidate transients.

These are the core functions required for the realfast project. Other features include:

• Support for GPU algorithms,
• Management of distributed futures,
• Blind injection and management of mock transients, and
• Reading of sdm data via sdmpy.

Requirements:

The realfast application is build upon the following libraries:

• Python 3.5 and the scientific python stack (numpy, etc.)
• evla_mcast
• vys and vysmaw_apps
• distributed
• rfpipe
• rfgpu
• sdmpy (to read/write sdm/bdf data)
• elasticsearch (version<7.0)
• supervisor (to manage/daemonize processes)
• click
• fetch (incl cuda, tensorflow, etc.)

Install

This application is designed to integrate specific protocols and services only availble on the correlator cluster at the VLA. If you'd like to build it yourself, you will need to use the anaconda installer and follow instructions at rfpipe. Then install the dependencies above (vys and pycuda being the trickiest) and:

python setup.py install

Contributors:

The custom realfast software includes contributions from:

• Sarah Burke-Spolaor -- WVU
• Paul Demorest -- NRAO
• Andrew Halle -- UC Berkeley
• Casey Law -- UC Berkeley
• Joe Lazio -- JPL
• Martin Pokorny -- NRAO