create tutorial

[telegraphic]

There currently isn't a tutorial about how to use turboseti. This should be rectified!

[telegraphic]

Install turboseti:
pip install git+https://github.com/ucberkeleyseti/turbo_seti
This will give you a command line tool turboSETI:
dcprice@dainkoru:/data/leda/hibayes$ turboSETI -h
Usage: turboSETI <FULL_PATH_TO_FIL_FILE> [options]
Options:
  -h, --help            show this help message and exit
  -M MAX_DRIFT, --max_drift=MAX_DRIFT
                        Set the drift rate to search. Unit: Hz/sec. Default:
                        10.0
  -s SNR, --snr=SNR     SNR threshold. Default: 25.0
  -o OUT_DIR, --out_dir=OUT_DIR
                        Location for output files. Default: local dir.
  -l LOGLEVEL, --loglevel=LOGLEVEL
                        Specify log level (info, debug)
  -c COARSE_CHANS, --coarse_chans=COARSE_CHANS
                        Coma separated list of coarse channels to analyze.(ie.
                        "5,8" to do from 5th to 8th coarse channels)
To get some test data, download turboseti repository with wget:
wget https://github.com/UCBerkeleySETI/turbo_seti/archive/master.zip
and unzip it.
There should be a directory called voyager_test with a file Voyager1.single_coarse.fine_res.h5 in it. This is spectral data with a narrowband signal from the Voyager1 spacecraft in it.
Once turboseti is installed, run:
turboSETI voyager_test/Voyager1.single_coarse.fine_res.h5
This will create a .dat file, which shows where the hits are:
# -------------------------- o --------------------------
# File ID: Voyager1.single_coarse.fine_res.h5
# -------------------------- o --------------------------
# Source:Voyager1
# MJD: 57650.782094907408   RA: 17h10m03.984s   DEC: 12d10m58.8s
# DELTAT:  18.253611    DELTAF(Hz):  -2.793968
# --------------------------
# Top_Hit_#     Drift_Rate  SNR     Uncorrected_Frequency   Corrected_Frequency     Index   freq_start  freq_end    SEFD    SEFD_freq   Coarse_Channel_Number   Full_number_of_hits
# --------------------------
001   0.392226   30.612128     8419.319368     8419.319368  739933     8419.319779     8419.318963  0.0       0.000000  1   1107
002   0.373093  245.707984     8419.297028     8419.297028  747929     8419.297439     8419.296623  0.0       0.000000  1   1107
003   0.392226   31.220652     8419.274374     8419.274374  756037     8419.274785     8419.273969  0.0       0.000000  1   1107
You can then plot the hits (maunually) using blimpy's watutil:
watutil -b 8419.197028 -e 8419.397028 Voyager1.single_coarse.fine_res.h5

[texadactyl]

Installation verified as stated with Anaconda for Python 3. But, turboSETI voyager_test/Voyager1.single_coarse.fine_res.h5 ===> turbo_seti/findevent/find_event.py", line 433 print 'Events found:'

Please change to print('Events found:') -- Python 2 is deprecated.

I opened a separate issue.

[texadactyl]

I fixed the print() syntax of find_event.py. Now,

$ turboSETI voyager_test/Voyager1.single_coarse.fine_res.h5
blimpy.io.file_wrapper INFO     Skipping loading data ...
root            ERROR    Error accessing header from file: voyager_test/Voyager1.single_coarse.fine_res.h5.
Traceback (most recent call last):
  File "/home/elkins/anaconda3/lib/python3.7/site-packages/turbo_seti/findoppler/data_handler.py", line 147, in __init__
    header = self.__make_data_header(self.fil_file.header)
  File "/home/elkins/anaconda3/lib/python3.7/site-packages/turbo_seti/findoppler/data_handler.py", line 219, in __make_data_header
    base_header[u'SOURCE'] = header[u'source_name'].replace('\xc2\xa0','_').replace(' ','_')
KeyError: 'source_name'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/elkins/anaconda3/lib/python3.7/site-packages/turbo_seti/findoppler/seti_event.py", line 75, in main
    find_seti_event = FinDoppler(filename, max_drift = opts.max_drift, snr = opts.snr, out_dir = opts.out_dir,coarse_chans = opts.coarse_chans, obs_info=obs_info)
  File "/home/elkins/anaconda3/lib/python3.7/site-packages/turbo_seti/findoppler/findopp.py", line 58, in __init__
    self.data_handle = DATAHandle(datafile,out_dir=out_dir)
  File "/home/elkins/anaconda3/lib/python3.7/site-packages/turbo_seti/findoppler/data_handler.py", line 49, in __init__
    data_obj = DATAH5(filename)
  File "/home/elkins/anaconda3/lib/python3.7/site-packages/turbo_seti/findoppler/data_handler.py", line 150, in __init__
    raise IOError("Error accessing header from file: %s."%self.filename)
OSError: Error accessing header from file: voyager_test/Voyager1.single_coarse.fine_res.h5.
Traceback (most recent call last):
  File "/home/elkins/anaconda3/lib/python3.7/site-packages/turbo_seti/findoppler/data_handler.py", line 147, in __init__
    header = self.__make_data_header(self.fil_file.header)
  File "/home/elkins/anaconda3/lib/python3.7/site-packages/turbo_seti/findoppler/data_handler.py", line 219, in __make_data_header
    base_header[u'SOURCE'] = header[u'source_name'].replace('\xc2\xa0','_').replace(' ','_')
KeyError: 'source_name'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/elkins/anaconda3/lib/python3.7/site-packages/turbo_seti/findoppler/seti_event.py", line 75, in main
    find_seti_event = FinDoppler(filename, max_drift = opts.max_drift, snr = opts.snr, out_dir = opts.out_dir,coarse_chans = opts.coarse_chans, obs_info=obs_info)
  File "/home/elkins/anaconda3/lib/python3.7/site-packages/turbo_seti/findoppler/findopp.py", line 58, in __init__
    self.data_handle = DATAHandle(datafile,out_dir=out_dir)
  File "/home/elkins/anaconda3/lib/python3.7/site-packages/turbo_seti/findoppler/data_handler.py", line 49, in __init__
    data_obj = DATAH5(filename)
  File "/home/elkins/anaconda3/lib/python3.7/site-packages/turbo_seti/findoppler/data_handler.py", line 150, in __init__
    raise IOError("Error accessing header from file: %s."%self.filename)
OSError: Error accessing header from file: voyager_test/Voyager1.single_coarse.fine_res.h5.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/elkins/anaconda3/bin/turboSETI", line 8, in <module>
    sys.exit(main())
  File "/home/elkins/anaconda3/lib/python3.7/site-packages/turbo_seti/findoppler/seti_event.py", line 84, in main
    raise Exception(1,'[turbo_SETI] Some issue with FinDoppler.',e)
Exception: (1, '[turbo_SETI] Some issue with FinDoppler.', OSError('Error accessing header from file: voyager_test/Voyager1.single_coarse.fine_res.h5.'))

I have the bitshuffle & h5py packages installed.

It looks like the issues that I raised in #16 have not been addressed. Good luck!

[texadactyl]

Once I place import hdf5plugin at the top of the source file, I can execute the h5py routines fine using Voyager1.single_coarse.fine_res.h5 as an input file. So, the (Python 3 related?) issues lie with turbo_seti or blimpy.

[telegraphic]

Will look into this -- the issues in #16 should have been addressed downstream in blimpy, which does now import hdf5plugin.

[texadactyl]

The following is the source file which accesses the Voyager file without issues as I mentioned previously:

import h5py
import numpy as np

PATH = "Voyager1.single_coarse.fine_res.h5"

hf = h5py.File(PATH, "r")

ix = 0
for name in hf:
    ix += 1
    print("{}. {}".format(ix, name))
    dset = hf[name]
    print("\tshape:", dset.shape)
    print("\tdtype:", dset.dtype)
    got = hf.get(name)
    print("\tget:", got)
    got_items = np.array(got)
    print("\tgot_items.shape:", got_items.shape)
    print("\tgot_items:\n", got_items)

hf.close()

[texadactyl]

@telegraphic Blimpy is fixed and in pypi. So, one can simply pip3 install blimpy. Works fine under Python 3 now: turboSETI Voyager1.single_coarse.fine_res.h5

[telegraphic]

We now have very basic sphinx documentation on readthedocs:

https://turbo-seti.readthedocs.io/

Lots of work to go to bring this up to scratch -- and still need an actual tutorial, but should have all the scaffolding done.

[dmkunsman]

Any thing I can do to help with the documentation?

[telegraphic]

@dmkunsman any improvements more than welcome! Tutorials, overviews, improving docstrings, explaining how the package fits together, etc.

The docs are in https://github.com/UCBerkeleySETI/turbo_seti/tree/master/docs, and use sphinx to build

[texadactyl]

Fixed by @ssmith00 August 6, 2020. See the tutorial subdirectory.