Schedule field separator

[keskitalo]

This PR adds an option to the scheduler to write observing schedules with arbitrary field separators.

[keskitalo]

I agree, lots of advantages to astropy tables. The one big minus that is holding me back is that you can no longer inspect and edit the files in a text editor.

[tskisner]

I just wanted to address this last comment. The whole point of the various ascii table formats is that they are human-readable and can be modified. Attached here is an example script that loads a ground schedule and dumps out the various formats supported by astropy.table. The ECSV case looks like this:

# %ECSV 1.0
# ---
# datatype:
# - {name: name, datatype: string}
# - {name: start, datatype: string}
# - {name: stop, datatype: string}
# - {name: boresight_angle, unit: deg, datatype: float64}
# - {name: az_min, unit: deg, datatype: float64}
# - {name: az_max, unit: deg, datatype: float64}
# - {name: el, unit: deg, datatype: float64}
# - {name: rising, datatype: bool}
# - {name: scan_indx, datatype: string}
# - {name: subscan_indx, datatype: string}
# meta:
#   __serialized_columns__:
#     az_max:
#       __class__: astropy.units.quantity.Quantity
#       unit: &id001 !astropy.units.Unit {unit: deg}
#       value: !astropy.table.SerializedColumn {name: az_max}
#     az_min:
#       __class__: astropy.units.quantity.Quantity
#       unit: *id001
#       value: !astropy.table.SerializedColumn {name: az_min}
#     boresight_angle:
#       __class__: astropy.units.quantity.Quantity
#       unit: *id001
#       value: !astropy.table.SerializedColumn {name: boresight_angle}
#     el:
#       __class__: astropy.units.quantity.Quantity
#       unit: *id001
#       value: !astropy.table.SerializedColumn {name: el}
#   site_alt: 5200.0 m
#   site_lat: -22.958 deg
#   site_lon: -67.786 deg
#   site_name: atacama
#   telescope_name: LAT
# schema: astropy-2.0
name start stop boresight_angle az_min az_max el rising scan_indx subscan_indx
RISING_SCAN_35 2027-01-01T00:00:00+00:00 2027-01-01T00:20:00+00:00 180.0 30.0 150.0 35.0 True 0 0
RISING_SCAN_35 2027-01-01T00:20:00+00:00 2027-01-01T00:40:00+00:00 180.0 30.0 150.0 35.0 True 0 1
RISING_SCAN_35 2027-01-01T00:40:00+00:00 2027-01-01T01:00:00+00:00 180.0 30.0 150.0 35.0 True 0 2
RISING_SCAN_35 2027-01-01T01:00:00+00:00 2027-01-01T01:20:00+00:00 180.0 30.0 150.0 35.0 True 0 3
RISING_SCAN_35 2027-01-01T01:20:00+00:00 2027-01-01T01:40:00+00:00 180.0 30.0 150.0 35.0 True 0 4

So you can see the metadata and column information is stored as comment lines. The full list of file formats is in the documentation.

#!/usr/bin/env python3

import sys
from datetime import datetime

from astropy import units as u
from astropy.table import Column, QTable

import toast
from toast.schedule import GroundSchedule

input = sys.argv[1]

schedule = GroundSchedule()
schedule.read(input)

# Create a table with columns matching the attributes of
# a GroundScan.

scanattr = [
    "name",
    "start",
    "stop",
    "boresight_angle",
    "az_min",
    "az_max",
    "el",
    "rising",
    "scan_indx",
    "subscan_indx",
]

cols = list()
for aname in scanattr:
    # See if this attribute has units and convert all scan
    # values to the same units.
    try:
        col_unit = getattr(schedule.scans[0], aname).unit
        col_data = [getattr(scn, aname).to_value(col_unit) for scn in schedule.scans]
        col_data = u.Quantity(col_data, col_unit)
    except AttributeError:
        col_unit = None
        if isinstance(getattr(schedule.scans[0], aname), datetime):
            col_data = [
                getattr(scn, aname).isoformat(timespec="seconds")
                for scn in schedule.scans
            ]
        else:
            col_data = [getattr(scn, aname) for scn in schedule.scans]
    cols.append(Column(name=aname, data=col_data, unit=col_unit))

# Get overall schedule metadata
meta_keys = [
    "site_name",
    "telescope_name",
    "site_lat",
    "site_lon",
    "site_alt",
]
meta = dict()
for k in meta_keys:
    meta[k] = str(getattr(schedule, k))

schedule_table = QTable(cols, meta=meta)

# Demonstrate different output formats

# ASCII ECSV
output = f"{input}.ecsv"
schedule_table.write(
    output,
    overwrite=True,
    format="ascii.ecsv",
    serialize_method="data_mask",
)

# FITS
output = f"{input}.fits"
schedule_table.write(
    output,
    overwrite=True,
    format="fits",
    serialize_method="data_mask",
)

# HDF5
output = f"{input}.h5"
schedule_table.write(
    output,
    overwrite=True,
    format="hdf5",
    path="/schedule",
    serialize_meta=True,
    serialize_method="data_mask",
)