Add capability to ingest ApRES data taken in attended mode

[Elizabethcase]

Adds "orientation" coord and function _get_orientation to obtain antenna orientation from the filename; if the filename returns something other that HH, HV, VH, or VV, then the orientation is logged as "unknown". Not very robust yet.

[jkingslake]

Could you add a .dat file from a polarmetric survey (i.e. which has HH, HV, VH, or VV in the filename) to the data directory,to help with testing this PR.

[Elizabethcase]

yes! done, added four files w/ HH, VV, HV, and VH orientations

[jkingslake]

This looks good @Elizabethcase.

I got it working as follows:

import sys
sys.path.append("/Users/jkingslake/Documents/science/ApRES/xapres_package/xapres_package")
import ApRESDefs

xa = ApRESDefs.xapres()

xa.load_all(directory='../../data')

xa.data.orientation

which gives

It also works for a remote load

xa2 = ApRESDefs.xapres()
xa2.load_all(directory='gs://ldeo-glaciology/GL_apres_2022', 
            remote_load = True,
            file_numbers_to_process = [0, 1], 
            bursts_to_process=[0, 1]
           )
xa2.data.orientation

[jkingslake]

The only thing is that this does not allow the really easy slicing based on orientation, i dont think, because orientation is not dimension.

Initially I was suggestions making orientation a dimension so that you could do things like xa.data.isel(orientation=1) to select all the HH data. Or xa.data.isel(time=3) to select all the data from a particular site, including all the orientations. This is more in line with how we dealt with attenuator settings, for example. As it stands, the latter example at least is not a straightforward xarray operation, I don't think.

What do you think @Elizabethcase?

cc @glugeorge

[jkingslake]

After our conversation today, I think we decided to replace the time dimension with a waypoint dim and an 'orientation' dim.

[jkingslake]

This code loops over the orientations and sorts out the dimensions and coordinates in a way that I think is what we decided earlier.

import sys
sys.path.append("/Users/jkingslake/Documents/science/ApRES/xapres_package/xapres_package")
import ApRESDefs
import xarray as xr
import numpy as np

# do the following for each waypoint, then concatenate the results over the waypoint dimension
waypoint_number = 1

# initialize an empy array to contain the individual xarras
list_of_singleOrientation_xarrays = []

# loop over the orientations
for fileNumber in [0, 1, 2, 3]:
    xa = ApRESDefs.xapres()
    xa.load_all(directory='../../data', file_numbers_to_process=[fileNumber])

    new_dims = {"orientation": 1, "waypoint": 1}

    # add a new coordinate waypoints, then remove the dimension time, then add two more dimensions orientation and waypoint
    temp = xa.data.assign_coords(waypoint = waypoint_number).squeeze(dim = 'time').expand_dims(dim =  new_dims)

    # the line above only adds coordinates and dimensions to the variables, these three lines add them to the coordintates too 
    temp['time'] = temp.time.expand_dims(dim =  new_dims)
    temp['filename'] = temp.filename.expand_dims(dim = new_dims)
    temp['burst_number'] = temp.burst_number.expand_dims(dim = new_dims)

    # append the new xarray to a list
    list_of_singleOrientation_xarrays.append(temp)

# concatenate the xarrays in the list along the orientation dimension
singleWaypoint = xr.concat(list_of_singleOrientation_xarrays, dim = 'orientation')

singleWaypoint

cc @Elizabethcase

[Elizabethcase]

This sounds good. Today we chatted about the following three use cases and how to implement them. Waypoint now is used in the place of time to track measurements for attended & polarmetric measurements

unattended (waypoint and time as the primary coordinates) attended (waypoint and repeat # as the primary coordinates) polarmetric (waypoint and orientation as the primary coordinates)

another use case we haven't talked about is CMP, which could use waypoint and separation distance as coordinates, similar to polarmetric

[jkingslake]

this PR is aiming to address #17

[jkingslake]

Thanks @Elizabethcase this is useful. One edit is that currently for unattended mode data structure, we dont have a waypoint as a dimension. The issue with this would be that every way point would have a different time dimension. Currently this is dealt with by the dataset from each waypoint has its own xarray (and ultimately zarr store). Perhaps this could be done in a cleaner way using a datatree (see #9).

[jkingslake]

closes #20 closes #6 closes #7