Add an example on reading multiple sweeps into a single object

[mgrover1]

Closes #64

This is a draft for now, waiting on data from open-radar-data

[kmuehlbauer]

@mgrover1 What is the status of this? If I can be of any help to get it over the line, let me know.

[mgrover1]

It is close to completion - finalizing a few things

[mgrover1]

@kmuehlbauer this is ready for review! I added in a bit using the facetgrid functionality from xarray.

[kmuehlbauer]

Sounds great, Max. I'm afk for today, but I'll review first thing tomorrow.

[codecov[bot]]

Codecov Report

Merging #69 (5ca8079) into main (9f2ce2a) will not change coverage. The diff coverage is n/a.

@@           Coverage Diff           @@
##             main      #69   +/-   ##
=======================================
  Coverage   87.75%   87.75%           
=======================================
  Files          19       19           
  Lines        3284     3284           
=======================================
  Hits         2882     2882           
  Misses        402      402           

Flag	Coverage Δ
unittests	87.75% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

[kmuehlbauer]

@mgrover1 the notebook would need linking into the docs. So far I can tell from the phone.

[kmuehlbauer]

@mgrover1 Please merge at your discretion. The only (stylistic) thing I'd change is wrapping the output of datasets with display().

[VieuxRoblochon]

I modified the data ingest to use multiple iris RAW files. Found that it was incorrectly computing the azimuth dimension (717 which should be 360). Changed angle_res = 1.0. The following now correctly computes azimuth for multiple RAW files. def fix_angle(ds): """ Aligns the radar volumes """ ds["time"] = ds.time.load() #;print(ds["time"]) # Convert time from dask to numpy

start_ang = 0  # Set consistent start/end values
stop_ang = 360
angle_res = 1. <------------no longer computed

Concatenate in xarray

ds0 = xr.open_mfdataset( fns, preprocess=fix_angle, engine="iris", group="sweep_0", concat_dim="volume_time", combine="nested", ) display(ds0) This produces output as expected except that there are NaT's in the time. I then apply some fixes to the times to fill with the timestamp, convert to hours since 1800 with dtype 'int64', fill NaN's in the dual pol moments. These are apparently necessary to be able to archive in zarr format by avoiding encoding errors. However, when this is saved to zarr archive, it yields the following error (extract) that arises from IrisRawFile. .... File ~\miniconda3\envs\xradar\Lib\site-packages\xradar\io\backends\iris.py:3743, in IrisArrayWrapper._getitem(self, key) 3741 def _getitem(self, key): 3742 # read the data and put it into dict -> 3743 self.datastore.root.get_moment(self.group, self.name) 3744 return self.datastore.ds["sweep_data"][self.name][key]

File ~\miniconda3\envs\xradar\Lib\site-packages\xradar\io\backends\iris.py:3484, in IrisRawFile.get_moment(self, sweep_number, moment) 3482 self.init_record(rnum) 3483 self.rh.pos = roff -> 3484 self.get_ray(raw_data[i]) 3486 xhdr_type = phdr["product_configuration"]["product_specific_info"]["xhdr_type"] 3488 data = raw_data[:, 6:]

File ~\miniconda3\envs\xradar\Lib\site-packages\xradar\io\backends\iris.py:3365, in IrisRawFile.get_ray(self, data, skip) 3363 self.skip_from_record(cmp_val, "int16") 3364 else: -> 3365 data[ray_pos : ray_pos + cmp_val] = self.read_from_record( 3366 cmp_val, "int16" 3367 ) 3368 # # compressed zeros follow 3369 # # can be skipped, if data array is created all zeros 3370 # else: (...) 3380 # return recnum, recoff 3381 # data[ray_pos : ray_pos + cmp_val] = 0 3383 ray_pos += cmp_val

ValueError: could not broadcast input array from shape (9,) into shape (0,)

[kmuehlbauer]

@VieuxRoblochon Could you please open a new issue describing that problem in more detail? Thanks!