JCSDA-internal / eva

Evaluation and Verification of the Analysis
Apache License 2.0
5 stars 12 forks source link

Extend mon_data_space.py to handle legacy DA monitor conventional data #167

Closed EdwardSafford-NOAA closed 10 months ago

EdwardSafford-NOAA commented 11 months ago

Description

Extend mon_data_space.py to load legacy DA monitor conventional data. The conventional data is (of course) slightly different from all others. It has 4 dimensions: data type (i.e. q120, 130, etc), region, vertical level, and time. The use of data type is unique among the DA monitor legacy data sources so support for that is needed.

Requirements/Acceptance Criteria:

Load legacy conventional DA monitor data and output time series plots.

Dependencies

None

EdwardSafford-NOAA commented 10 months ago

Naturally the conventional binary data time/vert files are a bit different from all the other legacy data. They have 4 dimensions: data type, region, level, and time. A given file has data for all available data types in that category -- (e.g. ges_q_stas.[yyyymmddhh] has all the q data (q120_0, q130_0, etc). That's the DataType dimension and I need to support data selection by DataType so a batch plot can step through data types in the same manner as channels.

I'm not quite there yet but I do have selection by data type working in single (non-batch) plots. Below is the time series plot for q120_0, region 1 (global), level all (0-2000mb). This matches the legacy plot, but I think maybe making the y axis a log scale would make sense given the wide spread.

q120 time

EdwardSafford-NOAA commented 10 months ago

I've hit something of a snag. I'm trying to convert this plot into a batch plot, selecting data by data type (q120_0, q130_0, etc) in the same manner as channels or levels are selected during batch plotting. But I'm running into problems mapping the datatype string values to DataType number ('q120_0' to 1, 'q130_0' to 2, etc). I've got to dig back into channel and re-learn how non-continuous channels work. I'm beginning to think I messed up the dataset design here.

Update -- yup, that's the problem. mon_data_space.py explicitly loads the actual channel numbers as the values for the Channeldimension in the dataset. I need to do that for DataType dimension too and hope that using strings instead of ints won't cause problems. Learning, slowly.

EdwardSafford-NOAA commented 10 months ago

Batch processing by DataType is now working. This is the product of the prototype ConMon time series plot for q sources:

q120_0 time

EdwardSafford-NOAA commented 10 months ago

Question for @kevindougherty-noaa . I'm working on the ConMon vertical plots, example from eva is below. In the lower left plot the data values are all 0. If I don't explicitly set the xticks the xaxis puts 0 in the middle of the plot with negative and positive values to left and right of 0. I don't really want to show any negative xtick values, since that's not a possibility for counts. Is there a way in emcpy of forcing the xtick values to all be positive values without setting explicit xtick values?

q120_0 time(2)

EdwardSafford-NOAA commented 10 months ago

Closed via PR #169