search

ARM-DOE / ACT

Atmospheric data Community Toolkit - A python based toolkit for exploring and analyzing time series atmospheric datasets
https://ARM-DOE.github.io/ACT/
Other
138 stars 35 forks source link

Streaking with missing data in 2D plots #766

Closed kenkehoe closed 9 months ago

kenkehoe commented 10 months ago

Description

There is a new issue with pcolormesh() causing streaking when there is larger data gaps between data periods. If you look closely at the plot there is a part with some structure at the start and end of the plotted periods. That is the actual data. The extended streaking part is an artifact of pcolormesh(). The issue is that we use add_nan=True to add one NaN value in the middle of the missing data period. pcolormesh() will extend the plot to ½ that window to the inserted NaN. All attempts at changing this behavior with shading keyword (set_shading in .plot() from timeseriesdisplay()) did not fix this issue. epcwsacrS2 a1 vpt 20231102

What I Did

If I modify data_utils.add_in_nan() to add two values during the missing data period, one right after last value and one just before next value, the issue is resolved. Figure_1

I have some code ready to go. It will add one NaN value in the middle of the data period for 1D data, same as normal. But will add a value right after last value and right before next value. With two values added the streaking is removed. With v2.0 coming will need to integrate correctly.

zssherman commented 10 months ago

That's interesting, yeah I'm not sure the best approach for this, I think your solution is fine, if you want to share the code or submit and PR and we can take a look!