search

ldeo-glaciology / xapres

package for processing ApRES data using xarray
MIT License
3 stars 2 forks source link

bug: trouble plotting caused by chirp_time.dtype #34

Open jkingslake opened 8 months ago

jkingslake commented 8 months ago

In the zarrs of the greenland lakes ApRES data (e.g., directory = "gs://ldeo-glaciology/apres/greenland/2022/single_zarrs_noencode/A101"), the chirp_time coordinates are in timedelta64[ns] format.

This causes plotting to fail when you try to plot a 2D array which has a length of >1 in the chirp_time dimension.

I am not sure how chirp_time became this format. Loading directly from dat files, with xa.load.generate_xarray produces floats for chirp_time.

This notebook explores the bug: notebooks/guides/bug_testing.ipynb

@glugeorge, do you know how/when chirp_time gets converted to timedelta64[ns]?

glugeorge commented 8 months ago

After skimming through this notebook, does this error also appear when you load from: directory = "gs://ldeo-glaciology/apres/greenland/2022/single_zarrs/A101“? I remember we explicitly got rid of encoding because imposing the float64 condition resulted in all the profiles losing their complex component. I’ll try to look into a workaround for plotting. My gut feel is that timedeltas should be able to be plotted in 2D and sliced.

On Fri, Oct 20, 2023 at 3:44 PM jkingslake @.***> wrote:

In the zarrs of the greenland lakes ApRES data (e.g., directory = "gs://ldeo-glaciology/apres/greenland/2022/single_zarrs_noencode/A101"), the chirp_time coordinates are in timedelta64[ns] format.

This causes plotting to fail when you try to plot a 2D array which has a length of >1 in the chirp_time dimension.

I am not sure how chirp_time became this format. Loading directly from dat files, with xa.load.generate_xarray produces floats for chirp_time.

This notebook explores the bug: notebooks/guides/bug_testing.ipynb https://github.com/ldeo-glaciology/xapres_package/blob/b7571fd0a7a0b95f1994a9352aa4d7c3aee08f4c/notebooks/guides/bug_testing.ipynb

@glugeorge https://github.com/glugeorge, do you know how/when chirp_time gets converted to timedelta64[ns]?

— Reply to this email directly, view it on GitHub https://github.com/ldeo-glaciology/xapres_package/issues/34, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGGJE2FHSKKMS6CPHJAVSLTYALICPAVCNFSM6AAAAAA6JP2JZ2VHI2DSMVQWIX3LMV43ASLTON2WKOZRHE2TIOJSGUZTGMA . You are receiving this because you were mentioned.Message ID: @.***>

glugeorge commented 8 months ago

Ok I went and tried that. Same bug. Looks like the conversion to zarr never set that float64 encoding to the chirp_time dimension ever. Perhaps the timedelta64[ns] is the default encoding for when converting to zarr. I reckon this is similar to the issue (#13) where A103 burst times where getting converted into days since rather than seconds since

jkingslake commented 8 months ago

OK, that's strange. We could do some testing of the write to zarr to see if it somehow changes it to timedeltas. Seems odd that it would only happen to the chirp_time, though.

For now we could add somthing like 

ds['chirp_time'] = ds.chirp_time.values.astype("float64")/1e9

here: https://github.com/ldeo-glaciology/xapres_package/pull/35/files#r1367571608