ACDguide / BigData

Working with big/challenging data collections
https://ACDguide.github.io/BigData
Other
5 stars 5 forks source link

add xbitinfo for compression #78

Open paolap opened 1 year ago

paolap commented 1 year ago

https://xbitinfo.readthedocs.io/en/latest/

paigem commented 1 year ago

We should also add a note about paying attention to precision when creating data (e.g. single or double).

Maybe we could add a small section on compression and precision to our Large-scale climate data section?

paolap commented 1 year ago

FYI we do mention compression in that section already it used to link to the CMS wiki

http://climate-cms.wikis.unsw.edu.au/NetCDF_Compression_Tools

As I need to find it a longer term location, I'm going to place the same content (hopefully will be improved too) in the Governance book, as we are covering how to create datasets there, I have already updated the "link" (however broken right now) in the Large-scale climate data section for it. We haven't covered choosing between single-double precision, but again we can do that in the same page. To get an idea what's covered already in the Governance you can check the "Concepts" and "Technical tips" sections: https://acdguide.github.io/Governance/concepts/concept-intro.html https://acdguide.github.io/Governance/tech/tech-intro.html

paolap commented 1 year ago

I'm referring to this in data-netcdf.md but the current Governance book version doesn't have this page yet, so this is just a reminder for me to update the link eventually