Closed alex-s-gardner closed 1 year ago
You should be able to open it with open_dataset as a dataset which is a collection of cubes. We should add this to the docs and give a hint in the error.
Ooopps... I think got guidance on this same issue last year. Guidance in the documentation would be greatly appreciated. I can add this in a PR if you tell me where you would like it added.
I think it would be good to have examples for a cube and open_dataset in the "Open NetCDF" and "Open Zarr" pages of the docs and then we would also need an explanation page for the differences between datasets and Cubes but this is something I could write after my vacation..
why not overload cube
to handle both cases?
The Cube function is using the open_dataset
function to load a netcdf file and then concatenates all cubes with the most dimensions into a single data cube with an additional Variable
dimension.
I don't understand how we could overload cube to handle both cases.
Playing around a bit with YAXArray I'm really excited by what you've done and I also I think there is room for improving the API. It seems that most users should simply interact with YAXArray using datasets. Cubes would only be for expert users... netcdfs and zarrs are nearly or always going to be datasets and this is how the user is going to expect to interact with the data. 80% of users will be read only. High level functionality similar to NCDatasets would make YAXarray more intuitive to use.
For example:
Maybe an exported function Dataset
instead of open_dataset
would be better and if Dataset
was used to read a Cube
it could return a Dataset
. This would be a bit more intuitive
and something like:
open_dataset(zopen(store, consolidated=true))
could be
Dataset(pat2file; kwg)
I'm still wrapping my head around the API, but I find the best time to give feedback is the first time you're working with the package as this is were new users will come from.
Some more notes as I work with the package: [1] Time is displayed in the Repl as:
Dim{:x} Sampled{Float32} 633950.0f0:100.0f0:706150.0f0 ForwardOrdered Regular Points,
Dim{:y} Sampled{Float32} 5.87665f6:-100.0f0:5.76265f6 ReverseOrdered Regular Points,
Ti Sampled{DateTime} DateTime[2000-07-28T00:00:00, …, 2019-08-18T00:00:00] ForwardOrdered Irregular Points
maybe Dim{:time}
would be more clear?
[2]
a function save
could replace savedataset
& savecube
. If save is passed a Cube it would call savecube
, if it was passed Dataset it could call savedataset
save
would also check for a file name extension (e.g. .zarr or .nc) to determine which driver to use... or the user could pass driver as a kwarg.
@alex-s-gardner could you provide a smaller file so that we can include this case directly into the docs already. Now is ~200MB which is not good for CI, having something smaller would help a lot. Thanks.
@lazarusA what about one of the files hosted here: https://www.unidata.ucar.edu/software/netcdf/examples/files.html
maybe this one: sresa1b_ncar_ccsm3-example.nc
which is 2.8 MB
@lazarusA look like you added an example thanks... closing this issue and starting new ones for some of the random comments I added to the thread.
Following the example in the docs for reading a netcdf throws error for
YAXArrays
but notNCDatasets
orNetCDF
:throws this error at a place that in the code that is flagged a being a bit hackey:
Check reading a single variable using NetCDF
Check using NCDatasets