ropensci / tidync

NetCDF exploration and data extraction
https://docs.ropensci.org/tidync
90 stars 12 forks source link

Opening MODIS Aqua nc file is not working #108

Closed PMassicotte closed 3 years ago

PMassicotte commented 3 years ago

I am trying to open a MODIS Aqua NetCDF file (https://oceandata.sci.gsfc.nasa.gov/MODIS-Aqua/L3SMI/2010/001/) using the tidync package.

library(tidync)

tidync("~/Desktop/A20120012012008.L3b_8D_CHL.nc")
#> Error: `distinct()` must use existing variables.
#> x `id` not found in `.data`.
#> x `name` not found in `.data`.
#> x `type` not found in `.data`.
#> x `ndims` not found in `.data`.
#> x `natts` not found in `.data`.

Any ideas on why this is not working?

Created on 2021-02-20 by the reprex package (v1.0.0)

matteodefelice commented 3 years ago

Can you post a ncdump - h of the file?

On Sat, 20 Feb 2021, 22:58 Philippe Massicotte, notifications@github.com wrote:

I am trying to open a MODIS Aqua NetCDF file ( https://oceandata.sci.gsfc.nasa.gov/MODIS-Aqua/L3SMI/2010/001/) using the tidync package.

library(tidync)

tidync("~/Desktop/A20120012012008.L3b_8D_CHL.nc")#> Error: distinct() must use existing variables.#> x id not found in .data.#> x name not found in .data.#> x type not found in .data.#> x ndims not found in .data.#> x natts not found in .data.

Any ideas on why this is not working?

Created on 2021-02-20 by the reprex package https://reprex.tidyverse.org (v1.0.0)

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/ropensci/tidync/issues/108, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABQQYASEWHQZH2R5ANITPUDTAAWAJANCNFSM4X6JDAOA .

mdsumner commented 3 years ago

the file you post is "L3b", which is "Level 3 bin" - tidync can't open these because neither RNetCDF or ncdf4 can access their data. (though, this context changes all the time so I might be a bit out of date).

The error message should be better so I'll have a look.

The folder you share is to the "SMI" "Standard Mapped Image" - there's no L3b in there - but the SMI should all work fine.

I expect you really don't want the Level 3 bins ... but if you do I can help, with other packages ;)

PMassicotte commented 3 years ago

Hi guys and thank you for your replies. I have found out that I could open it assuming it was hdf5 format...

library(rhdf5)
h5ls("~/Desktop/A20120012012008.L3b_8D_CHL.nc")

I am not sure why.

Based on your message, I was able to open:

I am not sure to understand why the L3B cannot be open.

PMassicotte commented 3 years ago

From here: https://oceancolor.gsfc.nasa.gov/docs/format/

Each Level-3 binned data product is stored in one or more HDF files. Each multi-file product includes a main file containing all product-level metadata and data for each bin that are common to all the binned geophysical parameters, and multiple subordinate files, each of which contains data of one binned geophysical parameter for all bins. Subordinate files must be read in conjunction with the associated main file.

It looks like they are using .nc files which are in fact HDF files.

mdsumner commented 3 years ago

There's two things, compound types (like lists of vectors, or tables) and groups. ncdf4 and RNetCDF do both support groups, these are like a directory of files within a file - but they don't support the compound types (hdf5 tools do support)

PMassicotte commented 3 years ago

Thank you for your answer. I will close this.