locationtech / geowave

GeoWave provides geospatial and temporal indexing on top of Accumulo, HBase, BigTable, Cassandra, Kudu, Redis, RocksDB, and DynamoDB.
Apache License 2.0
498 stars 190 forks source link

Ingesting NetCDF or GRIB files #1905

Open pflashgary opened 2 years ago

pflashgary commented 2 years ago

Hi rfecher, I infer from this line netcdf format is supported. What I would like to try is ingesting a folder containing NetCDF files. Due to the differences between NetCDF/GRIB files and GeoTiff files I have a few questions:

I have downloaded an ERA5 netcdf file just to test this and these are the steps I have taken:

source /tmp/geowave-env.sh
geowave store add nc --gwNamespace geowave.nc -t dynamodb --region US_EAST_2
geowave index add nc ncspatial_temporal -t spatial_temporal --partitionStrategy round_robin --numPartitions $NUM_PARTITIONS
geowave ingest localtogw adaptor.mars.internal-<some-file>.nc nc ncspatial_temporal

Nothing failed only some SLF4J warnings.

These are the tables:

aws dynamodb list-tables --region us-east-2
{
    "TableNames": [
        "geowave.nc_INDEX_GEOWAVE_METADATA",
        "geowave.nc_STATISTICS_GEOWAVE_METADATA",
        "geowave.nc_STORE_PROPERTIES_GEOWAVE_METADATA"
    ]
}

However, these tables contain no items (after running the describe table command on each table). That means ingesting didn't happen.

aws dynamodb describe-table --table-name geowave.nc_STORE_PROPERTIES_GEOWAVE_METADATA --region us-east-2

I am wondering if an example of ingesting from NetCDF /GRIB files can be included?