search

locationtech / geowave

GeoWave provides geospatial and temporal indexing on top of Accumulo, HBase, BigTable, Cassandra, Kudu, Redis, RocksDB, and DynamoDB.
Apache License 2.0
500 stars 190 forks source link

Grid coverage plugin in ingest framework triggering on shapefile #333

Open chrisbennight opened 9 years ago

chrisbennight commented 9 years ago

Probably not being selective enough in how it identifies files/formats - but it's not playing nicely

[bennight@cacti data]$ ls /data/natural_earth/
ne_10m_land.cpg  ne_10m_land.dbf  ne_10m_land.prj  ne_10m_land.README.html  ne_10m_land.shp  ne_10m_land.shx  ne_10m_land.VERSION.txt
[bennight@cacti data]$ geowave-ingest --localingest -b /data/natural_earth/ -i $INSTANCE -n geowave -p $GEOWAVE_PASS -u $GEOWAVE_USER -z $ZOOKEEPERS
Apr 11, 2015 5:04:20 PM org.geoserver.platform.GeoServerExtensions checkContext
WARNING: Extension lookup 'GeoServerResourceLoader', but ApplicationContext is unset.
Apr 11, 2015 5:04:20 PM org.geoserver.platform.GeoServerExtensions checkContext
WARNING: Extension lookup 'GeoServerResourceLoader', but ApplicationContext is unset.
Apr 11, 2015 5:04:20 PM org.geoserver.platform.GeoServerExtensions checkContext
WARNING: Extension lookup 'GeoServerResourceLoader', but ApplicationContext is unset.
Apr 11, 2015 5:04:20 PM org.geoserver.platform.GeoServerExtensions checkContext
WARNING: Extension lookup 'GeoServerResourceLoader', but ApplicationContext is unset.
Apr 11, 2015 5:04:21 PM org.geotools.gce.imagemosaic.Utils loadMosaicProperties
INFO: properties file doesn't exist
Apr 11, 2015 5:04:21 PM org.geotools.gce.imagemosaic.Utils loadMosaicProperties
INFO: properties file doesn't exist
Exception in thread "main" java.lang.UnsupportedOperationException: Trying to get a reader from an unknown format.
        at org.geotools.coverage.grid.io.UnknownFormat.getReader(UnknownFormat.java:62)
        at mil.nga.giat.geowave.types.geotools.raster.GeoToolsRasterDataStoreIngestPlugin.toGeoWaveData(GeoToolsRasterDataStoreIngestPlugin.java:75)
        at mil.nga.giat.geowave.types.geotools.raster.GeoToolsRasterDataStoreIngestPlugin.toGeoWaveData(GeoToolsRasterDataStoreIngestPlugin.java:35)
        at mil.nga.giat.geowave.ingest.local.LocalFileIngestDriver.processFile(LocalFileIngestDriver.java:146)
        at mil.nga.giat.geowave.ingest.local.LocalFileIngestDriver.processFile(LocalFileIngestDriver.java:33)
        at mil.nga.giat.geowave.ingest.local.LocalPluginFileVisitor.visitFile(LocalPluginFileVisitor.java:126)
        at mil.nga.giat.geowave.ingest.local.LocalPluginFileVisitor.visitFile(LocalPluginFileVisitor.java:30)
        at java.nio.file.Files.walkFileTree(Files.java:2670)
        at java.nio.file.Files.walkFileTree(Files.java:2742)
        at mil.nga.giat.geowave.ingest.local.AbstractLocalFileDriver.processInput(AbstractLocalFileDriver.java:60)
        at mil.nga.giat.geowave.ingest.local.LocalFileIngestDriver.runInternal(LocalFileIngestDriver.java:114)
        at mil.nga.giat.geowave.ingest.AbstractCommandLineDriver.run(AbstractCommandLineDriver.java:72)
        at mil.nga.giat.geowave.ingest.IngestMain.main(IngestMain.java:51)
[bennight@cacti data]$
ScottEvil commented 9 years ago

Is this an outstanding issue with the localingest example that's in the documentation? I'm running into these errors but it freezes at "WARNING: Extension lookup 'GeoServerResourceLoader', but ApplicationContext is unset". I did ^C and now Accumulo Master doesn't stay running.

The command I ran looks like this: geowave -localingest -b ./ingest -i accumulo -n geowave.ne_50m_admin_0_countries -f geotools-vector -u geowave -p hadoop -z localhost:2181

Full history is here: https://gist.github.com/ScottEvil/538cbcddd46acc88304b

spohnan commented 9 years ago

Did you bump up the heap space configurations for the Accumulo Master and Tablet Server processes? Most of the distros set these too low and the process dies trying to load the GeoWave iterator. http://ngageoint.github.io/geowave/documentation.html#accumulo-configuration

ScottEvil commented 9 years ago

@spohnan That was exactly the issue. I bumped up the heap sizes and it completed. Thank you!