Importer fails in 0.10.28-rc.10 : ERROR 1: Attempt to create -1x21 dataset is illegal,sizes must be larger than zero.

danhamill commented 2 years ago

Using vortex-0.10.28-rc.10 I cannot import, clip, and reproject using importer.

importer params:

Option	Value
grid	MRMS_GaugeCorr_QPE_01H_00.00_20191001-050000.grib2
projection	SHG
resolution	2000-meters
resample method	bilinear
clip shape	SL_Watershed_Buffer_SHG.shp
dss	vortex creates a new file

Data

Traceback

ERROR 1: Attempt to create -1x21 dataset is illegal,sizes must be larger than zero.
java.lang.reflect.InvocationTargetException
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.base/java.lang.reflect.Method.invoke(Unknown Source)
        at importer.controller.WizardController$1.call(WizardController.java:169)
        at importer.controller.WizardController$1.call(WizardController.java:163)
        at javafx.graphics/javafx.concurrent.Task$TaskCallable.call(Task.java:1425)
        at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
        at java.base/java.lang.Thread.run(Unknown Source)
Caused by: java.lang.NullPointerException
        at mil.army.usace.hec.vortex.geo.Resampler.resample(Resampler.java:205)
        at mil.army.usace.hec.vortex.geo.Resampler.resample(Resampler.java:97)
        at mil.army.usace.hec.vortex.io.ImportableUnit.lambda$process$0(ImportableUnit.java:153)
        at java.base/java.util.stream.ForEachOps$ForEachOp$OfInt.accept(Unknown Source)
        at java.base/java.util.stream.Streams$RangeIntSpliterator.forEachRemaining(Unknown Source)
        at java.base/java.util.Spliterator$OfInt.forEachRemaining(Unknown Source)
        at java.base/java.util.stream.AbstractPipeline.copyInto(Unknown Source)
        at java.base/java.util.stream.ForEachOps$ForEachTask.compute(Unknown Source)
        at java.base/java.util.concurrent.CountedCompleter.exec(Unknown Source)
        at java.base/java.util.concurrent.ForkJoinTask.doExec(Unknown Source)
        at java.base/java.util.concurrent.ForkJoinTask.doInvoke(Unknown Source)
        at java.base/java.util.concurrent.ForkJoinTask.invoke(Unknown Source)
        at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateParallel(Unknown Source)
        at java.base/java.util.stream.ForEachOps$ForEachOp$OfInt.evaluateParallel(Unknown Source)
        at java.base/java.util.stream.AbstractPipeline.evaluate(Unknown Source)
        at java.base/java.util.stream.IntPipeline.forEach(Unknown Source)
        at java.base/java.util.stream.IntPipeline$Head.forEach(Unknown Source)
        at mil.army.usace.hec.vortex.io.ImportableUnit.process(ImportableUnit.java:123)
        at mil.army.usace.hec.vortex.io.BatchImporter.lambda$process$4(BatchImporter.java:145)
        at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(Unknown Source)
        at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(Unknown Source)
        at java.base/java.util.stream.AbstractPipeline.copyInto(Unknown Source)
        at java.base/java.util.stream.ForEachOps$ForEachTask.compute(Unknown Source)
        at java.base/java.util.concurrent.CountedCompleter.exec(Unknown Source)
        at java.base/java.util.concurrent.ForkJoinTask.doExec(Unknown Source)
        at java.base/java.util.concurrent.ForkJoinTask.doInvoke(Unknown Source)
        at java.base/java.util.concurrent.ForkJoinTask.invoke(Unknown Source)
        at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateParallel(Unknown Source)
        at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateParallel(Unknown Source)
        at java.base/java.util.stream.AbstractPipeline.evaluate(Unknown Source)
        at java.base/java.util.stream.ReferencePipeline.forEach(Unknown Source)
        at java.base/java.util.stream.ReferencePipeline$Head.forEach(Unknown Source)
        at mil.army.usace.hec.vortex.io.BatchImporter.process(BatchImporter.java:137)
        at importer.controller.Step3Controller.submit(Step3Controller.java:209)
        ... 9 more

Notes

I was able to successfully import the CONUS grid with SHG projection and clip using clipper.

tombrauer commented 2 years ago

Without digging in to it, I'm guessing it has to do with the shapefile that is used for the clip. What projection is the shapefile in?

danhamill commented 2 years ago

I was kind of thinking that too, but I couldn't find anything wrong with it. The shapefile is in SHG.

Things i did:

inspect geometry shape: it is of type polygon (no tiny holes or multipolygon geometry)
validate geometry: the line string contained in the polygon geometry starts and ends at the same coordinate (e.g., geometry is valid)
plotted the shapefile is several platforms and everything plotted as expected.

C:\Program Files\GDAL>gdalsrsinfo "C:\Temp\hittle\SL_Boundary\SL_Watershed_Buffer_SHG.shp"

PROJ.4 : +proj=aea +lat_1=29.5 +lat_2=45.5 +lat_0=0 +lon_0=0 +x_0=0 +y_0=0 +datum=NAD83 +units=m +no_defs

OGC WKT :
PROJCS["Albers",
    GEOGCS["GCS_North_American_1983",
        DATUM["North_American_Datum_1983",
            SPHEROID["GRS_1980",6378137.0,298.257222101]],
        PRIMEM["Greenwich",0.0],
        UNIT["Degree",0.0174532925199433]],
    PROJECTION["Albers_Conic_Equal_Area"],
    PARAMETER["False_Easting",0.0],
    PARAMETER["False_Northing",0.0],
    PARAMETER["longitude_of_center",0.0],
    PARAMETER["Standard_Parallel_1",29.5],
    PARAMETER["Standard_Parallel_2",45.5],
    PARAMETER["latitude_of_center",0.0],
    UNIT["Meter",1.0]]

I am not sure why vortex would be able to clip after the mrms grid is already in dss. We sucessfully ran importer with the exact same data using v0.10.26

tombrauer commented 2 years ago

When I plot your shapefile in QGIS, it looks like this: This is with 0.0 rotation. Note: I added your shapefile first, so QGIS would adopt the CRS of the shapefile.

tombrauer commented 2 years ago

For SHG, latitude of false origin (false northing) should equal 23 and longitude of false origin (false easting) should equal -96. Yours is at 0, 0.

tombrauer commented 2 years ago

In 57d5fe53946d28ee62139f95433193a2fa362e7b I added a more robust handling of the clipping extent window. This avoids the negative domain allows the clip to proceed. I'm still not sure this will give you the desired outcome because it seems like there is some distortion that happens when reprojecting. I would reproject the clipping shapefile to SHG and then clip.

danhamill commented 2 years ago

Thanks for looking into this shapefile issue. Looks like I have become a little too comfortable with the cached user-defined crs in qgis.

So this is what I know SHG to be:

PROJCS["USA_Contiguous_Albers_Equal_Area_Conic_USGS_version",
    GEOGCS["GCS_North_American_1983",
        DATUM["D_North_American_1983",
            SPHEROID["GRS_1980",6378137.0,298.257222101]],
        PRIMEM["Greenwich",0.0],
        UNIT["Degree",0.0174532925199433]],
    PROJECTION["Albers"],
    PARAMETER["False_Easting",0.0],
    PARAMETER["False_Northing",0.0],
    PARAMETER["Central_Meridian",-96.0],
    PARAMETER["Standard_Parallel_1",29.5],
    PARAMETER["Standard_Parallel_2",45.5],
    PARAMETER["Latitude_Of_Origin",23.0],
    UNIT["Meter",1.0],
    AUTHORITY["ESRI","102039"]]

The parameter key values are a bit different than my comment above, but I interpret your "latitude of false origin" to be "Latitude of Origin" and "longitude of false origin" to be "central meridian" in the WKT shown in this comment.

tombrauer commented 2 years ago

@danhamill were you able to test the import/clip with a reprojected (to SHG) version of your shapefile?

danhamill commented 2 years ago

yes, I just tested with a co-worker using 0.10.29

HydrologicEngineeringCenter / Vortex