Closed aklink closed 6 years ago
Issue is specific to used image. Trying a landsat image (created with ERDAS Imagine) is ok (no issue).
Trying a different aerila image leads to different error:
15:05:03 ERROR org.apache.spark.executor.Executor - Exception in task 0.0 in stage 3.0 (TID 3)
geotrellis.raster.GeoAttrsError: invalid cols: 0
at geotrellis.raster.RasterExtent.
The first issue with "geotrellis.raster.io.geotiff.reader.GeoTiffCSParser.getEllipsoidInfo" is caused by an older SRS (Gauss-Krüger Zone 3, used in Baden-Württemberg) in GeoTiff file, can be fixed by applying "gdalwarp -t_srs EPSG:25832 srcfile targetfile" to file to use a newer SRS (ETRS89 UTM32N). Geotrellis seems not to support older SRS.
The second issue with "geotrellis.raster.RasterExtent.(RasterExtent.scala:72)" is caused by a missing SRS in GeoTiff file, can be fixed by applying "gdal_translate -a_srs EPSG:25832 srcfile targetfile" in advance to add a valid SRS (on test file, not in sourcecode).
Maybe related to this issues?
Problematic CRS is EPSG:31467 in following GDAL representation:
PROJCS["DHDN_3_Degree_Gauss_Zone_3",
GEOGCS["GCS_Deutsches_Hauptdreiecksnetz",
DATUM["Deutsches_Hauptdreiecksnetz",
SPHEROID["Bessel_1841",6377397.155,299.1528128,
AUTHORITY["EPSG","7004"]],
AUTHORITY["EPSG","6314"]],
PRIMEM["Greenwich",0],
UNIT["degree",0.0174532925199433]],
PROJECTION["Transverse_Mercator"],
PARAMETER["latitude_of_origin",0],
PARAMETER["central_meridian",9],
PARAMETER["scale_factor",1],
PARAMETER["false_easting",3500000],
PARAMETER["false_northing",0],
UNIT["metre",1,
AUTHORITY["EPSG","9001"]]]
It works for my TIF file in EPSG_31467:
$ gdalinfo morning_EPSG_31467.tif
Driver: GTiff/GeoTIFF
Files: morning_EPSG_31467.tif
Size is 5161, 7911
Coordinate System is:
PROJCS["DHDN / 3-degree Gauss-Kruger zone 3",
GEOGCS["DHDN",
DATUM["Deutsches_Hauptdreiecksnetz",
SPHEROID["Bessel 1841",6377397.155,299.1528128000008,
AUTHORITY["EPSG","7004"]],
TOWGS84[598.1,73.7,418.2,0.202,0.045,-2.455,6.7],
AUTHORITY["EPSG","6314"]],
PRIMEM["Greenwich",0],
UNIT["degree",0.0174532925199433],
AUTHORITY["EPSG","4314"]],
PROJECTION["Transverse_Mercator"],
PARAMETER["latitude_of_origin",0],
PARAMETER["central_meridian",9],
PARAMETER["scale_factor",1],
PARAMETER["false_easting",3500000],
PARAMETER["false_northing",0],
UNIT["metre",1,
AUTHORITY["EPSG","9001"]],
AUTHORITY["EPSG","31467"]]
Origin = (3445385.870000000111759,5447456.839999999850988)
Pixel Size = (5.000000000000000,-5.000000000000000)
Metadata:
AREA_OR_POINT=Area
TIFFTAG_IMAGEDESCRIPTION=
TIFFTAG_RESOLUTIONUNIT=3 (pixels/cm)
TIFFTAG_XRESOLUTION=100
TIFFTAG_YRESOLUTION=100
Image Structure Metadata:
INTERLEAVE=BAND
Corner Coordinates:
Upper Left ( 3445385.870, 5447456.840) ( 8d15' 3.88"E, 49d 9'46.43"N)
Lower Left ( 3445385.870, 5407901.840) ( 8d15'23.00"E, 48d48'25.95"N)
Upper Right ( 3471190.870, 5447456.840) ( 8d36'17.74"E, 49d 9'52.74"N)
Lower Right ( 3471190.870, 5407901.840) ( 8d36'27.83"E, 48d48'32.18"N)
Center ( 3458288.370, 5427679.340) ( 8d25'48.14"E, 48d59' 9.82"N)
Band 1 Block=5161x1 Type=Byte, ColorInterp=Palette
Color Table (RGB with 256 entries)
...
Seems like the major difference between the two CRS representations is missing AUTHORITY["EPSG","31467"] as well as TOWGS84[598.1,73.7,418.2,0.202,0.045,-2.455,6.7] in non-working variant (unfortunatelly all tiles in BW are using this non-working representation)
Is this still an issue? Can you throw us a non working example, or (as i can understand) the problem is in corrupted (or specific) geotiff metadata? Or it is https://github.com/geotrellis/geotrellis/issues/1415 ?
I am not sure if this exactly issue https://github.com/geotrellis/geotrellis/issues/1415 ...
However it is specific to SRS MetaData of DHDN_3_Degree_Gauss_Zone_3.
When looking on: http://spatialreference.org/ref/epsg/31467/ It is neither exactly http://spatialreference.org/ref/epsg/31467/ogcwkt/ nor http://spatialreference.org/ref/epsg/31467/esriwkt/ due to a slighty different string representation.
Please find attached file using DHDN_3_Degree_Gauss_Zone_3. DOP_RGBI_T2_400x400_epsg31467.zip
@aklink thx for attached file, so yes, i believe it is unsupported projection, good info, think it would be fixed (or we'll try to fix it :D) in terms of https://github.com/geotrellis/geotrellis/issues/1415 looks like a bug in a proj4j or in tiff metadata.
@pomadchin Thanks for looking into this issue. I really appreciate it and I am looking forward to have this projection supported by Geotrellis.
Yo guyz, can you try this PR https://github.com/locationtech/geotrellis/pull/2403 if it's not too late? :D basically it was a minor bug, sry that it took us half a year to fix it.
@pomadchin: Thanks for the update. I will have a look into it. Anyhow, I have trouble building it using ./scripts/publish-local.sh in Win10.
Downloading sbt launcher for 0.13.16:
From http://repo.typesafe.com/typesafe/ivy-releases/org.scala-sbt/sbt-launch/0.13.16/sbt-launch.jar
To /home/ak/.sbt/launchers/0.13.16/sbt-launch.jar
Invalid maximum heap size: -Xmx2g
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.
Is there any Maven repository with an already built test version? Or another simple option how to test it ontop of an "older" Geotrellis version?
@aklink sure, we have a location tech snapshot repo:
resolvers ++= Seq(
"LocationTech GeoTrellis Snapshots" at "https://repo.locationtech.org/content/repositories/geotrellis-snapshots"
)
After that just use 1.2.0-SNAPSHOT
as a GeoTrellis version in your deps.
@pomadchin Sorry for my late reply. Thanks! I have tested it with 1.2.0-RC1 and it seems to work fine. I did not check the georeferenced position yet if it is on the proper location, but at least the error is gone. Layer is created properly.
When I try:
java -cp target/biggis-landuse-0.0.1-SNAPSHOT.jar \ biggis.landuse.spark.examples.GeotiffToPyramid \ ./data/DOP_RGBI_T2.tif \ new_layer \ ./data/pyramid
I get the following error message:
15:58:50 ERROR org.apache.spark.executor.Executor - Exception in task 0.0 in stage 0.0 (TID 0) scala.MatchError: Some() (of class scala.Some) at geotrellis.raster.io.geotiff.reader.GeoTiffCSParser.getEllipsoidInfo(GeoTiffCSParser.scala:570) at geotrellis.raster.io.geotiff.reader.GeoTiffCSParser.createGeoTiffCSParameters(GeoTiffCSParser.scala:172) at geotrellis.raster.io.geotiff.reader.GeoTiffCSParser.geoTiffCSParameters$lzycompute(GeoTiffCSParser.scala:78) at geotrellis.raster.io.geotiff.reader.GeoTiffCSParser.geoTiffCSParameters(GeoTiffCSParser.scala:78) at geotrellis.raster.io.geotiff.reader.GeoTiffCSParser.model(GeoTiffCSParser.scala:80) at geotrellis.raster.io.geotiff.tags.TiffTags.crs$lzycompute(TiffTags.scala:207) at geotrellis.raster.io.geotiff.tags.TiffTags.crs(TiffTags.scala:205) at geotrellis.raster.io.geotiff.reader.GeoTiffReader$.readGeoTiffInfo(GeoTiffReader.scala:315) at geotrellis.raster.io.geotiff.reader.GeoTiffReader$.readSingleband(GeoTiffReader.scala:67) at geotrellis.raster.io.geotiff.reader.GeoTiffReader$.readSingleband(GeoTiffReader.scala:61) at geotrellis.raster.io.geotiff.SinglebandGeoTiff$.apply(SinglebandGeoTiff.scala:40) at geotrellis.spark.io.hadoop.formats.GeotiffInputFormat.read(GeotiffInputFormat.scala:28) at geotrellis.spark.io.hadoop.formats.BinaryFileInputFormat$$anonfun$createRecordReader$1.apply(BinaryFileInputFormat.scala:34) at geotrellis.spark.io.hadoop.formats.BinaryFileInputFormat$$anonfun$createRecordReader$1.apply(BinaryFileInputFormat.scala:34) at geotrellis.spark.io.hadoop.formats.BinaryFileRecordReader.initialize(BinaryFileInputFormat.scala:18) at org.apache.spark.rdd.NewHadoopRDD$$anon$1.(NewHadoopRDD.scala:158)
at org.apache.spark.rdd.NewHadoopRDD.compute(NewHadoopRDD.scala:129)
at org.apache.spark.rdd.NewHadoopRDD.compute(NewHadoopRDD.scala:64)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:270)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:270)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
at org.apache.spark.scheduler.Task.run(Task.scala:89)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:227)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)