search

gama-platform / gama.old

Main repository for developing the 1.x versions of GAMA
GNU General Public License v3.0
304 stars 99 forks source link

Error importing a Tiff file #3304

Closed benoitgaudou closed 1 year ago

benoitgaudou commented 2 years ago

Describe the bug I get an issue when trying to read the following TIFF file in GAMA. Patchwork_WL_201810141600.tif.zip

To Reproduce Steps to reproduce the behavior:

  1. Import the previous file with a simple model such as: 
    
model NewModel

global {
grid_file Patchwork_WL_2018101416000_grid_file <- grid_file("../includes/Patchwork_WL_201810141600.tif"); geometry shape <- envelope(Patchwork_WL_2018101416000_grid_file); }

grid c file: Patchwork_WL_2018101416000_grid_file;

experiment name type: gui { output {} }


Get the following GAMA error message:

2 occurrences in 2 agents at cycle 0: The format of Patchwork_WL_201810141600.tif seems incorrect: Could not initialize class it.geosolutions.imageioimpl.plugins.tiff.gdal.GDALMetadataParser in geometry shape <- envelope(Patchwork_WL_2018101416000_grid_file) ; when applying the envelope operator on msi.gama.util.file.GamaGridFile@b440413 in initializing attribute shape in agents Simulation 0, name0


Error in Eclipse:

L'erreur dans Eclipse :

org.geotools.data.DataSourceException at org.geotools.gce.geotiff.GeoTiffReader.(GeoTiffReader.java:279) at org.geotools.gce.geotiff.GeoTiffReader.(GeoTiffReader.java:192) at msi.gama.util.file.GamaGridFile.getOwnCRS(GamaGridFile.java:610) at msi.gama.util.file.GamaGisFile.lambda$1(GamaGisFile.java:111) at com.google.common.cache.LocalCache$LocalManualCache$1.load(LocalCache.java:4853) at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3529) at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2278) at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2155) at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2045) at com.google.common.cache.LocalCache.get(LocalCache.java:3951) at com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4848) at msi.gama.util.file.GamaGisFile.getExistingCRS(GamaGisFile.java:92) at msi.gama.util.file.GamaGridFile.privateCreateCoverage(GamaGridFile.java:373) at msi.gama.util.file.GamaGridFile.createCoverage(GamaGridFile.java:253) at msi.gama.util.file.GamaGridFile.getRows(GamaGridFile.java:709) at msi.gama.metamodel.topology.grid.GamaSpatialMatrix.(GamaSpatialMatrix.java:246) at msi.gama.metamodel.topology.grid.GridTopology.(GridTopology.java:126) at msi.gama.metamodel.population.GamaPopulation.buildGridTopology(GamaPopulation.java:701) at msi.gama.metamodel.population.GamaPopulation.createPopulation(GamaPopulation.java:120) at msi.gama.metamodel.agent.GamlAgent.initializeMicroPopulation(GamlAgent.java:228) at msi.gaml.descriptions.SpeciesDescription.lambda$4(SpeciesDescription.java:850) at msi.gaml.compilation.IGamaHelper.run(IGamaHelper.java:49) at msi.gaml.variables.Variable.initializeWith(Variable.java:590) at msi.gama.metamodel.population.GamaPopulation.createVariablesFor(GamaPopulation.java:546) at msi.gama.kernel.simulation.SimulationPopulation.initSimulation(SimulationPopulation.java:158) at msi.gama.kernel.simulation.SimulationPopulation.createAgents(SimulationPopulation.java:119) at msi.gama.metamodel.population.IPopulation.createAgents(IPopulation.java:229) at msi.gama.kernel.experiment.ExperimentAgent.createSimulation(ExperimentAgent.java:357) at msi.gama.kernel.experiment.ExperimentAgent.init(ExperimentAgent.java:316) at msi.gama.metamodel.agent.MinimalAgent.init(MinimalAgent.java:222) at msi.gama.kernel.experiment.ExperimentAgent.init(ExperimentAgent.java:333) at msi.gama.runtime.ExecutionScope.init(ExecutionScope.java:531) at msi.gama.kernel.experiment.ExperimentController.schedule(ExperimentController.java:279) at msi.gama.kernel.experiment.ExperimentAgent.schedule(ExperimentAgent.java:380) at msi.gama.kernel.experiment.ExperimentPlan.open(ExperimentPlan.java:564) at msi.gama.kernel.experiment.ExperimentPlan.open(ExperimentPlan.java:582) at msi.gama.kernel.experiment.ExperimentController.lambda$2(ExperimentController.java:119) at java.base/java.lang.Thread.run(Thread.java:833) Caused by: org.geotools.data.DataSourceException at org.geotools.gce.geotiff.GeoTiffReader.getHRInfo(GeoTiffReader.java:539) at org.geotools.gce.geotiff.GeoTiffReader.(GeoTiffReader.java:265) ... 37 more Caused by: java.lang.ExceptionInInitializerError at com.sun.xml.bind.v2.runtime.reflect.opt.AccessorInjector.prepare(AccessorInjector.java:81) at com.sun.xml.bind.v2.runtime.reflect.opt.OptimizedAccessorFactory.get(OptimizedAccessorFactory.java:179) at com.sun.xml.bind.v2.runtime.reflect.Accessor$FieldReflection.optimize(Accessor.java:285) at com.sun.xml.bind.v2.runtime.property.ArrayProperty.(ArrayProperty.java:68) at com.sun.xml.bind.v2.runtime.property.ArrayERProperty.(ArrayERProperty.java:88) at com.sun.xml.bind.v2.runtime.property.ArrayElementProperty.(ArrayElementProperty.java:100) at com.sun.xml.bind.v2.runtime.property.ArrayElementNodeProperty.(ArrayElementNodeProperty.java:62) at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499) at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:480) at com.sun.xml.bind.v2.runtime.property.PropertyFactory.create(PropertyFactory.java:128) at com.sun.xml.bind.v2.runtime.ClassBeanInfoImpl.(ClassBeanInfoImpl.java:181) at com.sun.xml.bind.v2.runtime.JAXBContextImpl.getOrCreate(JAXBContextImpl.java:503) at com.sun.xml.bind.v2.runtime.JAXBContextImpl.(JAXBContextImpl.java:320) at com.sun.xml.bind.v2.runtime.JAXBContextImpl.(JAXBContextImpl.java:139) at com.sun.xml.bind.v2.runtime.JAXBContextImpl$JAXBContextBuilder.build(JAXBContextImpl.java:1138) at com.sun.xml.bind.v2.ContextFactory.createContext(ContextFactory.java:162) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at javax.xml.bind.ContextFinder.newInstance(ContextFinder.java:262) at javax.xml.bind.ContextFinder.newInstance(ContextFinder.java:249) at javax.xml.bind.ContextFinder.find(ContextFinder.java:456) at javax.xml.bind.JAXBContext.newInstance(JAXBContext.java:656) at javax.xml.bind.JAXBContext.newInstance(JAXBContext.java:599) at it.geosolutions.imageioimpl.plugins.tiff.gdal.GDALMetadataParser.(GDALMetadataParser.java:42) at it.geosolutions.imageioimpl.plugins.tiff.TIFFImageReader.initializeFromMetadata(TIFFImageReader.java:1303) at it.geosolutions.imageioimpl.plugins.tiff.TIFFImageReader.seekToImage(TIFFImageReader.java:831) at it.geosolutions.imageioimpl.plugins.tiff.TIFFImageReader.getImageMetadata(TIFFImageReader.java:1442) at org.geotools.gce.geotiff.GeoTiffReader.getHRInfo(GeoTiffReader.java:329) ... 38 more Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make protected final java.lang.Class java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int) throws java.lang.ClassFormatError accessible: module java.base does not "opens java.lang" to unnamed module @3a01773b at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:354) at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:297) at java.base/java.lang.reflect.Method.checkCanSetAccessible(Method.java:199) at java.base/java.lang.reflect.Method.setAccessible(Method.java:193) at com.sun.xml.bind.v2.runtime.reflect.opt.Injector.getMethod(Injector.java:184) at com.sun.xml.bind.v2.runtime.reflect.opt.Injector.access$000(Injector.java:69) at com.sun.xml.bind.v2.runtime.reflect.opt.Injector$1.run(Injector.java:168) at com.sun.xml.bind.v2.runtime.reflect.opt.Injector$1.run(Injector.java:165) at java.base/java.security.AccessController.doPrivileged(AccessController.java:318) at com.sun.xml.bind.v2.runtime.reflect.opt.Injector.(Injector.java:164) ... 71 more



**Desktop (please complete the following information):**
 - OS: macOS Big sur
 - GAMA version: git 1.8.2
 - Java version: 17
AlexisDrogoul commented 2 years ago

Can you try adding these options to the run configuration (or Gama.in) : 

--illegal-access=warn
--add-opens java.base/java.lang=ALL-UNNAMED

and report if it solves the problem ? In that case, we'll add this to the standard configuration.

benoitgaudou commented 2 years ago

Thanks for the solution. It seems to solve the issue: GAMA is now able to read the tiff file (both in the datafile visualiser and in a model).

A few remarks : I added the 2 options in the VM arguments (in the program arguments, it does not worked). I get the following message at run:

Java HotSpot(TM) 64-Bit Server VM warning: Ignoring option --illegal-access=warn; support was removed in 17.0

By the way, even if the file seems to be read properly, I still get the following exception (only in Eclipse console):

org.geotools.data.DataSourceException: Cannot invoke "Object.equals(Object)" because "unit" is null
    at org.geotools.gce.geotiff.GeoTiffReader.<init>(GeoTiffReader.java:279)
    at org.geotools.gce.geotiff.GeoTiffReader.<init>(GeoTiffReader.java:192)
    at msi.gama.util.file.GamaGridFile.getOwnCRS(GamaGridFile.java:610)
    at msi.gama.util.file.GamaGisFile.lambda$1(GamaGisFile.java:111)
    at com.google.common.cache.LocalCache$LocalManualCache$1.load(LocalCache.java:4853)
    at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3529)
    at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2278)
    at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2155)
    at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2045)
    at com.google.common.cache.LocalCache.get(LocalCache.java:3951)
    at com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4848)
    at msi.gama.util.file.GamaGisFile.getExistingCRS(GamaGisFile.java:92)
    at msi.gama.util.file.GamaGridFile.privateCreateCoverage(GamaGridFile.java:373)
    at msi.gama.util.file.GamaGridFile.createCoverage(GamaGridFile.java:253)
    at msi.gama.util.file.GamaGridFile.computeEnvelope(GamaGridFile.java:534)
    at msi.gama.common.geometry.GeometryUtils.computeEnvelopeFrom(GeometryUtils.java:1123)
    at msi.gaml.operators.Spatial$Creation.envelope(Spatial.java:2040)
    at gaml.additions.core.GamlAdditions.lambda$868(GamlAdditions.java:928)
    at msi.gaml.expressions.operators.UnaryOperator._value(UnaryOperator.java:98)
    at msi.gaml.expressions.AbstractExpression.value(AbstractExpression.java:81)
    at msi.gama.runtime.ExecutionScope.evaluate(ExecutionScope.java:552)
    at msi.gaml.variables.Variable.initializeWith(Variable.java:588)
    at msi.gama.metamodel.population.GamaPopulation.createVariablesFor(GamaPopulation.java:546)
    at msi.gama.kernel.simulation.SimulationPopulation.initSimulation(SimulationPopulation.java:158)
    at msi.gama.kernel.simulation.SimulationPopulation.createAgents(SimulationPopulation.java:119)
    at msi.gama.metamodel.population.IPopulation.createAgents(IPopulation.java:229)
    at msi.gama.kernel.experiment.ExperimentAgent.createSimulation(ExperimentAgent.java:357)
    at msi.gama.kernel.experiment.ExperimentAgent._init_(ExperimentAgent.java:316)
    at msi.gama.metamodel.agent.MinimalAgent.init(MinimalAgent.java:222)
    at msi.gama.kernel.experiment.ExperimentAgent.init(ExperimentAgent.java:333)
    at msi.gama.runtime.ExecutionScope.init(ExecutionScope.java:531)
    at msi.gama.kernel.experiment.ExperimentController.schedule(ExperimentController.java:279)
    at msi.gama.kernel.experiment.ExperimentAgent.schedule(ExperimentAgent.java:380)
    at msi.gama.kernel.experiment.ExperimentPlan.open(ExperimentPlan.java:564)
    at msi.gama.kernel.experiment.ExperimentPlan.open(ExperimentPlan.java:582)
    at msi.gama.kernel.experiment.ExperimentController.lambda$2(ExperimentController.java:119)
    at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: org.geotools.data.DataSourceException: Cannot invoke "Object.equals(Object)" because "unit" is null
    at org.geotools.gce.geotiff.GeoTiffReader.getHRInfo(GeoTiffReader.java:539)
    at org.geotools.gce.geotiff.GeoTiffReader.<init>(GeoTiffReader.java:265)
    ... 36 more
Caused by: java.lang.NullPointerException: Cannot invoke "Object.equals(Object)" because "unit" is null
    at org.geotools.referencing.cs.AbstractCS.axisUsingUnit(AbstractCS.java:501)
    at org.geotools.referencing.cs.DefaultCartesianCS.usingUnit(DefaultCartesianCS.java:290)
    at org.geotools.coverage.grid.io.imageio.geotiff.GeoTiffMetadata2CRSAdapter.createUserDefinedPCS(GeoTiffMetadata2CRSAdapter.java:740)
    at org.geotools.coverage.grid.io.imageio.geotiff.GeoTiffMetadata2CRSAdapter.createProjectedCoordinateReferenceSystem(GeoTiffMetadata2CRSAdapter.java:242)
    at org.geotools.coverage.grid.io.imageio.geotiff.GeoTiffMetadata2CRSAdapter.createCoordinateSystem(GeoTiffMetadata2CRSAdapter.java:181)
    at org.geotools.gce.geotiff.GeoTiffReader.getHRInfo(GeoTiffReader.java:355)
    ... 37 more
AlexisDrogoul commented 2 years ago

This one is for @ptaillandier 😀 (master of georeferenced TIFF files)

AlexisDrogoul commented 2 years ago

Can you commit the corresponding product files ?

AlexisDrogoul commented 2 years ago

I can observe the same error with what I assume is the same file. Have you tried playing with it in QGis and trying to force a "unit" (whatever this means) or use a standard projection ? The error seems to come from a place where GeoTools tries to translate a "user defined projection" to something standard.

Furthermore, it is clearly a GeoTools bug, because one of the ancestor methods says something about the unit parameter :

    /**
     * We have a user defined {@link ProjectedCRS}, let's try to parse it.
     *
     * @param linearUnit is the UoM that this {@link ProjectedCRS} will use. It could be null.
     * @return a user-defined {@link ProjectedCRS}.
     */
    private ProjectedCRS createUserDefinedPCS(
            final GeoTiffIIOMetadataDecoder metadata, Unit<?> linearUnit) throws Exception {

But fails when it is null.

And the reason why it is null is also unclear by the way. If you go back to the creation of this null unit, GeoTools launches a method called GeoTiffMetadata2CRSAdapter.createUnit(int, int, Unit<Q>, Unit<Q>, GeoTiffIIOMetadataDecoder), which retrieves the code EPSG:9001 from the metadata of the TIFF file, from which the unit should be retrieved in turn. However, this computation is interrupted by an exception raised because no authorities can find "EPSG"... The exact message of the hidden exception is : org.opengis.referencing.NoSuchAuthorityCodeException: Authority "EPSG" is unknown or doesn't match the supplied hints. Maybe it is defined in an unreachable JAR file?

AlexisDrogoul commented 2 years ago

@ptaillandier any clue on this one ?

benoitgaudou commented 2 years ago

Looking at the file using QGIS, the CRS seems to be EPSG:2154 - RGF93 / Lambert-93 - Projected and unit is set to meters ... it seems geotools is not able to retrieve it properly.

Screenshot 2022-03-10 at 05 08 00
ptaillandier commented 2 years ago

It seems to be linked to the prj of the geotiff. For a reason that I don't understand, GAMA is not able to find the good CRS. Two workarounds:

PROJCS["RGF93 / Lambert-93",GEOGCS["RGF93",DATUM["D_RGF_1993",SPHEROID["GRS_1980",6378137,298.257222101]],PRIMEM["Greenwich",0],UNIT["Degree",0.017453292519943295]],PROJECTION["Lambert_Conformal_Conic"],PARAMETER["standard_parallel_1",49],PARAMETER["standard_parallel_2",44],PARAMETER["latitude_of_origin",46.5],PARAMETER["central_meridian",3],PARAMETER["false_easting",700000],PARAMETER["false_northing",6600000],UNIT["Meter",1]]

ptaillandier commented 2 years ago

Now, GAMA throws a warning when it cannot read the CRS of the grid file (better than nothing). We can try to find a better solution for GAMA 2.0, but it is enough for GAMA 1.8.2

ptaillandier commented 2 years ago

It seems to be linked to the prj of the geotiff. For a reason that I don't understand, GAMA is not able to find the good CRS. Two workarounds:

Le jeu. 10 mars 2022 à 11:09, benoitgaudou @.***> a écrit :

Looking at the file using QGIS, the CRS seems to be EPSG:2154 - RGF93 / Lambert-93 - Projected and unit is set to meters ... it seems geotools is not able to retrieve it properly.

[image: Screenshot 2022-03-10 at 05 08 00] https://user-images.githubusercontent.com/1555274/157586651-833341c1-cd39-4f27-89de-f601edd828b3.png

— Reply to this email directly, view it on GitHub https://github.com/gama-platform/gama/issues/3304#issuecomment-1063644527, or unsubscribe https://github.com/notifications/unsubscribe-auth/AALPWHO7LQRCZO2LDOH6CVDU7FYZBANCNFSM5O6DY4TA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you were mentioned.Message ID: @.***>

AlexisDrogoul commented 1 year ago

I suggest to close this issue since emitting a warning because the CRS is not found is in itself a sufficient indication for modellers.