search

atolcd / pentaho-gis-plugins

🗺 GIS plugins for Pentaho Data Integration
https://www.atolcd.com/expertise/solutions-geographiques-open-source-sig
48 stars 20 forks source link

Type conversion error if properties instances have different numeric types in GeoJSON file #23

Open dlutz2 opened 4 years ago

dlutz2 commented 4 years ago

If different instance of a property have different numeric types (integer, decimal) then a type conversion error results:

For instance:

{
  "type": "FeatureCollection",
  "features": [
    {
      "type": "Feature",
      "geometry": {
        "type": "Point",
        "coordinates": [
          102.0,
          0.5
        ]
      },
      "properties": {
        "property1": 1.0
      }
    },
    {
      "type": "Feature",
      "geometry": {
        "type": "Point",
        "coordinates": [
          100.0,
          0.5
        ]
      },
      "properties": {
        "property1": 2
      }
    }
  ]
}

causes 

2020/07/20 01:53:55 - GIS File input.0 - ERROR (version 8.3.0.0-371, build 8.3.0.0-371 from 2019-06-11 11.09.08 by buildguy) : Unexpected error
2020/07/20 01:53:55 - GIS File input.0 - ERROR (version 8.3.0.0-371, build 8.3.0.0-371 from 2019-06-11 11.09.08 by buildguy) : org.pentaho.di.core.exception.KettleValueException: 
2020/07/20 01:53:55 - GIS File input.0 - Unexpected conversion error while converting value [property Number] to a Number
2020/07/20 01:53:55 - GIS File input.0 - java.lang.Integer cannot be cast to java.lang.Double
2020/07/20 01:53:55 - GIS File input.0 - 
2020/07/20 01:53:55 - GIS File input.0 -    at org.pentaho.di.core.row.value.ValueMetaBase.getNumber(ValueMetaBase.java:2035)
2020/07/20 01:53:55 - GIS File input.0 -    at com.atolcd.pentaho.di.gis.io.features.FeatureConverter.getRow(FeatureConverter.java:217)
2020/07/20 01:53:55 - GIS File input.0 -    at com.atolcd.pentaho.di.trans.steps.gisfileinput.GisFileInput.processRow(GisFileInput.java:106)
2020/07/20 01:53:55 - GIS File input.0 -    at org.pentaho.di.trans.step.RunThread.run(RunThread.java:62)
2020/07/20 01:53:55 - GIS File input.0 -    at java.lang.Thread.run(Thread.java:748)

I believe the cause of this error is the assumption in the GeoJSONReader constructor (line 68): org.wololo.geojson.Feature geoJsonfeature = ((FeatureCollection) json).getFeatures()[0];

which determines the structure of GeoJSON properties by looking at only the first feature. This will also cause any property that does not appear in the first record (or has a null value ) to be ignored. (see #22 for an issue handling null property values)

Using Kettle 8.3.0.0-371 and GIS Plugin V1.3.0