search

telefonicaid / fiware-cygnus

A connector in charge of persisting context data sources into other third-party databases and storage systems, creating a historical view of the context
https://fiware-cygnus.rtfd.io/
GNU Affero General Public License v3.0
65 stars 105 forks source link

Arcgis. Error when update a feature with geometry `esriGeometryPolyline` #2392

Open danielvillalbamota opened 1 month ago

danielvillalbamota commented 1 month ago

Description

Having a feature layer with geometry defined as "Geometry Type: esriGeometryPolyline" is raising some errors traces when I try to update an existing feature. Creating a feature is not a problem, I can create it properly, as do not need to check the geometry.

Example

When I try to update an existing feature which has an existing geometry that is not updated in the request (is not included in notified attributes) I get the following traces:

...
time=2024-07-23T15:52:58.018Z | lvl=DEBUG | corr=0677fb2f-92f2-4528-808e-...; cbnotif=1 | trans=ef67f73d-db7d-4221-9c6e-... | srv=N/A | subsrv=N/A | comp=cygnus-ngsi | op=isExpired | msg=com.telefonica.iot.cygnus.backends.arcgis.model.Credential[103] : Checking Credential expiration time: 2024-07-23T17:27:58.931
time=2024-07-23T15:52:58.018Z | lvl=DEBUG | corr=0677fb2f-92f2-4528-808e-...; cbnotif=1 | trans=ef67f73d-db7d-4221-9c6e-... | srv=N/A | subsrv=N/A | comp=cygnus-ngsi | op=getFeatureList | msg=com.telefonica.iot.cygnus.backends.arcgis.restutils.RestFeatureTable[194] : getFeatureList - Connecting Feature table: https://<arcgis_host>/server/rest/services/<path_to_feature_layer>
time=2024-07-23T15:52:58.018Z | lvl=DEBUG | corr=0677fb2f-92f2-4528-808e-...; cbnotif=1 | trans=ef67f73d-db7d-4221-9c6e-... | srv=N/A | subsrv=N/A | comp=cygnus-ngsi | op=requestHTTP | msg=com.telefonica.iot.cygnus.backends.arcgis.restutils.RestApi[179] : requesting HTTP: https://<arcgis_host>/server/rest/services/<path_to_feature_layer>/query?outSR=%7B%22wkid%22%3A4326%7D&outFields=*&resultOffset=0&where=fiwareid+IN+%28%27EstadoTrafico_1%27%29&token=nLAno...XT9ku-9&f=json---METHOD: GET---BODY:
time=2024-07-23T15:52:58.155Z | lvl=DEBUG | corr=0677fb2f-92f2-4528-808e-...; cbnotif=1 | trans=ef67f73d-db7d-4221-9c6e-... | srv=N/A | subsrv=N/A | comp=cygnus-ngsi | op=requestHTTP | msg=com.telefonica.iot.cygnus.backends.arcgis.restutils.RestApi[263] : Disposing connection objects
time=2024-07-23T15:52:58.155Z | lvl=DEBUG | corr=0677fb2f-92f2-4528-808e-...; cbnotif=1 | trans=ef67f73d-db7d-4221-9c6e-... | srv=N/A | subsrv=N/A | comp=cygnus-ngsi | op=getFeatureList | msg=com.telefonica.iot.cygnus.backends.arcgis.restutils.RestFeatureTable[216] : Response code: 200
time=2024-07-23T15:52:58.155Z | lvl=DEBUG | corr=0677fb2f-92f2-4528-808e-...; cbnotif=1 | trans=ef67f73d-db7d-4221-9c6e-... | srv=N/A | subsrv=N/A | comp=cygnus-ngsi | op=checkResponse | msg=com.telefonica.iot.cygnus.backends.arcgis.restutils.RestFeatureTable[238] : Feature/Feature action ended successfully
time=2024-07-23T15:52:58.156Z | lvl=ERROR | corr=0677fb2f-92f2-4528-808e-...; cbnotif=1 | trans=ef67f73d-db7d-4221-9c6e-... | srv=N/A | subsrv=N/A | comp=cygnus-ngsi | op=createInstanceFromJson | msg=com.telefonica.iot.cygnus.backends.arcgis.model.Point[145] : NullPointerException  Cannot invoke "com.google.gson.JsonElement.getAsDouble()" because the return value of "com.google.gson.JsonObject.get(String)" is null
time=2024-07-23T15:52:58.157Z | lvl=ERROR | corr=0677fb2f-92f2-4528-808e-...; cbnotif=1 | trans=ef67f73d-db7d-4221-9c6e-... | srv=N/A | subsrv=N/A | comp=cygnus-ngsi | op=queryFeatures | msg=com.telefonica.iot.cygnus.backends.arcgis.restutils.ArcgisFeatureTable[509] : Can't cast Feature from Json, ArcgisException - Unable to parse Point from json Cannot invoke "com.google.gson.JsonElement.getAsDouble()" because the return value of "com.google.gson.JsonObject.get(String)" is null
time=2024-07-23T15:52:58.157Z | lvl=ERROR | corr=0677fb2f-92f2-4528-808e-...; cbnotif=1 | trans=ef67f73d-db7d-4221-9c6e-... | srv=N/A | subsrv=N/A | comp=cygnus-ngsi | op=setError | msg=com.telefonica.iot.cygnus.backends.arcgis.restutils.ArcgisFeatureTable[596] : Error Message: Can't cast Feature from Json, ArcgisException - Unable to parse Point from json Cannot invoke "com.google.gson.JsonElement.getAsDouble()" because the return value of "com.google.gson.JsonObject.get(String)" is null
time=2024-07-23T15:52:58.157Z | lvl=ERROR | corr=0677fb2f-92f2-4528-808e-...; cbnotif=1 | trans=ef67f73d-db7d-4221-9c6e-... | srv=N/A | subsrv=N/A | comp=cygnus-ngsi | op=persistAggregation | msg=com.telefonica.iot.cygnus.sinks.NGSIArcgisFeatureTableSink[403] : [arcgis-sink-no-nm] Error persisting batch, NullPointerException - Cannot invoke "java.util.List.size()" because "serverFeatures" is null
time=2024-07-23T15:52:58.157Z | lvl=ERROR | corr=0677fb2f-92f2-4528-808e-...; cbnotif=1 | trans=ef67f73d-db7d-4221-9c6e-... | srv=N/A | subsrv=N/A | comp=cygnus-ngsi | op=persistBatch | msg=com.telefonica.iot.cygnus.sinks.NGSIArcgisFeatureTableSink[310] : [arcgis-sink-no-nm] Error persisting batch, CygnusRuntimeError.CygnusRuntimeError. Cannot invoke "java.util.List.size()" because "serverFeatures" is null.
time=2024-07-23T15:52:58.157Z | lvl=ERROR | corr=0677fb2f-92f2-4528-808e-...; cbnotif=1 | trans=ef67f73d-db7d-4221-9c6e-... | srv=N/A | subsrv=N/A | comp=cygnus-ngsi | op=processNewBatches | msg=com.telefonica.iot.cygnus.sinks.NGSISink[610] : CygnusRuntimeError. CygnusRuntimeError. Cannot invoke "java.util.List.size()" because "serverFeatures" is null. .  Sink: arcgis-sink-no-nm Destination: <path_to_feature_layer>_EstadoTrafico_1_fiwareid Stack trace: [com.telefonica.iot.cygnus.sinks.NGSIArcgisFeatureTableSink.persistBatch(NGSIArcgisFeatureTableSink.java:311), com.telefonica.iot.cygnus.sinks.NGSISink.processNewBatches(NGSISink.java:597), com.telefonica.iot.cygnus.sinks.NGSISink.process(NGSISink.java:372), com.telefonica.iot.cygnus.sinks.NGSIArcgisFeatureTableSink.process(NGSIArcgisFeatureTableSink.java:325), org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:39), org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:145), java.base/java.lang.Thread.run(Thread.java:840)]

This exception is ocurring in https://github.com/telefonicaid/fiware-cygnus/blob/master/cygnus-common/src/main/java/com/telefonica/iot/cygnus/backends/arcgis/model/Point.java#L136

This method is assuming that geometry is a Point but in this case is a polyline. Geometry objects in Arcgis

If I make the request https://<arcgis_host>/server/rest/services/<path_to_feature_layer>/query?outSR=%7B%22wkid%22%3A4326%7D&outFields=*&resultOffset=0&where=fiwareid+IN+%28%27EstadoTrafico_1%27%29&token=nLAno...XT9ku-9&f=json by the API I get the following body:

{
  "objectIdFieldName": "gid",
  "globalIdFieldName": "",
  "geometryType": "esriGeometryPolyline",
  "spatialReference": {
    "wkid": 4326,
    "latestWkid": 4326
  },
  "fields": [
    {
      "name": "gid",
      "alias": "gid",
      "type": "esriFieldTypeOID"
    },
    ...
    {
      "name": "fiwareid",
      "alias": "fiwareid",
      "type": "esriFieldTypeString",
      "length": 50
    }
  ],
  "features": [
    {
      "attributes": {
        "gid": 3...8,
        ...
        "fiwareid": "EstadoTrafico_1"
      },
      "geometry": {
        "paths": [
          [
            [
              -1.3,
              9.4
            ],
            [
              -1.31,
              9.41
            ]
          ]
        ]
      }
    }
  ]
}

So, the sink should manage different geometries.

AlvaroVega commented 1 month ago

https://github.com/telefonicaid/fiware-cygnus/blob/45972dee76d556ca54bdf8e6a30c9a46b85eb4c3/cygnus-common/src/main/java/com/telefonica/iot/cygnus/backends/arcgis/model/Feature.java#L314-L315

It seems only esri Geometry Point is supported, not "multipoint", "polyline", "polygon", "extent", "mesh"

fgalan commented 1 month ago

Fixed by PR https://github.com/telefonicaid/fiware-cygnus/pull/2394

However, it will be wise to let this issue opened for a while, so @danielvillalbamota could provide hopefully positive feedback before closing it.