Closed AlvaroVega closed 1 month ago
Stack trace: [com.telefonica.iot.cygnus.sinks.NGSIArcgisFeatureTableSink.persistBatch(NGSIArcgisFeatureTableSink.java:311), com.telefonica.iot.cygnus.sinks.NGSISink.processNewBatches(NGSISink.java:597), com.telefonica.iot.cygnus.sinks.NGSISink.process(NGSISink.java:372), com.telefonica.iot.cygnus.sinks.NGSIArcgisFeatureTableSink.process(NGSIArcgisFeatureTableSink.java:325), org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:39), org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:145), java.base/java.lang.Thread.run(Thread.java:840)] class java.net.SocketException time=2024-09-12T11:25:58.527Z | lvl=ERROR | corr=c1e15f7e-70f9-11ef-862c-0a580a8102cd; cbnotif=6 | trans=7f93b6ea-97df-4a57-b503-d4adb702baa2 | srv=N/A | subsrv=N/A | comp=cygnus-ngsi | op=processNewBatches | msg=com.telefonica.iot.cygnus.sinks.NGSISink[610] : CygnusRuntimeError. CygnusRuntimeError. Error: 0 Connection reset. class java.net.SocketException time=2024-09-12T11:25:58.526Z | lvl=ERROR | corr=c1e15f7e-70f9-11ef-862c-0a580a8102cd; cbnotif=6 | trans=7f93b6ea-97df-4a57-b503-d4adb702baa2 | srv=N/A | subsrv=N/A | comp=cygnus-ngsi | op=persistBatch | msg=com.telefonica.iot.cygnus.sinks.NGSIArcgisFeatureTableSink[310] : [sink-arcgis_geoportal_ayto_pro] Error persisting batch, CygnusRuntimeError.CygnusRuntimeError. Error: 0 Connection reset class java.net.SocketException time=2024-09-12T11:25:58.526Z | lvl=ERROR | corr=c1e15f7e-70f9-11ef-862c-0a580a8102cd; cbnotif=6 | trans=7f93b6ea-97df-4a57-b503-d4adb702baa2 | srv=N/A | subsrv=N/A | comp=cygnus-ngsi | op=persistAggregation | msg=com.telefonica.iot.cygnus.sinks.NGSIArcgisFeatureTableSink[403] : [sink-arcgis_geoportal_ayto_pro] Error persisting batch, ArcgisException - Error: 0 ConnectietException time=2024-09-12T11:25:58.526Z | lvl=ERROR | corr=c1e15f7e-70f9-11ef-862c-0a580a8102cd; cbnotif=6 | trans=7f93b6ea-97df-4a57-b503-d4adb702baa2 | srv=N/A | subsrv=N/A | comp=cygnus-ngsi | op=setError | msg=com.telefonica.iot.cygnus.backends.arcgis.restutils.ArcgisFeatureTable[596] : Error Message: Error: 0 Connection reset class java.net.SocketException time=2024-09-12T11:25:58.526Z | lvl=ERROR | corr=c1e15f7e-70f9-11ef-862c-0a580a8102cd; cbnotif=6 | trans=7f93b6ea-97df-4a57-b503-d4adb702baa2 | srv=N/A | subsrv=N/A | comp=cygnus-ngsi | op=commitFeatures | msg=com.telefonica.iot.cygnus.backends.arcgis.restutils.ArcgisFeatureTable[549] : Error: 0 Connection reset class java.net.SocketException time=2024-09-12T11:25:58.526Z | lvl=ERROR | corr=c1e15f7e-70f9-11ef-862c-0a580a8102cd; cbnotif=6 | trans=7f93b6ea-97df-4a57-b503-d4adb702baa2 | srv=N/A | subsrv=N/A | comp=cygnus-ngsi | op=checkResponse | msg=com.telefonica.iot.cygnus.backends.arcgis.restutils.RestFeatureTable[235] : Error: 0
Configuring a non existent arcgis endpoint, and sending events to arcgis sink I've got the following traces each time that batch is full or reaches timeout:
time=2024-09-24T09:12:05.250Z | lvl=DEBUG | corr=d9c3ce3f-211f-4036-ac8c-ad2ffcf8ec30; cbnotif=1 | trans=804cc855-a934-44c2-96a1-0aa9317507dd | srv=N/A | subsrv=N/A | comp=cygnus-ngsi | op=processNewBatches | msg=com.telefonica.iot.cygnus.sinks.NGSISink[583] : Batch completed
time=2024-09-24T09:12:05.251Z | lvl=DEBUG | corr=d9c3ce3f-211f-4036-ac8c-ad2ffcf8ec30; cbnotif=1 | trans=804cc855-a934-44c2-96a1-0aa9317507dd | srv=N/A | subsrv=N/A | comp=cygnus-ngsi | op=persistBatch | msg=com.telefonica.iot.cygnus.sinks.NGSIArcgisFeatureTableSink[291] : [arcgis-sink] Processing sub-batch regarding the smartcity_/_thing:disp1_thing destination
time=2024-09-24T09:12:05.251Z | lvl=DEBUG | corr=d9c3ce3f-211f-4036-ac8c-ad2ffcf8ec30; cbnotif=1 | trans=804cc855-a934-44c2-96a1-0aa9317507dd | srv=N/A | subsrv=N/A | comp=cygnus-ngsi | op=aggregate | msg=com.telefonica.iot.cygnus.sinks.NGSIArcgisFeatureTableSink$NGSIArcgisAggregator[457] : [NGSIArcgisAggregator] aggregate - ContextElement ->{"id":"thing:disp1","type":"thing","isPattern":"null","attributes":[{"name":"b","type":"Number","value":60,"metadatas":[{"name":"TimeInstant","type":"DateTime","value":"2024-09-23T09:50:28.502Z"}]},{"name":"precio","type":"number","value":66,"metadatas":[{"name":"TimeInstant","type":"DateTime","value":"2024-09-23T09:50:28.502Z"}]},{"name":"color","type":"string","value":"blue","metadatas":[{"name":"units","type":"text","value":"cms"},{"name":"TimeInstant","type":"DateTime","value":"2024-09-23T09:50:28.502Z"}]},{"name":"level","type":"Text","value":55,"metadatas":[{"name":"TimeInstant","type":"DateTime","value":"2024-09-23T09:50:28.502Z"}]},{"name":"temperature","type":"Number","value":40,"metadatas":[{"name":"TimeInstant","type":"DateTime","value":"2023-05-22T08:25:11.391Z"}]},{"name":"TimeInstant","type":"DateTime","value":"2024-09-24T09:11:28.502Z","metadatas":[]},{"name":"status","type":"Text","value":"ALARMA12","metadatas":[{"name":"TimeInstant","type":"DateTime","value":"2023-05-22T08:25:11.391Z"}]}]}
time=2024-09-24T09:12:05.252Z | lvl=DEBUG | corr=d9c3ce3f-211f-4036-ac8c-ad2ffcf8ec30; cbnotif=1 | trans=804cc855-a934-44c2-96a1-0aa9317507dd | srv=N/A | subsrv=N/A | comp=cygnus-ngsi | op=aggregate | msg=com.telefonica.iot.cygnus.sinks.NGSIArcgisFeatureTableSink$NGSIArcgisAggregator[458] : [NGSIArcgisAggregator] aggregate - MappedCE ->{"id":"thing:disp1","type":"thing","isPattern":"null","attributes":[{"name":"b","type":"Number","value":60,"metadatas":[{"name":"TimeInstant","type":"DateTime","value":"2024-09-23T09:50:28.502Z"}]},{"name":"precio","type":"number","value":66,"metadatas":[{"name":"TimeInstant","type":"DateTime","value":"2024-09-23T09:50:28.502Z"}]},{"name":"color","type":"string","value":"blue","metadatas":[{"name":"units","type":"text","value":"cms"},{"name":"TimeInstant","type":"DateTime","value":"2024-09-23T09:50:28.502Z"}]},{"name":"level","type":"Text","value":55,"metadatas":[{"name":"TimeInstant","type":"DateTime","value":"2024-09-23T09:50:28.502Z"}]},{"name":"temperature","type":"Number","value":40,"metadatas":[{"name":"TimeInstant","type":"DateTime","value":"2023-05-22T08:25:11.391Z"}]},{"name":"TimeInstant","type":"DateTime","value":"2024-09-24T09:11:28.502Z","metadatas":[]},{"name":"status","type":"Text","value":"ALARMA12","metadatas":[{"name":"TimeInstant","type":"DateTime","value":"2023-05-22T08:25:11.391Z"}]}]}
time=2024-09-24T09:12:05.253Z | lvl=DEBUG | corr=d9c3ce3f-211f-4036-ac8c-ad2ffcf8ec30; cbnotif=1 | trans=804cc855-a934-44c2-96a1-0aa9317507dd | srv=N/A | subsrv=N/A | comp=cygnus-ngsi | op=aggregate | msg=com.telefonica.iot.cygnus.sinks.NGSIArcgisFeatureTableSink$NGSIArcgisAggregator[459] : [NGSIArcgisAggregator] aggregate - OriginalCE ->{"id":"thing:disp1","type":"thing","isPattern":"null","attributes":[{"name":"b","type":"Number","value":60,"metadatas":[{"name":"TimeInstant","type":"DateTime","value":"2024-09-23T09:50:28.502Z"}]},{"name":"precio","type":"number","value":66,"metadatas":[{"name":"TimeInstant","type":"DateTime","value":"2024-09-23T09:50:28.502Z"}]},{"name":"color","type":"string","value":"blue","metadatas":[{"name":"units","type":"text","value":"cms"},{"name":"TimeInstant","type":"DateTime","value":"2024-09-23T09:50:28.502Z"}]},{"name":"level","type":"Text","value":55,"metadatas":[{"name":"TimeInstant","type":"DateTime","value":"2024-09-23T09:50:28.502Z"}]},{"name":"temperature","type":"Number","value":40,"metadatas":[{"name":"TimeInstant","type":"DateTime","value":"2023-05-22T08:25:11.391Z"}]},{"name":"TimeInstant","type":"DateTime","value":"2024-09-24T09:11:28.502Z","metadatas":[]},{"name":"status","type":"Text","value":"ALARMA12","metadatas":[{"name":"TimeInstant","type":"DateTime","value":"2023-05-22T08:25:11.391Z"}]}]}
time=2024-09-24T09:12:05.253Z | lvl=DEBUG | corr=d9c3ce3f-211f-4036-ac8c-ad2ffcf8ec30; cbnotif=1 | trans=804cc855-a934-44c2-96a1-0aa9317507dd | srv=N/A | subsrv=N/A | comp=cygnus-ngsi | op=aggregate | msg=com.telefonica.iot.cygnus.sinks.NGSIArcgisFeatureTableSink$NGSIArcgisAggregator[460] : [NGSIArcgisAggregator] aggregate - enableNameMappings status -> false
time=2024-09-24T09:12:05.253Z | lvl=DEBUG | corr=d9c3ce3f-211f-4036-ac8c-ad2ffcf8ec30; cbnotif=1 | trans=804cc855-a934-44c2-96a1-0aa9317507dd | srv=N/A | subsrv=N/A | comp=cygnus-ngsi | op=aggregate | msg=com.telefonica.iot.cygnus.sinks.NGSIArcgisFeatureTableSink$NGSIArcgisAggregator[464] : [NGSIArcgisAggregator] aggregate - creating new aggregation object.
time=2024-09-24T09:12:05.253Z | lvl=DEBUG | corr=d9c3ce3f-211f-4036-ac8c-ad2ffcf8ec30; cbnotif=1 | trans=804cc855-a934-44c2-96a1-0aa9317507dd | srv=N/A | subsrv=N/A | comp=cygnus-ngsi | op=
time=2024-09-24T09:12:05.268Z | lvl=DEBUG | corr=d9c3ce3f-211f-4036-ac8c-ad2ffcf8ec30; cbnotif=1 | trans=804cc855-a934-44c2-96a1-0aa9317507dd | srv=N/A | subsrv=N/A | comp=cygnus-ngsi | op=persistAggregation | msg=com.telefonica.iot.cygnus.sinks.NGSIArcgisFeatureTableSink[400] : PersistAggregation Error: com.telefonica.iot.cygnus.errors.CygnusRuntimeError: CygnusRuntimeError (com.telefonica.iot.cygnus.errors.CygnusRuntimeError). Error creating new persistence Backend. (CygnusRuntimeError. [arcgis-sinkError creating Persistence backend: -1 - { "responseCode" : 404, "responseMessage": "class java.net.UnknownHostException none.smartcity."} https://none.smartcity.es/server/rest/services/smartcity/ - https://none.smartcity./portal/sharing/rest/generateToken. ). at com.telefonica.iot.cygnus.sinks.NGSIArcgisFeatureTableSink.getPersistenceBackend(NGSIArcgisFeatureTableSink.java:195) at com.telefonica.iot.cygnus.sinks.NGSIArcgisFeatureTableSink.persistAggregation(NGSIArcgisFeatureTableSink.java:386) at com.telefonica.iot.cygnus.sinks.NGSIArcgisFeatureTableSink.persistBatch(NGSIArcgisFeatureTableSink.java:305) at com.telefonica.iot.cygnus.sinks.NGSISink.processNewBatches(NGSISink.java:597) at com.telefonica.iot.cygnus.sinks.NGSISink.process(NGSISink.java:372) at com.telefonica.iot.cygnus.sinks.NGSIArcgisFeatureTableSink.process(NGSIArcgisFeatureTableSink.java:325) at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:39) at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:145) at java.base/java.lang.Thread.run(Thread.java:840)
time=2024-09-24T09:12:05.268Z | lvl=ERROR | corr=d9c3ce3f-211f-4036-ac8c-ad2ffcf8ec30; cbnotif=1 | trans=804cc855-a934-44c2-96a1-0aa9317507dd | srv=N/A | subsrv=N/A | comp=cygnus-ngsi | op=persistBatch | msg=com.telefonica.iot.cygnus.sinks.NGSIArcgisFeatureTableSink[310] : [arcgis-sink] Error persisting batch, CygnusRuntimeError.CygnusRuntimeError (com.telefonica.iot.cygnus.errors.CygnusRuntimeError). Error creating new persistence Backend. (CygnusRuntimeError. [arcgis-sinkError creating Persistence backend: -1 - { "responseCode" : 404, "responseMessage": "class java.net.UnknownHostException none.smartcity."} https://none.smartcity.es/server/rest/services/smartcity/ - https://none.smartcity./portal/sharing/rest/generateToken. ). time=2024-09-24T09:12:05.268Z | lvl=ERROR | corr=d9c3ce3f-211f-4036-ac8c-ad2ffcf8ec30; cbnotif=1 | trans=804cc855-a934-44c2-96a1-0aa9317507dd | srv=N/A | subsrv=N/A | comp=cygnus-ngsi | op=processNewBatches | msg=com.telefonica.iot.cygnus.sinks.NGSISink[610] : CygnusRuntimeError. CygnusRuntimeError (com.telefonica.iot.cygnus.errors.CygnusRuntimeError). Error creating new persistence Backend. (CygnusRuntimeError. [arcgis-sinkError creating Persistence backend: -1 - { "responseCode" : 404, "responseMessage": "class java.net.UnknownHostException none.smartcity."} https://none.smartcity.es/server/rest/services/smartcity/ - https://none.smartcity./portal/sharing/rest/generateToken. ). . Sink: arcgis-sink Destination: smartcity_/_thing:disp1_thing Stack trace: [com.telefonica.iot.cygnus.sinks.NGSIArcgisFeatureTableSink.persistBatch(NGSIArcgisFeatureTableSink.java:311), com.telefonica.iot.cygnus.sinks.NGSISink.processNewBatches(NGSISink.java:597), com.telefonica.iot.cygnus.sinks.NGSISink.process(NGSISink.java:372), com.telefonica.iot.cygnus.sinks.NGSIArcgisFeatureTableSink.process(NGSIArcgisFeatureTableSink.java:325), org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:39), org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:145), java.base/java.lang.Thread.run(Thread.java:840)] time=2024-09-24T09:12:05.268Z | lvl=DEBUG | corr=N/A | trans=N/A | srv=N/A | subsrv=N/A | comp=cygnus-ngsi | op=checkTimeouts | msg=com.telefonica.iot.cygnus.sinks.NGSIArcgisFeatureTableSink[346] : [arcgis-sink] No Feature table Timeouts found. Features in batch: 0
TBD: more info is needed:
Arcgis sink is using a custom aggregator (NGSIArcgisAggregator). So main debug logs are produced by this aggregator.