telefonicaid / fiware-cygnus

A connector in charge of persisting context data sources into other third-party databases and storage systems, creating a historical view of the context
https://fiware-cygnus.rtfd.io/
GNU Affero General Public License v3.0
64 stars 104 forks source link

Cygnus breaks when attribute mapping is partially specified #1924

Closed fgalan closed 4 years ago

fgalan commented 4 years ago

If we have something like this in an attribute mapping:

              "attributeMappings": [{
                "originalAttributeType": "geox:json",
                "newAttributeType": "geo:json"
              }]

Then Cygnus breaks entirely. This appears in the traces:

time=2020-09-04T07:59:57.432Z | lvl=ERROR | corr=N/A | trans=N/A | srv=N/A | subsrv=N/A | comp=cygnus-ngsi | op=run | msg=org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable[251] : Unable to start EventDrivenSourceRunner: { source:org.apache.flume.source.http.HTTPSource{name:http-source-postgis,state:IDLE} } - Exception follows.
java.lang.NullPointerException
        at java.util.regex.Pattern.<init>(Pattern.java:1350)
        at java.util.regex.Pattern.compile(Pattern.java:1028)
        at com.telefonica.iot.cygnus.containers.NameMappings$AttributeMapping.compilePatterns(NameMappings.java:635)
        at com.telefonica.iot.cygnus.containers.NameMappings$EntityMapping.compilePatterns(NameMappings.java:505)
        at com.telefonica.iot.cygnus.containers.NameMappings$ServicePathMapping.compilePatterns(NameMappings.java:344)
        at com.telefonica.iot.cygnus.containers.NameMappings$ServiceMapping.compilePatterns(NameMappings.java:203)
        at com.telefonica.iot.cygnus.containers.NameMappings.compilePatterns(NameMappings.java:63)
        at com.telefonica.iot.cygnus.interceptors.NGSINameMappingsInterceptor.loadNameMappings(NGSINameMappingsInterceptor.java:287)
        at com.telefonica.iot.cygnus.interceptors.NGSINameMappingsInterceptor.loadNameMappings(NGSINameMappingsInterceptor.java:249)
        at com.telefonica.iot.cygnus.interceptors.NGSINameMappingsInterceptor.initialize(NGSINameMappingsInterceptor.java:69)
        at org.apache.flume.interceptor.InterceptorChain.initialize(InterceptorChain.java:74)
        at org.apache.flume.channel.ChannelProcessor.initialize(ChannelProcessor.java:68)
        at org.apache.flume.source.EventDrivenSourceRunner.start(EventDrivenSourceRunner.java:43)
        at org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:249)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
time=2020-09-04T07:59:58.824Z | lvl=DEBUG | corr=N/A | trans=N/A | srv=N/A | subsrv=N/A | comp=cygnus-ngsi | op=processNewBatches | msg=com.telefonica.iot.cygnus.sinks.NGSISink[502] : Batch accumulation time reached, the batch will be processed as it is

The poblem gets solve if we provide the 4 fields corresponding to the attribute mapping, i.e:

              "attributeMappings": [{
        "originalAttributeName": "polygon",
                "originalAttributeType": "geox:json",
        "newAttributeName": "polygon",              
                "newAttributeType": "geo:json"
              }]

Not sure which solution is better either:

but something about this should be done

fgalan commented 4 years ago

Now that PR #1941 has been merged, should this issue be closed?