search

telefonicaid / fiware-cygnus

A connector in charge of persisting context data sources into other third-party databases and storage systems, creating a historical view of the context
https://fiware-cygnus.rtfd.io/
GNU Affero General Public License v3.0
64 stars 105 forks source link

Agent is not accepting configuration parameters (docker) #1696

Closed jmezzera closed 4 years ago

jmezzera commented 5 years ago

Im running cygnus-ngsi on Docker (latest image, ID 6ae9dce7db6f) on a CentOS 7 machine, mounting the config file as a volume and I'm not being able to send notifications to the API endpoint. This instance is new, so there was no point in which it worked correctly.

The problem began when I created a subscription on Orion CB to cygnus. I updated the CB entity and the subscription status changed to failed with a reason of "Couldn't connect to server". Digging a little bit deeper I found out that getting the API/v1/stats responded with:

{
    "success": "true",
    "stats": {
        "sources": [],
        "channels": [],
        "sinks": []
    }
}

That pointed me to what may be the problem. I think that maybe cygnus is not getting the configuration from the file. (#1496 could have something to do with this issue). However, I have two reasons to believe that the cygnus process is doing something with the config file I'm trying to mount:

Some interesting information it may help: Entering the container and curling 127.0.0.1:5050/notify gets a connection refused. I know that the /notify endpoint should not receive a GET request, however, I looked the source code of the NGSIRestHandler and, according to lines 261, 263, 268; if this were to happen (a request that is not a POST) it should log it and respond with 400. So this makes me think the agent is not listening.

Any help will be appreciated. Thanks in advance

jmezzera commented 5 years ago

So, after a few days of trial and error I made some progress in figuring out what the underlying cause of this issue might be. Every time I restart the container, the config file /opt/apache-flume/conf/agent.conf gets overridden with a template which looks like this:

cygnus-ngsi.sources =
cygnus-ngsi.sinks =
cygnus-ngsi.channels =

followed by a template configuration for all sinks and channels. This clearly explains the behaviour stated in the original post about the response of the /stats endpoint.

Changing the file from the container itself, seems to make some changes, the log states that the sources, channels and sinks are created, but it then crashes with some JAVA runtime errors. The logs are the following:

cygnus          | 19/07/11 22:45:43 INFO node.PollingPropertiesFileConfigurationProvider: Reloading configuration file:/opt/apache-flume/conf/agent.conf
cygnus          | 19/07/11 22:45:43 INFO conf.FlumeConfiguration: Processing:mysql-channel
cygnus          | 19/07/11 22:45:43 INFO conf.FlumeConfiguration: Processing:mysql-sink
cygnus          | 19/07/11 22:45:43 INFO conf.FlumeConfiguration: Processing:http-source
cygnus          | 19/07/11 22:45:43 INFO conf.FlumeConfiguration: Processing:mysql-sink
cygnus          | 19/07/11 22:45:43 INFO conf.FlumeConfiguration: Processing:mongo-channel
cygnus          | 19/07/11 22:45:43 INFO conf.FlumeConfiguration: Processing:http-source
cygnus          | 19/07/11 22:45:43 INFO conf.FlumeConfiguration: Processing:mysql-sink
cygnus          | 19/07/11 22:45:43 INFO conf.FlumeConfiguration: Added sinks: mysql-sink mongo-sink Agent: cygnusagent
cygnus          | 19/07/11 22:45:43 INFO conf.FlumeConfiguration: Processing:mongo-sink
cygnus          | 19/07/11 22:45:43 INFO conf.FlumeConfiguration: Processing:http-source
cygnus          | 19/07/11 22:45:43 INFO conf.FlumeConfiguration: Processing:mongo-channel
cygnus          | 19/07/11 22:45:43 INFO conf.FlumeConfiguration: Processing:mysql-sink
cygnus          | 19/07/11 22:45:43 INFO conf.FlumeConfiguration: Processing:mongo-sink
cygnus          | 19/07/11 22:45:43 INFO conf.FlumeConfiguration: Processing:mysql-sink
cygnus          | 19/07/11 22:45:43 INFO conf.FlumeConfiguration: Processing:http-source
cygnus          | 19/07/11 22:45:43 INFO conf.FlumeConfiguration: Processing:http-source
cygnus          | 19/07/11 22:45:43 INFO conf.FlumeConfiguration: Processing:mongo-sink
cygnus          | 19/07/11 22:45:43 INFO conf.FlumeConfiguration: Processing:http-source
cygnus          | 19/07/11 22:45:43 INFO conf.FlumeConfiguration: Processing:mysql-channel
cygnus          | 19/07/11 22:45:43 INFO conf.FlumeConfiguration: Processing:mysql-sink
cygnus          | 19/07/11 22:45:43 INFO conf.FlumeConfiguration: Processing:mysql-channel
cygnus          | 19/07/11 22:45:43 INFO conf.FlumeConfiguration: Processing:http-source
cygnus          | 19/07/11 22:45:43 INFO conf.FlumeConfiguration: Processing:http-source
cygnus          | 19/07/11 22:45:43 INFO conf.FlumeConfiguration: Processing:mongo-channel
cygnus          | 19/07/11 22:45:43 INFO conf.FlumeConfiguration: Processing:http-source
cygnus          | 19/07/11 22:45:43 INFO conf.FlumeConfiguration: Processing:http-source
cygnus          | 19/07/11 22:45:43 INFO conf.FlumeConfiguration: Processing:mysql-sink
cygnus          | 19/07/11 22:45:43 INFO conf.FlumeConfiguration: Processing:http-source
cygnus          | 19/07/11 22:45:43 INFO conf.FlumeConfiguration: Processing:http-source
cygnus          | 19/07/11 22:45:43 INFO conf.FlumeConfiguration: Processing:http-source
cygnus          | 19/07/11 22:45:43 WARN conf.FlumeConfiguration: Agent configuration for 'cygnusagent' has no configfilters.
cygnus          | 19/07/11 22:45:43 INFO conf.FlumeConfiguration: Post-validation flume configuration contains configuration for agents: [cygnusagent]
cygnus          | 19/07/11 22:45:43 INFO node.AbstractConfigurationProvider: Creating channels
cygnus          | 19/07/11 22:45:43 INFO channel.DefaultChannelFactory: Creating instance of channel mongo-channel type memory
cygnus          | 19/07/11 22:45:43 INFO node.AbstractConfigurationProvider: Created channel mongo-channel
cygnus          | 19/07/11 22:45:43 INFO channel.DefaultChannelFactory: Creating instance of channel mysql-channel type memory
cygnus          | 19/07/11 22:45:43 INFO node.AbstractConfigurationProvider: Created channel mysql-channel
cygnus          | 19/07/11 22:45:43 INFO source.DefaultSourceFactory: Creating instance of source http-source, type org.apache.flume.source.http.HTTPSource
cygnus          | 19/07/11 22:45:43 INFO handlers.NGSIRestHandler: [NGSIRestHandler] Startup completed
cygnus          | 19/07/11 22:45:43 INFO sink.DefaultSinkFactory: Creating instance of sink: mongo-sink, type: com.telefonica.iot.cygnus.sinks.NGSIMongoSink
cygnus          | 19/07/11 22:45:43 INFO sink.DefaultSinkFactory: Creating instance of sink: mysql-sink, type: com.telefonica.iot.cygnus.sinks.NGSIMySQLSink
cygnus          | 19/07/11 22:45:43 INFO node.AbstractConfigurationProvider: Channel mongo-channel connected to [http-source, mongo-sink]
cygnus          | 19/07/11 22:45:43 INFO node.AbstractConfigurationProvider: Channel mysql-channel connected to [http-source, mysql-sink]
cygnus          | 19/07/11 22:45:43 INFO node.Application: Shutting down configuration: { sourceRunners:{} sinkRunners:{} channels:{} }
cygnus          | 19/07/11 22:45:43 INFO node.Application: Starting new configuration:{ sourceRunners:{http-source=EventDrivenSourceRunner: { source:org.apache.flume.source.http.HTTPSource{name:http-source,state:IDLE} }} sinkRunners:{mongo-sink=SinkRunner: { policy:org.apache.flume.sink.DefaultSinkProcessor@77894a79 counterGroup:{ name:null counters:{} } }, mysql-sink=SinkRunner: { policy:org.apache.flume.sink.DefaultSinkProcessor@2adb1d79 counterGroup:{ name:null counters:{} } }} channels:{mongo-channel=org.apache.flume.channel.MemoryChannel{name: mongo-channel}, mysql-channel=org.apache.flume.channel.MemoryChannel{name: mysql-channel}} }
cygnus          | 19/07/11 22:45:43 INFO node.Application: Starting Channel mongo-channel
cygnus          | 19/07/11 22:45:43 INFO node.Application: Starting Channel mysql-channel
cygnus          | 19/07/11 22:45:43 INFO instrumentation.MonitoredCounterGroup: Monitored counter group for type: CHANNEL, name: mongo-channel: Successfully registered new MBean.
cygnus          | 19/07/11 22:45:43 INFO instrumentation.MonitoredCounterGroup: Component type: CHANNEL, name: mongo-channel started
cygnus          | 19/07/11 22:45:43 INFO instrumentation.MonitoredCounterGroup: Monitored counter group for type: CHANNEL, name: mysql-channel: Successfully registered new MBean.
cygnus          | 19/07/11 22:45:43 INFO instrumentation.MonitoredCounterGroup: Component type: CHANNEL, name: mysql-channel started
cygnus          | 19/07/11 22:45:43 INFO node.Application: Starting Sink mongo-sink
cygnus          | 19/07/11 22:45:43 INFO node.Application: Starting Sink mysql-sink
cygnus          | 19/07/11 22:45:43 INFO node.Application: Starting Source http-source
cygnus          | 19/07/11 22:45:43 INFO node.Application: Shutting down configuration: { sourceRunners:{http-source=EventDrivenSourceRunner: { source:org.apache.flume.source.http.HTTPSource{name:http-source,state:IDLE} }} sinkRunners:{mongo-sink=SinkRunner: { policy:org.apache.flume.sink.DefaultSinkProcessor@77894a79 counterGroup:{ name:null counters:{} } }, mysql-sink=SinkRunner: { policy:org.apache.flume.sink.DefaultSinkProcessor@2adb1d79 counterGroup:{ name:null counters:{} } }} channels:{mongo-channel=org.apache.flume.channel.MemoryChannel{name: mongo-channel}, mysql-channel=org.apache.flume.channel.MemoryChannel{name: mysql-channel}} }
cygnus          | 19/07/11 22:45:43 INFO node.Application: Stopping Source http-source
cygnus          | 19/07/11 22:45:43 INFO lifecycle.LifecycleSupervisor: Stopping component: EventDrivenSourceRunner: { source:org.apache.flume.source.http.HTTPSource{name:http-source,state:IDLE} }
cygnus          | 19/07/11 22:45:43 ERROR http.HTTPSource: Error while stopping HTTPSource. Exception follows.
cygnus          | java.lang.NullPointerException
cygnus          |       at org.apache.flume.source.http.HTTPSource.stop(HTTPSource.java:216)
cygnus          |       at org.apache.flume.source.EventDrivenSourceRunner.stop(EventDrivenSourceRunner.java:51)
cygnus          |       at org.apache.flume.lifecycle.LifecycleSupervisor.unsupervise(LifecycleSupervisor.java:170)
cygnus          |       at org.apache.flume.node.Application.stopAllComponents(Application.java:130)
cygnus          |       at org.apache.flume.node.Application.handleConfigurationEvent(Application.java:97)
cygnus          |       at com.telefonica.iot.cygnus.nodes.CygnusApplication.handleConfigurationEvent(CygnusApplication.java:155)
cygnus          |       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
cygnus          |       at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
cygnus          |       at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
cygnus          |       at java.lang.reflect.Method.invoke(Method.java:498)
cygnus          |       at com.google.common.eventbus.EventHandler.handleEvent(EventHandler.java:68)
cygnus          |       at com.google.common.eventbus.SynchronizedEventHandler.handleEvent(SynchronizedEventHandler.java:45)
cygnus          |       at com.google.common.eventbus.EventBus.dispatch(EventBus.java:313)
cygnus          |       at com.google.common.eventbus.EventBus.dispatchQueuedEvents(EventBus.java:296)
cygnus          |       at com.google.common.eventbus.EventBus.post(EventBus.java:264)
cygnus          |       at org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run(PollingPropertiesFileConfigurationProvider.java:145)
cygnus          |       at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
cygnus          |       at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
cygnus          |       at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
cygnus          |       at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
cygnus          |       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
cygnus          |       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
cygnus          |       at java.lang.Thread.run(Thread.java:748)
cygnus          | 19/07/11 22:45:43 INFO instrumentation.MonitoredCounterGroup: Component type: SOURCE, name: http-source stopped
cygnus          | 19/07/11 22:45:43 INFO instrumentation.MonitoredCounterGroup: Shutdown Metric for type: SOURCE, name: http-source. source.start.time == 0
cygnus          | 19/07/11 22:45:43 INFO instrumentation.MonitoredCounterGroup: Shutdown Metric for type: SOURCE, name: http-source. source.stop.time == 1562885143545
cygnus          | 19/07/11 22:45:43 INFO instrumentation.MonitoredCounterGroup: Shutdown Metric for type: SOURCE, name: http-source. src.append-batch.accepted == 0
cygnus          | 19/07/11 22:45:43 INFO instrumentation.MonitoredCounterGroup: Shutdown Metric for type: SOURCE, name: http-source. src.append-batch.received == 0
cygnus          | 19/07/11 22:45:43 INFO instrumentation.MonitoredCounterGroup: Shutdown Metric for type: SOURCE, name: http-source. src.append.accepted == 0
cygnus          | 19/07/11 22:45:43 INFO instrumentation.MonitoredCounterGroup: Shutdown Metric for type: SOURCE, name: http-source. src.append.received == 0
cygnus          | 19/07/11 22:45:43 INFO instrumentation.MonitoredCounterGroup: Shutdown Metric for type: SOURCE, name: http-source. src.channel.write.fail == 0
cygnus          | 19/07/11 22:45:43 INFO instrumentation.MonitoredCounterGroup: Shutdown Metric for type: SOURCE, name: http-source. src.event.read.fail == 0
cygnus          | 19/07/11 22:45:43 INFO instrumentation.MonitoredCounterGroup: Shutdown Metric for type: SOURCE, name: http-source. src.events.accepted == 0
cygnus          | 19/07/11 22:45:43 INFO instrumentation.MonitoredCounterGroup: Shutdown Metric for type: SOURCE, name: http-source. src.events.received == 0
cygnus          | 19/07/11 22:45:43 INFO instrumentation.MonitoredCounterGroup: Shutdown Metric for type: SOURCE, name: http-source. src.generic.processing.fail == 0
cygnus          | 19/07/11 22:45:43 INFO instrumentation.MonitoredCounterGroup: Shutdown Metric for type: SOURCE, name: http-source. src.open-connection.count == 0
cygnus          | 19/07/11 22:45:43 INFO http.HTTPSource: Http source http-source stopped. Metrics: SOURCE:http-source{src.events.accepted=0, src.open-connection.count=0, src.append.received=0, src.channel.write.fail=0, src.append-batch.received=0, src.generic.processing.fail=0, src.append-batch.accepted=0, src.append.accepted=0, src.events.received=0, src.event.read.fail=0}
cygnus          | 19/07/11 22:45:43 ERROR node.Application: Error while stopping EventDrivenSourceRunner: { source:org.apache.flume.source.http.HTTPSource{name:http-source,state:IDLE} }
cygnus          | java.lang.NullPointerException
cygnus          |       at com.telefonica.iot.cygnus.interceptors.NGSIGroupingInterceptor.close(NGSIGroupingInterceptor.java:157)
cygnus          |       at org.apache.flume.interceptor.InterceptorChain.close(InterceptorChain.java:83)
cygnus          |       at org.apache.flume.channel.ChannelProcessor.close(ChannelProcessor.java:72)
cygnus          |       at org.apache.flume.source.EventDrivenSourceRunner.stop(EventDrivenSourceRunner.java:53)
cygnus          |       at org.apache.flume.lifecycle.LifecycleSupervisor.unsupervise(LifecycleSupervisor.java:170)
cygnus          |       at org.apache.flume.node.Application.stopAllComponents(Application.java:130)
cygnus          |       at org.apache.flume.node.Application.handleConfigurationEvent(Application.java:97)
cygnus          |       at com.telefonica.iot.cygnus.nodes.CygnusApplication.handleConfigurationEvent(CygnusApplication.java:155)
cygnus          |       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
cygnus          |       at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
cygnus          |       at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
cygnus          |       at java.lang.reflect.Method.invoke(Method.java:498)
cygnus          |       at com.google.common.eventbus.EventHandler.handleEvent(EventHandler.java:68)
cygnus          |       at com.google.common.eventbus.SynchronizedEventHandler.handleEvent(SynchronizedEventHandler.java:45)
cygnus          |       at com.google.common.eventbus.EventBus.dispatch(EventBus.java:313)
cygnus          |       at com.google.common.eventbus.EventBus.dispatchQueuedEvents(EventBus.java:296)
cygnus          |       at com.google.common.eventbus.EventBus.post(EventBus.java:264)
cygnus          |       at org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run(PollingPropertiesFileConfigurationProvider.java:145)
cygnus          |       at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
cygnus          |       at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
cygnus          |       at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
cygnus          |       at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
cygnus          |       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
cygnus          |       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
cygnus          |       at java.lang.Thread.run(Thread.java:748)
cygnus          | 19/07/11 22:45:43 INFO node.Application: Stopping Sink mongo-sink
cygnus          | 19/07/11 22:45:43 INFO sinks.NGSISink: [mongo-sink] Startup completed
cygnus          | 19/07/11 22:45:43 INFO lifecycle.LifecycleSupervisor: Component has already been stopped EventDrivenSourceRunner: { source:org.apache.flume.source.http.HTTPSource{name:http-source,state:IDLE} }
cygnus          | 19/07/11 22:45:43 INFO lifecycle.LifecycleSupervisor: Stopping component: SinkRunner: { policy:org.apache.flume.sink.DefaultSinkProcessor@77894a79 counterGroup:{ name:null counters:{} } }
cygnus          | 19/07/11 22:45:43 INFO sinks.NGSISink: [mysql-sink] Startup completed
cygnus          | 19/07/11 22:45:43 INFO node.Application: Stopping Sink mysql-sink
cygnus          | 19/07/11 22:45:43 INFO lifecycle.LifecycleSupervisor: Stopping component: SinkRunner: { policy:org.apache.flume.sink.DefaultSinkProcessor@2adb1d79 counterGroup:{ name:null counters:{} } }
cygnus          | 19/07/11 22:45:43 INFO node.Application: Stopping Channel mongo-channel
cygnus          | 19/07/11 22:45:43 INFO lifecycle.LifecycleSupervisor: Stopping component: org.apache.flume.channel.MemoryChannel{name: mongo-channel}
cygnus          | 19/07/11 22:45:43 INFO instrumentation.MonitoredCounterGroup: Component type: CHANNEL, name: mongo-channel stopped
cygnus          | 19/07/11 22:45:43 INFO instrumentation.MonitoredCounterGroup: Shutdown Metric for type: CHANNEL, name: mongo-channel. channel.start.time == 1562885143536
cygnus          | 19/07/11 22:45:43 INFO instrumentation.MonitoredCounterGroup: Shutdown Metric for type: CHANNEL, name: mongo-channel. channel.stop.time == 1562885143586
cygnus          | 19/07/11 22:45:43 INFO instrumentation.MonitoredCounterGroup: Shutdown Metric for type: CHANNEL, name: mongo-channel. channel.capacity == 100000
cygnus          | 19/07/11 22:45:43 INFO instrumentation.MonitoredCounterGroup: Shutdown Metric for type: CHANNEL, name: mongo-channel. channel.current.size == 0
cygnus          | 19/07/11 22:45:43 INFO instrumentation.MonitoredCounterGroup: Shutdown Metric for type: CHANNEL, name: mongo-channel. channel.event.put.attempt == 0
cygnus          | 19/07/11 22:45:43 INFO instrumentation.MonitoredCounterGroup: Shutdown Metric for type: CHANNEL, name: mongo-channel. channel.event.put.success == 0
cygnus          | 19/07/11 22:45:43 INFO instrumentation.MonitoredCounterGroup: Shutdown Metric for type: CHANNEL, name: mongo-channel. channel.event.take.attempt == 0
cygnus          | 19/07/11 22:45:43 INFO instrumentation.MonitoredCounterGroup: Shutdown Metric for type: CHANNEL, name: mongo-channel. channel.event.take.success == 0
cygnus          | 19/07/11 22:45:43 INFO node.Application: Stopping Channel mysql-channel
cygnus          | 19/07/11 22:45:43 INFO lifecycle.LifecycleSupervisor: Stopping component: org.apache.flume.channel.MemoryChannel{name: mysql-channel}
cygnus          | 19/07/11 22:45:43 INFO instrumentation.MonitoredCounterGroup: Component type: CHANNEL, name: mysql-channel stopped
cygnus          | 19/07/11 22:45:43 INFO instrumentation.MonitoredCounterGroup: Shutdown Metric for type: CHANNEL, name: mysql-channel. channel.start.time == 1562885143537
cygnus          | 19/07/11 22:45:43 INFO instrumentation.MonitoredCounterGroup: Shutdown Metric for type: CHANNEL, name: mysql-channel. channel.stop.time == 1562885143587
cygnus          | 19/07/11 22:45:43 INFO instrumentation.MonitoredCounterGroup: Shutdown Metric for type: CHANNEL, name: mysql-channel. channel.capacity == 100000
cygnus          | 19/07/11 22:45:43 INFO instrumentation.MonitoredCounterGroup: Shutdown Metric for type: CHANNEL, name: mysql-channel. channel.current.size == 0
cygnus          | 19/07/11 22:45:43 INFO instrumentation.MonitoredCounterGroup: Shutdown Metric for type: CHANNEL, name: mysql-channel. channel.event.put.attempt == 0
cygnus          | 19/07/11 22:45:43 INFO instrumentation.MonitoredCounterGroup: Shutdown Metric for type: CHANNEL, name: mysql-channel. channel.event.put.success == 0
cygnus          | 19/07/11 22:45:43 INFO instrumentation.MonitoredCounterGroup: Shutdown Metric for type: CHANNEL, name: mysql-channel. channel.event.take.attempt == 0
cygnus          | 19/07/11 22:45:43 INFO instrumentation.MonitoredCounterGroup: Shutdown Metric for type: CHANNEL, name: mysql-channel. channel.event.take.success == 0
cygnus          | 19/07/11 22:45:43 INFO node.Application: Starting new configuration:{ sourceRunners:{http-source=EventDrivenSourceRunner: { source:org.apache.flume.source.http.HTTPSource{name:http-source,state:IDLE} }} sinkRunners:{mongo-sink=SinkRunner: { policy:org.apache.flume.sink.DefaultSinkProcessor@77894a79 counterGroup:{ name:null counters:{} } }, mysql-sink=SinkRunner: { policy:org.apache.flume.sink.DefaultSinkProcessor@2adb1d79 counterGroup:{ name:null counters:{} } }} channels:{mongo-channel=org.apache.flume.channel.MemoryChannel{name: mongo-channel}, mysql-channel=org.apache.flume.channel.MemoryChannel{name: mysql-channel}} }
cygnus          | 19/07/11 22:45:43 INFO node.Application: Starting Channel mongo-channel
cygnus          | 19/07/11 22:45:43 INFO node.Application: Starting Channel mysql-channel
cygnus          | 19/07/11 22:45:43 INFO node.Application: Waiting for channel: mongo-channel to start. Sleeping for 500 ms
cygnus          | 19/07/11 22:45:43 INFO instrumentation.MonitoredCounterGroup: Component type: CHANNEL, name: mongo-channel started
cygnus          | 19/07/11 22:45:43 INFO instrumentation.MonitoredCounterGroup: Component type: CHANNEL, name: mysql-channel started
cygnus          | 19/07/11 22:45:44 INFO node.Application: Starting Sink mongo-sink
cygnus          | 19/07/11 22:45:44 INFO node.Application: Starting Sink mysql-sink
cygnus          | 19/07/11 22:45:44 INFO node.Application: Starting Source http-source
cygnus          | 19/07/11 22:45:44 ERROR node.Application: Error while starting EventDrivenSourceRunner: { source:org.apache.flume.source.http.HTTPSource{name:http-source,state:IDLE} }
cygnus          | java.lang.IllegalStateException: Refusing to supervise EventDrivenSourceRunner: { source:org.apache.flume.source.http.HTTPSource{name:http-source,state:IDLE} } more than once
cygnus          |       at com.google.common.base.Preconditions.checkState(Preconditions.java:145)
cygnus          |       at org.apache.flume.lifecycle.LifecycleSupervisor.supervise(LifecycleSupervisor.java:131)
cygnus          |       at org.apache.flume.node.Application.startAllComponents(Application.java:208)
cygnus          |       at org.apache.flume.node.Application.handleConfigurationEvent(Application.java:98)
cygnus          |       at com.telefonica.iot.cygnus.nodes.CygnusApplication.handleConfigurationEvent(CygnusApplication.java:155)
cygnus          |       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
cygnus          |       at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
cygnus          |       at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
cygnus          |       at java.lang.reflect.Method.invoke(Method.java:498)
cygnus          |       at com.google.common.eventbus.EventHandler.handleEvent(EventHandler.java:68)
cygnus          |       at com.google.common.eventbus.SynchronizedEventHandler.handleEvent(SynchronizedEventHandler.java:45)
cygnus          |       at com.google.common.eventbus.EventBus.dispatch(EventBus.java:313)
cygnus          |       at com.google.common.eventbus.EventBus.dispatchQueuedEvents(EventBus.java:296)
cygnus          |       at com.google.common.eventbus.EventBus.post(EventBus.java:264)
cygnus          |       at org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run(PollingPropertiesFileConfigurationProvider.java:145)
cygnus          |       at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
cygnus          |       at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
cygnus          |       at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
cygnus          |       at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
cygnus          |       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
cygnus          |       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
cygnus          |       at java.lang.Thread.run(Thread.java:748)
cygnus          | 19/07/11 22:45:44 INFO sinks.NGSISink: [mongo-sink] Startup completed
cygnus          | 19/07/11 22:45:44 INFO sinks.NGSISink: [mysql-sink] Startup completed

Any suggestions?

jmezzera commented 5 years ago

I got around this problem by mounting the volume as read-only. However, I think this cannot be the correct approach.

AlvaroVega commented 5 years ago

@jmezzera could you provide full detail of how do you start cygnus docker container like all cygnus env vars that you are using? Depending on your configuration, if you are providen external agent.conf file to docker, then you should ensure you have set CYGNUS_SKIP_CONF_GENERATION env var to true. https://github.com/telefonicaid/fiware-cygnus/blob/master/doc/cygnus-ngsi/installation_and_administration_guide/install_with_docker.md#using-a-specific-configuration

Handle generation of configuration: CYGNUS_SKIP_CONF_GENERATION. If enabled (i.e. env var set to false) a configuration will be created for container, if disabled (i.e. env var set to true) then your should provide a configuration file.
jmezzera commented 5 years ago

@AlvaroVega thanks for your response. I did not know about that variable you mentioned. However, when I try to run the container with CYGNUS_SKIP_CONF_GENERATION set to true, the agent.conf file gets deleted from the host OS. I'm running the container with docker-compose, as follows:

version: "2.4"

services:
  cygnus:
    image: fiware/cygnus-ngsi
    hostname: cygnus
    container_name: cygnus
    environment:
      - CYGNUS_AGENT_NAME=cygnusagent
    ports:
      - "5050:5050"
      - "5080:5080"
    volumes:
      - ./flume/conf/:/opt/apache-flume/conf/:ro
AlvaroVega commented 5 years ago

Try with: 

version: "2.4"

services:
  cygnus:
    image: fiware/cygnus-ngsi
    hostname: cygnus
    container_name: cygnus
    environment:
      - CYGNUS_AGENT_NAME=cygnusagent
      - CYGNUS_MULTIAGENT=false
      - CYGNUS_SKIP_CONF_GENERATION=true
    ports:
      - "5050:5050"
      - "5080:5080"
   volumes:
      - ./my_custom_agent.conf:/opt/apache-flume/conf/agent.conf
AlvaroVega commented 4 years ago

Shall we close this issue ?

jmezzera commented 4 years ago

Yes, sorry