Closed ferrarimarco closed 6 years ago
raw-data-analyzer does not start due to an error regarding brokers not being available
vagrant --version
docker --version
scissorproject-raw-data-analyzer | Waiting for kafka to be ready on port 9092 scissorproject-raw-data-analyzer | Ncat: Version 7.70 ( https://nmap.org/ncat ) scissorproject-raw-data-analyzer | NCAT DEBUG: Using system default trusted CA certificates and those in /usr/share/ncat/ca-bundle.crt. scissorproject-raw-data-analyzer | libnsock nsock_iod_new2(): nsock_iod_new (IOD #1) scissorproject-raw-data-analyzer | libnsock nsock_connect_tcp(): TCP connection requested to 172.19.0.8:9092 (IOD #1) EID 8 scissorproject-raw-data-analyzer | libnsock nsock_trace_handler_callback(): Callback: CONNECT ERROR [Connection refused (111)] for EID 8 [172.19.0.8:9092] scissorproject-raw-data-analyzer | Ncat: Connection refused. scissorproject-raw-data-analyzer | Wating 10 secs... scissorproject-raw-data-analyzer | Ncat: Version 7.70 ( https://nmap.org/ncat ) scissorproject-raw-data-analyzer | NCAT DEBUG: Using system default trusted CA certificates and those in /usr/share/ncat/ca-bundle.crt. scissorproject-raw-data-analyzer | libnsock nsock_iod_new2(): nsock_iod_new (IOD #1) scissorproject-raw-data-analyzer | libnsock nsock_connect_tcp(): TCP connection requested to 172.19.0.8:9092 (IOD #1) EID 8 scissorproject-raw-data-analyzer | libnsock nsock_trace_handler_callback(): Callback: CONNECT ERROR [Connection refused (111)] for EID 8 [172.19.0.8:9092] scissorproject-raw-data-analyzer | Ncat: Connection refused. scissorproject-raw-data-analyzer | Wating 10 secs... scissorproject-raw-data-analyzer | Ncat: Version 7.70 ( https://nmap.org/ncat ) scissorproject-raw-data-analyzer | NCAT DEBUG: Using system default trusted CA certificates and those in /usr/share/ncat/ca-bundle.crt. scissorproject-raw-data-analyzer | libnsock nsock_iod_new2(): nsock_iod_new (IOD #1) scissorproject-raw-data-analyzer | libnsock nsock_connect_tcp(): TCP connection requested to 172.19.0.8:9092 (IOD #1) EID 8 scissorproject-raw-data-analyzer | libnsock nsock_trace_handler_callback(): Callback: CONNECT SUCCESS for EID 8 [172.19.0.8:9092] scissorproject-raw-data-analyzer | Ncat: Connected to 172.19.0.8:9092. scissorproject-raw-data-analyzer | libnsock nsock_iod_new2(): nsock_iod_new (IOD #2) scissorproject-raw-data-analyzer | Ncat: 0 bytes sent, 0 bytes received in 0.28 seconds. scissorproject-raw-data-analyzer | libnsock nsock_iod_delete(): nsock_iod_delete (IOD #1) scissorproject-raw-data-analyzer | libnsock nsock_iod_delete(): nsock_iod_delete (IOD #2) scissorproject-raw-data-analyzer | Using configuration file : /etc/envMonAnalyzer.conf scissorproject-raw-data-analyzer | Converter type : EnvMonAnalyzer scissorproject-raw-data-analyzer | Connecting to kafka 185.19.28.53:9092 scissorproject-raw-data-analyzer | Reading from topics ['env'] scissorproject-raw-data-analyzer | Writing to topic IDMEF scissorproject-raw-data-analyzer | Traceback (most recent call last): scissorproject-raw-data-analyzer | File "/usr/bin/converter", line 9, in <module> scissorproject-raw-data-analyzer | load_entry_point('kafka-idmef-converter==0.2.4', 'console_scripts', 'converter')() scissorproject-raw-data-analyzer | File "/usr/lib/python2.7/site-packages/converter/__main__.py", line 32, in main scissorproject-raw-data-analyzer | analyzer = AnalyzerFactory.getAnalyzer(analyzer_type=conf["converter_type"], params={'conf':conf}) scissorproject-raw-data-analyzer | File "/usr/lib/python2.7/site-packages/converter/analyzer_factory.py", line 11, in getAnalyzer scissorproject-raw-data-analyzer | return analyzerClass(**params) scissorproject-raw-data-analyzer | File "/usr/lib/python2.7/site-packages/converter/analyzers/EnvMonAnalyzer.py", line 40, in __init__ scissorproject-raw-data-analyzer | super(EnvMonAnalyzer, self).__init__(conf) scissorproject-raw-data-analyzer | File "/usr/lib/python2.7/site-packages/converter/converter.py", line 143, in __init__ scissorproject-raw-data-analyzer | self.kafka_consumer = self.build_kafka_consumer() scissorproject-raw-data-analyzer | File "/usr/lib/python2.7/site-packages/converter/converter.py", line 148, in build_kafka_consumer scissorproject-raw-data-analyzer | kafka_consumer = kafka.KafkaConsumer(bootstrap_servers="%s:%s" % (self.kafka_server, self.kafka_port)) scissorproject-raw-data-analyzer | File "/usr/lib/python2.7/site-packages/kafka/consumer/group.py", line 324, in __init__ scissorproject-raw-data-analyzer | self._client = KafkaClient(metrics=self._metrics, **self.config) scissorproject-raw-data-analyzer | File "/usr/lib/python2.7/site-packages/kafka/client_async.py", line 221, in __init__ scissorproject-raw-data-analyzer | self.config['api_version'] = self.check_version(timeout=check_timeout) scissorproject-raw-data-analyzer | File "/usr/lib/python2.7/site-packages/kafka/client_async.py", line 826, in check_version scissorproject-raw-data-analyzer | raise Errors.NoBrokersAvailable() scissorproject-raw-data-analyzer | kafka.errors.NoBrokersAvailable: NoBrokersAvailable
test/test-docker-images.sh --skip-pull --docker-context-path=docker
Expected behavior:
✔ Processes converter should exist ✔ Processes converter entries.length should eq 1
Actual behavior: [What actually happens]
× Processes converter should exist expected Processes converter to exist × Processes converter entries.length should eq 1 expected: 1 got: 0 (compared using ==)
Reproduces how often: not always. I suspect it's due to a race condition between containers. What are the external dependencies of this process?
Prerequisites
Description
raw-data-analyzer does not start due to an error regarding brokers not being available
Host Environment
vagrant --version
): 2.0.3docker --version
): 18.02-ceOutput
Steps to Reproduce
test/test-docker-images.sh --skip-pull --docker-context-path=docker
Expected behavior:
Actual behavior: [What actually happens]
Reproduces how often: not always. I suspect it's due to a race condition between containers. What are the external dependencies of this process?