search

scissor-project / open-scissor

OpenSCISSOR provisioning and orchestration
Apache License 2.0
2 stars 0 forks source link

Error while starting raw-data-analyzer #76

Closed ferrarimarco closed 6 years ago

ferrarimarco commented 6 years ago

Prerequisites

Description

raw-data-analyzer does not start due to an error regarding brokers not being available

Host Environment

Output

scissorproject-raw-data-analyzer | Waiting for kafka to be ready on port 9092
scissorproject-raw-data-analyzer | Ncat: Version 7.70 ( https://nmap.org/ncat )
scissorproject-raw-data-analyzer | NCAT DEBUG: Using system default trusted CA certificates and those in /usr/share/ncat/ca-bundle.crt.
scissorproject-raw-data-analyzer | libnsock nsock_iod_new2(): nsock_iod_new (IOD #1)
scissorproject-raw-data-analyzer | libnsock nsock_connect_tcp(): TCP connection requested to 172.19.0.8:9092 (IOD #1) EID 8
scissorproject-raw-data-analyzer | libnsock nsock_trace_handler_callback(): Callback: CONNECT ERROR [Connection refused (111)] for EID 8 [172.19.0.8:9092]
scissorproject-raw-data-analyzer | Ncat: Connection refused.
scissorproject-raw-data-analyzer | Wating 10 secs...
scissorproject-raw-data-analyzer | Ncat: Version 7.70 ( https://nmap.org/ncat )
scissorproject-raw-data-analyzer | NCAT DEBUG: Using system default trusted CA certificates and those in /usr/share/ncat/ca-bundle.crt.
scissorproject-raw-data-analyzer | libnsock nsock_iod_new2(): nsock_iod_new (IOD #1)
scissorproject-raw-data-analyzer | libnsock nsock_connect_tcp(): TCP connection requested to 172.19.0.8:9092 (IOD #1) EID 8
scissorproject-raw-data-analyzer | libnsock nsock_trace_handler_callback(): Callback: CONNECT ERROR [Connection refused (111)] for EID 8 [172.19.0.8:9092]
scissorproject-raw-data-analyzer | Ncat: Connection refused.
scissorproject-raw-data-analyzer | Wating 10 secs...
scissorproject-raw-data-analyzer | Ncat: Version 7.70 ( https://nmap.org/ncat )
scissorproject-raw-data-analyzer | NCAT DEBUG: Using system default trusted CA certificates and those in /usr/share/ncat/ca-bundle.crt.
scissorproject-raw-data-analyzer | libnsock nsock_iod_new2(): nsock_iod_new (IOD #1)
scissorproject-raw-data-analyzer | libnsock nsock_connect_tcp(): TCP connection requested to 172.19.0.8:9092 (IOD #1) EID 8
scissorproject-raw-data-analyzer | libnsock nsock_trace_handler_callback(): Callback: CONNECT SUCCESS for EID 8 [172.19.0.8:9092]
scissorproject-raw-data-analyzer | Ncat: Connected to 172.19.0.8:9092.
scissorproject-raw-data-analyzer | libnsock nsock_iod_new2(): nsock_iod_new (IOD #2)
scissorproject-raw-data-analyzer | Ncat: 0 bytes sent, 0 bytes received in 0.28 seconds.
scissorproject-raw-data-analyzer | libnsock nsock_iod_delete(): nsock_iod_delete (IOD #1)
scissorproject-raw-data-analyzer | libnsock nsock_iod_delete(): nsock_iod_delete (IOD #2)
scissorproject-raw-data-analyzer | Using configuration file : /etc/envMonAnalyzer.conf
scissorproject-raw-data-analyzer | Converter type : EnvMonAnalyzer
scissorproject-raw-data-analyzer | Connecting to kafka 185.19.28.53:9092
scissorproject-raw-data-analyzer | Reading from topics ['env']
scissorproject-raw-data-analyzer | Writing to topic IDMEF
scissorproject-raw-data-analyzer | Traceback (most recent call last):
scissorproject-raw-data-analyzer |   File "/usr/bin/converter", line 9, in <module>
scissorproject-raw-data-analyzer |     load_entry_point('kafka-idmef-converter==0.2.4', 'console_scripts', 'converter')()
scissorproject-raw-data-analyzer |   File "/usr/lib/python2.7/site-packages/converter/__main__.py", line 32, in main
scissorproject-raw-data-analyzer |     analyzer = AnalyzerFactory.getAnalyzer(analyzer_type=conf["converter_type"], params={'conf':conf})
scissorproject-raw-data-analyzer |   File "/usr/lib/python2.7/site-packages/converter/analyzer_factory.py", line 11, in getAnalyzer
scissorproject-raw-data-analyzer |     return analyzerClass(**params)
scissorproject-raw-data-analyzer |   File "/usr/lib/python2.7/site-packages/converter/analyzers/EnvMonAnalyzer.py", line 40, in __init__
scissorproject-raw-data-analyzer |     super(EnvMonAnalyzer, self).__init__(conf)
scissorproject-raw-data-analyzer |   File "/usr/lib/python2.7/site-packages/converter/converter.py", line 143, in __init__
scissorproject-raw-data-analyzer |     self.kafka_consumer = self.build_kafka_consumer()
scissorproject-raw-data-analyzer |   File "/usr/lib/python2.7/site-packages/converter/converter.py", line 148, in build_kafka_consumer
scissorproject-raw-data-analyzer |     kafka_consumer = kafka.KafkaConsumer(bootstrap_servers="%s:%s" % (self.kafka_server, self.kafka_port))
scissorproject-raw-data-analyzer |   File "/usr/lib/python2.7/site-packages/kafka/consumer/group.py", line 324, in __init__
scissorproject-raw-data-analyzer |     self._client = KafkaClient(metrics=self._metrics, **self.config)
scissorproject-raw-data-analyzer |   File "/usr/lib/python2.7/site-packages/kafka/client_async.py", line 221, in __init__
scissorproject-raw-data-analyzer |     self.config['api_version'] = self.check_version(timeout=check_timeout)
scissorproject-raw-data-analyzer |   File "/usr/lib/python2.7/site-packages/kafka/client_async.py", line 826, in check_version
scissorproject-raw-data-analyzer |     raise Errors.NoBrokersAvailable()
scissorproject-raw-data-analyzer | kafka.errors.NoBrokersAvailable: NoBrokersAvailable

Steps to Reproduce

  1. test/test-docker-images.sh --skip-pull --docker-context-path=docker

Expected behavior:

     ✔  Processes converter should exist
     ✔  Processes converter entries.length should eq 1

Actual behavior: [What actually happens]

     ×  Processes converter should exist
     expected Processes converter to exist
     ×  Processes converter entries.length should eq 1

     expected: 1
          got: 0

     (compared using ==)

Reproduces how often: not always. I suspect it's due to a race condition between containers. What are the external dependencies of this process?