Error in installation

[DGRamya]

step : python twitter_data_producer.py

I tried installing the above step, but I get the following error

(env) Jerrins-MBP:twitter_feed_microservice DG$ python twitter_data_producer.py Traceback (most recent call last): File "twitter_data_producer.py", line 10, in access_token = configParser.get('twitter-token-config', 'access_token') File "/Users/DG/Documents/SGA_AUTOBOTS_TEST/autobots/backend/twitter_feed_microservice/env/lib/python2.7/site-packages/backports/configparser/init.py", line 792, in get d = self._unify_values(section, vars) File "/Users/DG/Documents/SGA_AUTOBOTS_TEST/autobots/backend/twitter_feed_microservice/env/lib/python2.7/site-packages/backports/configparser/init.py", line 1162, in _unify_values raise NoSectionError(section) backports.configparser.NoSectionError: No section: 'twitter-token-config'

Also, I tried installing python3, but the issue is not resolved. Unable to complete installation

[gowtham06]

You need to add a config file containing the api credentials. We added the config file directly into the jet-stream cloud. However if we do it manually, we need to add the file. Since,it contains security credentials, we have not added that in our github repo. Again , thanks for pointing out, I will update it in the installation wiki. I think @naveenkumarmarri will be able to give you the config file.

[naveenkumarmarri]

@DGRamya I've shared the config.txt file to your email. Can you please retry?

[gowtham06]

@DGRamya I have also sent you the config.txt file to your gmail. Sorry for the confusion. Hope this helps.

[soutri]

Hello

I am assigned to review your work and am facing the same issue, could you send me the required files as well

Thanks

[gowtham06]

@soutri sent the file to your iu mail. Please follow the instructions to run. Hope, this helps.

Thanks, Gowtham

[DGRamya]

@gowtham06 @naveenkumarmarri

I did the steps as mentioned in the mail, I am getting the below error

Traceback (most recent call last): File "twitter_data_producer.py", line 31, in kafka = KafkaClient("localhost:9092") File "/Users/DG/Documents/SGA_AUTOBOTS_TEST/autobots/backend/twitter_feed_microservice/env/lib/python2.7/site-packages/kafka/init.py", line 45, in init super(KafkaClient, self).init(*args, **kwargs) File "/Users/DG/Documents/SGA_AUTOBOTS_TEST/autobots/backend/twitter_feed_microservice/env/lib/python2.7/site-packages/kafka/client.py", line 57, in init self.load_metadata_for_topics() # bootstrap with all metadata File "/Users/DG/Documents/SGA_AUTOBOTS_TEST/autobots/backend/twitter_feed_microservice/env/lib/python2.7/site-packages/kafka/client.py", line 525, in load_metadata_for_topics resp = self.send_metadata_request(topics) File "/Users/DG/Documents/SGA_AUTOBOTS_TEST/autobots/backend/twitter_feed_microservice/env/lib/python2.7/site-packages/kafka/client.py", line 590, in send_metadata_request return self._send_broker_unaware_request(payloads, encoder, decoder) File "/Users/DG/Documents/SGA_AUTOBOTS_TEST/autobots/backend/twitter_feed_microservice/env/lib/python2.7/site-packages/kafka/client.py", line 177, in _send_broker_unaware_request raise KafkaUnavailableError('All servers failed to process request: %s' % hosts) kafka.errors.KafkaUnavailableError: KafkaUnavailableError: All servers failed to process request: [('localhost', 9092, 0)] (env) Jerrins-MBP:twitter_feed_microservice DG$

[DGRamya]

Or could you please the mail the credentials of jetstream CI/CD server, @gowtham06 @naveenkumarmarri

[shivteja14]

please share the config.txt file @naveenkumarmarri

[DGRamya]

@gowtham06 @naveenkumarmarri . I tried restarting the node server and then started the kafka producer as " python twitter_data_producer.py" . Now, Node service is running but I am getting the below error for KAFKA.

Note: config.txt file is placed in the folder and access tokens are accessible

*(env) Jerrins-MBP:twitter_feed_microservice DG$ python twitter_data_producer.py Starting the kafka producer Traceback (most recent call last): File "twitter_data_producer.py", line 31, in kafka = KafkaClient("localhost:9092") File "/Users/DG/Documents/SGA_AUTOBOTS_TEST/autobots/backend/twitter_feed_microservice/env/lib/python2.7/site-packages/kafka/init.py", line 45, in init super(KafkaClient, self).init(args, kwargs) File "/Users/DG/Documents/SGA_AUTOBOTS_TEST/autobots/backend/twitter_feed_microservice/env/lib/python2.7/site-packages/kafka/client.py", line 57, in init self.load_metadata_for_topics() # bootstrap with all metadata File "/Users/DG/Documents/SGA_AUTOBOTS_TEST/autobots/backend/twitter_feed_microservice/env/lib/python2.7/site-packages/kafka/client.py", line 525, in load_metadata_for_topics resp = self.send_metadata_request(topics) File "/Users/DG/Documents/SGA_AUTOBOTS_TEST/autobots/backend/twitter_feed_microservice/env/lib/python2.7/site-packages/kafka/client.py", line 590, in send_metadata_request return self._send_broker_unaware_request(payloads, encoder, decoder) File "/Users/DG/Documents/SGA_AUTOBOTS_TEST/autobots/backend/twitter_feed_microservice/env/lib/python2.7/site-packages/kafka/client.py", line 177, in _send_broker_unaware_request raise KafkaUnavailableError('All servers failed to process request: %s' % hosts) kafka.errors.KafkaUnavailableError: KafkaUnavailableError: All servers failed to process request: [('localhost', 9092, 0)]

[naveenkumarmarri]

Can try adding 127.0.0.1 broker in your in /etc/hosts file?

On Tue, Sep 25, 2018, 5:38 PM Ramyashree DG notifications@github.com wrote:

I tried restarting the node server and then started the kafka producer as " python twitter_data_producer.py" . Now, Node service is running but I am getting the below error for KAFKA.

Note: config.txt file is placed in the folder and access tokens are accessible

(env) Jerrins-MBP:twitter_feed_microservice DG$ python twitter_data_producer.py Starting the kafka producer Traceback (most recent call last): File "twitter_data_producer.py", line 31, in kafka = KafkaClient("localhost:9092") File "/Users/DG/Documents/SGA_AUTOBOTS_TEST/autobots/backend/twitter_feed_microservice/env/lib/python2.7/site-packages/kafka/ init.py", line 45, in init super(KafkaClient, self).init(*args, *kwargs) File "/Users/DG/Documents/SGA_AUTOBOTS_TEST/autobots/backend/twitter_feed_microservice/env/lib/python2.7/site-packages/kafka/client.py", line 57, in init* self.load_metadata_for_topics() # bootstrap with all metadata File "/Users/DG/Documents/SGA_AUTOBOTS_TEST/autobots/backend/twitter_feed_microservice/env/lib/python2.7/site-packages/kafka/client.py", line 525, in load_metadata_for_topics resp = self.send_metadata_request(topics) File "/Users/DG/Documents/SGA_AUTOBOTS_TEST/autobots/backend/twitter_feed_microservice/env/lib/python2.7/site-packages/kafka/client.py", line 590, in send_metadata_request return self._send_broker_unaware_request(payloads, encoder, decoder) File "/Users/DG/Documents/SGA_AUTOBOTS_TEST/autobots/backend/twitter_feed_microservice/env/lib/python2.7/site-packages/kafka/client.py", line 177, in _send_broker_unaware_request raise KafkaUnavailableError('All servers failed to process request: %s' % hosts) kafka.errors.KafkaUnavailableError: KafkaUnavailableError: All servers failed to process request: [('localhost', 9092, 0)]

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/airavata-courses/autobots/issues/34#issuecomment-424510535, or mute the thread https://github.com/notifications/unsubscribe-auth/ACXnJILtiyMziCbVmpEBlS9tYww6jbMoks5ueqJRgaJpZM4W36af .

[gowtham06]

please share the config.txt file @naveenkumarmarri Can you share your email id please?

[shivteja14]

please share the config.txt file @naveenkumarmarri Can you share your email id please?

shayyag@iu.edu

[DGRamya]

@gowtham06 @naveenkumarmarri I added the host file entry, but not successful yet. I also re-started the node server.

ERROR: (env) Jerrins-MBP:twitter_feed_microservice DG$ python twitter_data_producer.py Starting the kafka producer Traceback (most recent call last): File "twitter_data_producer.py", line 31, in kafka = KafkaClient("localhost:9092") File "/Users/DG/Documents/SGA_AUTOBOTS_TEST/autobots/backend/twitter_feed_microservice/env/lib/python2.7/site-packages/kafka/init.py", line 45, in init super(KafkaClient, self).init(*args, **kwargs) File "/Users/DG/Documents/SGA_AUTOBOTS_TEST/autobots/backend/twitter_feed_microservice/env/lib/python2.7/site-packages/kafka/client.py", line 57, in init self.load_metadata_for_topics() # bootstrap with all metadata File "/Users/DG/Documents/SGA_AUTOBOTS_TEST/autobots/backend/twitter_feed_microservice/env/lib/python2.7/site-packages/kafka/client.py", line 525, in load_metadata_for_topics resp = self.send_metadata_request(topics) File "/Users/DG/Documents/SGA_AUTOBOTS_TEST/autobots/backend/twitter_feed_microservice/env/lib/python2.7/site-packages/kafka/client.py", line 590, in send_metadata_request return self._send_broker_unaware_request(payloads, encoder, decoder) File "/Users/DG/Documents/SGA_AUTOBOTS_TEST/autobots/backend/twitter_feed_microservice/env/lib/python2.7/site-packages/kafka/client.py", line 177, in _send_broker_unaware_request raise KafkaUnavailableError('All servers failed to process request: %s' % hosts) kafka.errors.KafkaUnavailableError: KafkaUnavailableError: All servers failed to process request: [('localhost', 9092, 0)]

[gowtham06]

@DGRamya looking into the issue.

[gowtham06]

@DGRamya Are you running it on windows machine or a linux machine. ?

[naveenkumarmarri]

@DGRamya I've updated the installation guide which was missing part about creating topic

Please let us know if you have issue after creating topic and changing /etc/hosts file

[shivteja14]

Hello, @naveenkumarmarri I am also facing the same issue. "kafka.errors.KafkaUnavailableError: KafkaUnavailableError: All servers failed to process request: [('localhost', 9092, 0)]"

[naveenkumarmarri]

@shivteja14 did you try the step of modifying /etc/hosts and creating topicas mentioned in https://github.com/airavata-courses/autobots/wiki/Installation#creating-topic ?

[DGRamya]

@DGRamya I've updated the installation guide which was missing part about creating topic

Please let us know if you have issue after creating topic and changing /etc/hosts file

@DGRamya Are you running it on windows machine or a linux machine. ?

@gowtham06 I am running it on MAC.

[DGRamya]

@DGRamya I've updated the installation guide which was missing part about creating topic

Please let us know if you have issue after creating topic and changing /etc/hosts file

@naveenkumarmarri I followed the updated steps but dint succeed.

(env) Jerrins-MBP:twitter_feed_microservice DG$ python twitter_data_producer.py Starting the kafka producer Traceback (most recent call last): File "twitter_data_producer.py", line 31, in kafka = KafkaClient("localhost:9092") File "/Users/DG/Documents/SGA_AUTOBOTS_TEST/autobots/backend/twitter_feed_microservice/env/lib/python2.7/site-packages/kafka/init.py", line 45, in init super(KafkaClient, self).init(*args, **kwargs) File "/Users/DG/Documents/SGA_AUTOBOTS_TEST/autobots/backend/twitter_feed_microservice/env/lib/python2.7/site-packages/kafka/client.py", line 57, in init self.load_metadata_for_topics() # bootstrap with all metadata File "/Users/DG/Documents/SGA_AUTOBOTS_TEST/autobots/backend/twitter_feed_microservice/env/lib/python2.7/site-packages/kafka/client.py", line 525, in load_metadata_for_topics resp = self.send_metadata_request(topics) File "/Users/DG/Documents/SGA_AUTOBOTS_TEST/autobots/backend/twitter_feed_microservice/env/lib/python2.7/site-packages/kafka/client.py", line 590, in send_metadata_request return self._send_broker_unaware_request(payloads, encoder, decoder) File "/Users/DG/Documents/SGA_AUTOBOTS_TEST/autobots/backend/twitter_feed_microservice/env/lib/python2.7/site-packages/kafka/client.py", line 177, in _send_broker_unaware_request raise KafkaUnavailableError('All servers failed to process request: %s' % hosts) kafka.errors.KafkaUnavailableError: KafkaUnavailableError: All servers failed to process request: [('localhost', 9092, 0)]

[naveenkumarmarri]

@DGRamya can you share the output if you run the following command netstat -lntp | grep 9092

Just to make sure your kafka is running

[DGRamya]

@naveenkumarmarri I tried to follow the steps from beginning , now it is giving me error in creating topic

Jerrins-MacBook-Pro:autobots DG$ docker-compose exec broker kafka-topics --create --zookeeper zookeeper:2181 --replication-factor 1 --partitions 1 --topic trump Error while executing topic command : Topic 'trump' already exists. [2018-09-26 01:59:45,677] ERROR org.apache.kafka.common.errors.TopicExistsException: Topic 'trump' already exists.

[DGRamya]

@DGRamya can you share the output if you run the following command netstat -lntp | grep 9092

Just to make sure your kafka is running

Kafka server dint start right, it gave error for the starting command

[naveenkumarmarri]

@DGRamya That is because there is already an existing topic named trump, IMO that should be fine

[naveenkumarmarri]

@DGRamya can you share the output if you run the following command netstat -lntp | grep 9092 Just to make sure your kafka is running

Kafka server dint start right, it gave error for the starting command

So, by running this command, you didn't get any output?

Ideally, if the kafka service is up, you should see the following output

naveen@tourist:~/iu_backup/git/crest/object_detection/gt_training$ netstat -lntp | grep 9092
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
tcp6       0      0 :::9092                 :::*                    LISTEN      -               

[DGRamya]

I had to repeat the installation procedure again, kindly request you to update the document properly.

(env) Jerrins-MacBook-Pro:twitter_feed_microservice DG$ python twitter_data_producer.py Starting the kafka producer

works fine now

[naveenkumarmarri]

Did you try some other work around to get it working?

[shivteja14]

@shivteja14 did you try the step of modifying /etc/hosts as mentioned in https://github.com/airavata-courses/autobots/wiki/Installation#creating-topic ?

Yes,I tried. But its not working. Let me run the commands in the last quotes and try again.