KaraMelih
commented
2 years ago This was the reply we got back from Scimma
This is usually caused by network admins blocking the port needed to communicate with Kafka. Using a VPN will defeat that by tunnelling your traffic out (on some port that isn’t blocked), so some otehr network which (hopefully) doesn’t have such blocking. Using tools like nmap, one can check whether port 9092 on kafka.scimma.org is reachable. It can sometimes also be diagnosed with a test like echo test | nc -w 10 kafka.scimma.org 9092; echo $? ; if that quickly prints P0 the port is not blocked, while if it takes the full ten seocnds to time out and prints only 0, the port is likely blocked. However, there’s some complexity with this; we’re seeing some cases (like at MSU) where connections hang and get cut off, even though they are initially established. We’re still trying to prove where the sourec of the problem is, although we continue to suspect the network. If you find that connections on port 9092 are blocked, you can try to get around it by working form another network (via a VPN, from a machine located on a science DMZ, etc.), or you can try convincing the IT department responsible to remove the block.
justinvasel
I am getting this Kafka error when trying to do subscribe to snews_pt or run the hop connection with snewpdag:
Traceback (most recent call last): File "/home/marta/miniconda3/envs/snewpy_env/bin/snews_pt", line 8, in
sys.exit(main())
File "/home/marta/miniconda3/envs/snewpy_env/lib/python3.8/site-packages/click/core.py", line 1130, in call
return self.main(args, kwargs)
File "/home/marta/miniconda3/envs/snewpy_env/lib/python3.8/site-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
File "/home/marta/miniconda3/envs/snewpy_env/lib/python3.8/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/marta/miniconda3/envs/snewpy_env/lib/python3.8/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, ctx.params)
File "/home/marta/miniconda3/envs/snewpy_env/lib/python3.8/site-packages/click/core.py", line 760, in invoke
return __callback(args, *kwargs)
File "/home/marta/miniconda3/envs/snewpy_env/lib/python3.8/site-packages/click/decorators.py", line 26, in new_func
return f(get_current_context(), args, **kwargs)
File "/home/marta/miniconda3/envs/snewpy_env/lib/python3.8/site-packages/snews_pt/main.py", line 79, in subscribe
sub.subscribe()
File "/home/marta/miniconda3/envs/snewpy_env/lib/python3.8/site-packages/snews_pt/snews_sub.py", line 93, in subscribe with stream.open(self.alert_topic, "r") as s: File "/home/marta/miniconda3/envs/snewpy_env/lib/python3.8/site-packages/hop/io.py", line 119, in open return Consumer( File "/home/marta/miniconda3/envs/snewpy_env/lib/python3.8/site-packages/hop/io.py", line 299, in init self._consumer.subscribe(topics) File "/home/marta/miniconda3/envs/snewpy_env/lib/python3.8/site-packages/adc/consumer.py", line 41, in subscribe topic_meta = self.describe_topic(topic, timeout) File "/home/marta/miniconda3/envs/snewpy_env/lib/python3.8/site-packages/adc/consumer.py", line 63, in describe_topic cluster_meta = self._consumer.list_topics(timeout=timeout.total_seconds()) cimpl.KafkaException: KafkaError{code=_TRANSPORT,val=-195,str="Failed to get metadata: Local: Broker transport failure"}
If anyone has a hint on this issue and can help, I would appreciate it.