StackStorm-Exchange / stackstorm-kafka

Integration pack for Apache Kafka message broker
https://exchange.stackstorm.org/
Apache License 2.0
5 stars 11 forks source link

Add logging #5

Open surfer190 opened 5 years ago

surfer190 commented 5 years ago

To prevent unhelpful error messages like this:

[cent@st2 kafka]$ st2 run kafka.produce topic='test' message='Hello from stackstorm'
.
id: 5cd40d3152364c11e30049b3
status: failed
parameters: 
  message: Hello from stackstorm
  topic: test
result: 
  exit_code: 1
  result: None
  stderr: "No handlers could be found for logger "kafka.conn"
Traceback (most recent call last):
  File "/opt/stackstorm/st2/lib/python2.7/site-packages/python_runner/python_action_wrapper.py", line 334, in <module>
    obj.run()
  File "/opt/stackstorm/st2/lib/python2.7/site-packages/python_runner/python_action_wrapper.py", line 193, in run
    output = action.run(**self._parameters)
  File "/opt/stackstorm/packs/kafka/actions/produce.py", line 42, in run
    result = producer.send_messages(topic, kafka_bytestring(message))
  File "/opt/stackstorm/virtualenvs/kafka/lib/python2.7/site-packages/kafka/producer/simple.py", line 54, in send_messages
    topic, partition, *msg
  File "/opt/stackstorm/virtualenvs/kafka/lib/python2.7/site-packages/kafka/producer/base.py", line 364, in send_messages
    return self._send_messages(topic, partition, *msg)
  File "/opt/stackstorm/virtualenvs/kafka/lib/python2.7/site-packages/kafka/producer/base.py", line 411, in _send_messages
    fail_on_error=self.sync_fail_on_error
  File "/opt/stackstorm/virtualenvs/kafka/lib/python2.7/site-packages/kafka/client.py", line 614, in send_produce_request
    (not fail_on_error or not self._raise_on_response_error(resp))]
  File "/opt/stackstorm/virtualenvs/kafka/lib/python2.7/site-packages/kafka/client.py", line 374, in _raise_on_response_error
    raise resp
kafka.common.FailedPayloadsError
"
  stdout: ''

Could we add a logger to catch and give useful information like that described in this kafka stackoverflow logger not found answer.

It helped quite a bit:

import logging
logging.basicConfig(
    format='%(asctime)s.%(msecs)s:%(name)s:%(thread)d:%(levelname)s:%(process)d:%(message)s',
    level=logging.DEBUG
)
arm4b commented 5 years ago

Contributions to improve error reporting and catch unreadable errors like that are definitely welcome!

If you could workaround that particular case and utilize ST2 logger (https://docs.stackstorm.com/actions.html#logging) or just re-throw more user-friendly error message that would be great.