search

eht16 / python-logstash-async

Python logging handler for sending log events asynchronously to Logstash.
MIT License
186 stars 51 forks source link

Cant't send tags as 1st level of document #29

Closed manthanBhayani closed 6 years ago

manthanBhayani commented 6 years ago

Hi, I am sending logs to ELK and i want to send some extrainfo without extra fields mentions while logging message. I want to send environment name with all messages but not able to send it, is there any central place where I can set.

below is my config file.

`# loggers [loggers] keys = root

[logger_root] name = python-app level = INFO handlers = console,logstash,file propagate = 1 qualname = root

[handlers] keys = console,logstash,file

[handler_console] class = StreamHandler level = NOTSET formatter = console args = (sys.stdout,)

[handler_logstash] class = logstash_async.handler.AsynchronousLogstashHandler level = NOTSET formatter = logstash args = ('%(host)s', %(port)s, '%(database_path)s', '%(transport)s',%(enable)s) transport = logstash_async.transport.UdpTransport host = 10.10.0.19 port = 9563 enable = True database_path =

[handler_file] class : logging.handlers.RotatingFileHandler level : NOTSET formatter: file args = ('%(filename)s','a','%(maxBytes)s','%(backupCount)s',None,0) filename: /var/log/es-streaming-transporter/transporter.log maxBytes: 1024 backupCount: 3

[formatters] keys = console,logstash,file

[formatter_console] format = [%(asctime)s][%(module)s][%(levelname)s] %(message)s

[formatter_file] format = [%(asctime)s][%(module)s][%(levelname)s] %(message)s

[formatter_logstash] class = logstash_async.formatter.LogstashFormatter

format = python-logstash style = True`

eht16 commented 6 years ago

It's already possible what you want. See http://python-logstash-async.readthedocs.io/en/latest/config.html#options-for-configuring-the-log-formatter and in detail you need to set the optons extra and extra_prefix.

extra is a dictionary of additional message fields to be submitted to Logstash, e.g. your environment. extra_prefix is the prefix or namespace into which the extra fields should be submitted. If you set this to None or the empty string, the additional fields will be submitted to the top level of the event, this is probably what you want.

Unfortunately, you cannot change these options with the ini configuration file due to the limited options available to the formatter section. This is a limitation of Python's logging framework. I'd suggest to migrate your logging configuration to the dict configuration which is way more flexible. Examples can be found in the docs: http://python-logstash-async.readthedocs.io/en/latest/usage.html.

manthanBhayani commented 6 years ago

@eht16 Thank you. I have created dict and issue got resolved.