[log_format] Add an option to change the log format

[goldyfruit]

Description

Having an option to change the log format could be useful when logs are shipped into an aggregator such as Elasticsearch.

The current format is very hard to parse for example with Logstash or Filebeat

How to test

Edit mycroft.conf and add log_format option.

{
  "max_allowed_core_version": 21.2,
  "log_format": "{asctime} {levelname} {process} {name} {message}"
}

Before:

2021-10-27 01:51:20.169 | DEBUG    |     1 | mycroft.messagebus.service.event_handler:on_message:38 | {"type": "mycroft-date-time.mycroftai:TimeSkillupdate_display", "data": {}, "context": {}}

After:

2021-10-27 01:51:20.169 DEBUG 1 mycroft.messagebus.service.event_handler:on_message:38 {"type": "mycroft-date-time.mycroftai:TimeSkillupdate_display", "data": {}, "context": {}}

Contributor license agreement signed?

CLA [ YES ]

[devops-mycroft]

Voight Kampff Integration Test Succeeded (Results)

[codecov-commenter]

Codecov Report

Merging #3016 (6fc8979) into dev (3495acf) will increase coverage by 0.00%. The diff coverage is 33.33%.

@@           Coverage Diff           @@
##              dev    #3016   +/-   ##
=======================================
  Coverage   53.04%   53.04%           
=======================================
  Files         123      123           
  Lines       11167    11170    +3     
=======================================
+ Hits         5923     5925    +2     
- Misses       5244     5245    +1     

Impacted Files	Coverage Δ
mycroft/util/log.py	91.22% <33.33%> (-3.22%)	:arrow_down:
mycroft/client/speech/listener.py	48.39% <0.00%> (+0.35%)	:arrow_up:

---

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 3495acf...6fc8979. Read the comment docs.

[goldyfruit]

Yeah love it - goodbye PID!

I had one minor issue where the stream handler must be initialized before we try to get the config.

What was the issue you got? I'm not able to see any in the logs :/

Log snippet of bus.logs:

2021-10-27 16:05:20.736 INFO 1 __main__:main:50 Starting message bus service...
2021-10-27 16:05:20.738 INFO 1 mycroft.messagebus.load_config:load_message_bus_config:33 Loading message bus configs
2021-10-27 16:05:20.740 DEBUG 1 asyncio Using selector: EpollSelector
2021-10-27 16:05:20.755 INFO 1 __main__:on_ready:37 Message bus service started!
2021-10-27 16:05:23.899 INFO 1 tornado.access 101 GET /core (127.0.0.1) 2.85ms
2021-10-27 16:05:23.907 INFO 1 tornado.access 101 GET /core (127.0.0.1) 4.40ms
2021-10-27 16:05:23.909 INFO 1 tornado.access 101 GET /core (127.0.0.1) 5.34ms
2021-10-27 16:05:23.911 INFO 1 tornado.access 101 GET /core (127.0.0.1) 6.66ms
2021-10-27 16:05:23.914 INFO 1 tornado.access 101 GET /core (127.0.0.1) 8.22ms
2021-10-27 16:05:23.928 DEBUG 1 mycroft.messagebus.service.event_handler:on_message:38 {"type": "enclosure.reset", "data": {}, "context": {"destination": ["enclosure"]}}
2021-10-27 16:05:24.084 DEBUG 1 mycroft.messagebus.service.event_handler:on_message:38 {"type": "mycroft-reminder.mycroftai:reminder", "data": {}, "context": {}}
2021-10-27 16:05:30.089 DEBUG 1 mycroft.messagebus.service.event_handler:on_message:38 {"type": "mycroft-date-time.mycroftai:TimeSkillupdate_display", "data": {}, "context": {}}

[goldyfruit]

I was not able to get any error/warning but I think I understand your issue.

Having mycroft.configuration.Configuration() class been called before configure the Formatter could be an issue because mycroft.configuration.Configuration() class imports LOG class as well.

So the second commit set by default the Formatter and if log_format exists in mycroft.conf then the Formatter is updated.