search

mal-lang / mal-simulator

Apache License 2.0
2 stars 1 forks source link

Reduce the amount of log processing done when log level is reduced #59

Closed kasanari closed 2 months ago

kasanari commented 2 months ago

Results from profiling shows that the simulator spends a fair amount of time formatting strings intended for debug logging, even if the output is ignored.

kasanari commented 2 months ago

Some form of lazy evaluation might be preferable: https://stackoverflow.com/questions/4148790/lazy-logger-message-string-evaluation

kasanari commented 2 months ago

PyLint has opinions on how logger messages should be formatted: https://pylint.readthedocs.io/en/latest/user_guide/messages/warning/logging-format-interpolation.html

andrewbwm commented 2 months ago

PyLint has opinions on how logger messages should be formatted: https://pylint.readthedocs.io/en/latest/user_guide/messages/warning/logging-format-interpolation.html

Yes, the mal-toolbox was updated to follow both of these patterns, to use c type formats and only performs the string processing when logging matches. So, it's something we intend to do, but it isn't the highest priority right now.

mrkickling commented 2 months ago

I believe that logging of the mapping tables and the whole observation is what is taking most time (done in .init()) so that felt worth adding DEBUG checks to.

I created a branch where they are factored out and DEBUG-checked so they should only be calculated in DEBUG mode:

61

@kasanari / @andrewbwm take a look at it if you like, I think it could be merged right away and from the little profiling I did it seems to make the init() function run about 40% faster.

Fixing the logging format for the rest of the code base should be done at another time, I will make an issue for it.