search

certtools / intelmq

IntelMQ is a solution for IT security teams for collecting and processing security feeds using a message queuing protocol.
https://docs.intelmq.org/latest/
GNU Affero General Public License v3.0
976 stars 297 forks source link

Only load the config once on intelmqctl init #2444

Closed DigitalTrustCenter closed 10 months ago

DigitalTrustCenter commented 11 months ago

Every time intelmqctl is started, the runtime yaml config is loaded twice in the init method.

  1. The global settings are loaded. This is done by loading the whole runtime config and taking the global section.
  2. The runtime config is loaded and stored in the _runtime_configuration variable.

This fix changes intelmqctl init so that it only loads (and parses) the runtime config once. The global settings are taken from the config that is already loaded.

This looks like a small optimization, but the current (pure python) YAML parser is quite slow. The IntelMQ manager calls the IntelMQ API. The IntelMQ API calls intelmqctl. Hence on a quite large runtime config, the manager feels sluggish.

In our test, we reduced the time it took for a single API call by 0.4 seconds with this fix.

DigitalTrustCenter commented 10 months ago

It looks like the failed CI checks also appear on other pull requests. Can we ignore the failed check?

DigitalTrustCenter commented 10 months ago

Added a changelog entry. Surprisingly all checks pass now.

aaronkaplan commented 10 months ago

looks good to me. Thx @DigitalTrustCenter !!