ENH: Added new expert plugin - extra_tags

[manuel-subredu]

• added a new plugin that allows manipulation of extra_tags. Main use case is to allow usage of same pipeline for both dev and prod setups but be able to differentiate between messages and select a different path for your dev stage (ex: route dev messages to local directory for examination rather then production database)

Description

ExtraTags expert plugin aims to ease the testing of an existing pipeline and allow easy routing of the messages in development phase. Main use case:

1. use extra_tags plugin right after a parser (or input) and mark the messages as beloning to development stage {'stage': 'dev'}
2. insert a prod-dev-routing decision right before final output and with filter plugin route non-production messages to various backends to inspection

Testing

(.venv) ➜  intelmq git:(develop) ✗ pytest intelmq/tests/bots/experts/extra_tags -s
=========================================================================================================== test session starts ============================================================================================================
platform darwin -- Python 3.12.0, pytest-8.0.2, pluggy-1.4.0
rootdir: ws/extra-tags/intelmq
configfile: pytest.ini
plugins: cov-4.1.0, requests-mock-1.11.0
collected 5 items

intelmq/tests/bots/experts/extra_tags/test_expert.py ...True
..

---------- coverage: platform darwin, python 3.12.0-final-0 ----------
Coverage XML written to file coverage.xml

============================================================================================================ 5 passed in 2.01s =============================================================================================================
(.venv) ➜  intelmq git:(develop)

[kamil-certat]

Hey, thanks for the commitment! I've looked through your code, and I think it's somehow similar to the sieve bot. Have you tried it before? I feel like it may be very near to accomplish what you want, although I'm not sure if it supports manipulating a dict inside a field.

May I ask you to have a look at the sieve bot, and see if it's either already solve the problem, or would be relatively easily possible to extend it? I think it may be nice to extend it as it has much more features to manipulate an event :)

[manuel-subredu]

Mea culpa ! I thought that Sieve is somehow related to mail area and didn't looked into the specifics. You are right, sieve expert includes the functionality I actually needed. I think we can drop this pull request. Sorry for the noise.

[kamil-certat]

No problem!