Rewrite the dispatcher module

[kidanger]

This PR is relatively large but simplifies a lot the code. The class Dispatcher is now used directly by the core module. Core exposes the endpoint get_demorunners_stats for the CP2. Typing annotations are added and most try/except blocks are removed. Error management is simplified using result.

Unit tests are cleaned-up, more exhaustive and reimplemented using pytest

Removed endpoints:

• /api/dispatcher/index, default, ping, shutdown
• /api/dispatcher/refresh_demorunners
• /api/dispatcher/set_policy

The last two were removed because I think it's better to have a system that is as much as possible declarative and not imperative. The dispatcher's policy is now set from core's config file.

[mcolom]

Before approving these changes, I have some questions:

• The Dispatcher's test has been removed from ci_tests/all.py, but we seems to have it at ipol_demo/modules/dispatcher/test.py since there are changes at that file. Why the Dispatcher's test was removed from the list at the first file?
• The service's URL has moved from /api/dispatcher/get_demorunners_stats to /api/core/get_demorunners_stats. I don't think we should change that URL simply because the code was moved. The API is not reflecting the structure of the system, but an independent interface. It's better to keep the old URL.
• In ipol_demo/modules/core/core.py we have at line 51 an import using sys.path.append... I guess this is because you're keeping the folder's structure and it's the only way you have to import code from there, since you can't do it with an standard Python's import. I don't think this is complete. If you move the Dispatcher's to Core, the right way would be to actually move it into Core's folder as a Python module and import it normally. Otherwise it's a workaround.

[kidanger]

Thank you for the review.

1. The new tests are using pytest and should be ran with pytest ipol_demo/modules/dispatcher/test.py. Right now ci_tests/ is dedicated to run integration tests against each service. The new tests for dispatcher are more meant to be ran to validate individual commits / PR. I can set-up a first github actions workflow to run the dispatcher tests from there, to avoid the blind spot that my removal of dispatcher in all.py creates.
2. I agree; I'll take a look at cherrypy to see how we can have both URLs served by a single server.
3. OK. I'll can move the files to the core folder. Once most modules are converted, we can think about clean-up the folder tree a bit.

[mcolom]

Thanks for the changes! You can merge it into devel when you wish.

[kidanger]

TODO when deploying on prod:

• update core's venv
• update nginx config (/api/dispatcher -> proxy to 8080 without rewrite)