extracted logic from predict and evaluate endpoints into evaluation-handler class. The evaluation-handler defines a method that allows to trigger prediction and evaluation at once, as well as separately
Triggered evaluations are now saved in DB with reference to the user that started it
predictions/metrics will only be computed if not already done before
whenever a skill is created or updated via the skill-manager the evaluation will be triggerd on the default metric of the selected datasets
How to run locally:
you need both the skill-manager and the evaluator to be running locally!
in root folder: create env and activate it
go into skill-manager and do make install and make install-dev
What does this PR do?
How to run locally: you need both the skill-manager and the evaluator to be running locally!
make install
andmake install-dev
EVALUATOR_API_URL
in skill-managers.local.env file
(http://square-core-evaluator-1:8081/api)make install
andmake install-dev
SKILL_MANAGER_API_URL
in evaluators.local.env file
(http://square-core-skill-manager-1:8000/api)in root folder:
docker compose --env-file evaluator/.local.env up -d
volumes: mongo-data: