nclskfm / square-core

SQuARE: Software for question answering research.
https://square.ukp-lab.de
2 stars 0 forks source link

Auto-evaluate skills after saving #28

Closed BenediktStuhrmann closed 1 year ago

BenediktStuhrmann commented 1 year ago

Whenever a skill gets saved/updated, it should be automatically evaluated on all the datasets (with the default metric of the dataset) that were selected for the skill.

To be discussed: What happens when the skill gets deleted? Delete results? Keep them?