Add a console warning, for when the set of tags used in a given test tagging, is not the same as the set of tags reflected in the gold tagging. The warning should say what is the difference between the sets (preferably in a very concise way).
If an allowed set of tags is provided in the mapping configuration file, the set of gold tags for this purpose should be considered as that set. We can call it very generally, the effective set of tags of the evaluation, and the warning message should explain how this set was obtained (i.e. implicitly or explicitly from the provided mapping file).
Add a console warning, for when the set of tags used in a given test tagging, is not the same as the set of tags reflected in the gold tagging. The warning should say what is the difference between the sets (preferably in a very concise way).
If an allowed set of tags is provided in the mapping configuration file, the set of gold tags for this purpose should be considered as that set. We can call it very generally, the effective set of tags of the evaluation, and the warning message should explain how this set was obtained (i.e. implicitly or explicitly from the provided mapping file).