I don't think the relevant line in the "check_sample" function is what we want.
It enforces that the specified tasks are all in the samples. Some specified task might not have an equivalent sample (like 't' --> translation; there is no "sample" that corresponds to translation, but we may still want to specify it as a task).
Also, the "filter_samples()" function does enforces a weaker version of this, and there is a test case in "test_data.py" that ensures it is working properly
vict0rsch
commented
4 years ago
I don't think the relevant line in the "check_sample" function is what we want.
It enforces that the specified tasks are all in the samples. Some specified task might not have an equivalent sample (like 't' --> translation; there is no "sample" that corresponds to translation, but we may still want to specify it as a task).
Also, the "filter_samples()" function does enforces a weaker version of this, and there is a test case in "test_data.py" that ensures it is working properly