We now need to have a tool for ensuring that some sounds are completely labeled (the content has all the labels that applies to it, and they are as specific as possible).
For that we'll take the tools from the annotation-tools-FRUCT2018 branch.
The idea is to combine the two different tools, so that our expert annotators can:
inspect existing labels and make them more precise or refine their degree of presence
add missing labels & define their degree of presence
In terms of data management, after some discussion with @ffont, we think that the best way is to enable some sort of expert votes. When an expert annotator would:
modify the degree of presence of a ground truth annotation, it would create an expert vote for its associated candidate annotation. Then the process of generating the ground truth annotation should be re-triggered and should give credit to the expert vote without considering the non-expert votes.
make a label more precise and add a missing label, it would create a candidate annotation if needed, create an expert vote and then proceed the same way explained before.
Work to be done:
Back:
Add expert boolean field in Vote model
Adapt the ground_truth_state() property method in CandidateAnnotation model in order to give credit to expert votes.
Adapt the save() method of Vote model for enabling it to modify a ground truth annotation when it already exists (it should be possible to change the degree of presence stored in the ground_truth field).
Create a view function which can create new candidates & expert votes associated to them.
Front:
Combine the annotation tools so that the annotators can inspect existing labels, refine them, make them more precise and add new labels.
The added label section should list the existing ground truth annotations.
When an annotator uses the text search engine to find a category, enable him to directly add the label or locate it in the taxonomy table (to speed up their work).
Can we find a way to have the added label section always visible?
Tests:
Modifying the degree of presence of an existing ground truth annotation:
creates an expert vote
modify the ground truth annotation's ground_truth field.
Generating a label (make more precise & generate new label should work the same way):
creates a candidate annotation if it does not exist yet
We now need to have a tool for ensuring that some sounds are completely labeled (the content has all the labels that applies to it, and they are as specific as possible).
For that we'll take the tools from the annotation-tools-FRUCT2018 branch. The idea is to combine the two different tools, so that our expert annotators can:
In terms of data management, after some discussion with @ffont, we think that the best way is to enable some sort of expert votes. When an expert annotator would:
Work to be done:
Back:
expertboolean field inVotemodelground_truth_state()property method inCandidateAnnotationmodel in order to give credit to expert votes.save()method ofVotemodel for enabling it to modify a ground truth annotation when it already exists (it should be possible to change the degree of presence stored in theground_truthfield).Front:
Tests:
ground_truthfield.