Check dice scores

[davidborland]

Check dice scores by downloading training volumes and diffing.

[davidborland]

Downloaded and calculated dice score in Matlab, and got the same results as in ninjato: 0.9397 0.9458 0.9397 0.9425 0.9558 0.9397 0.9459 0.9397 0.9397 0.9510

@talafar Has already confirmed that the 5 volumes with the same value have differences.

[davidborland]

Turns out that some assignments had not been verified, and thus not updated in the whole volume for each trainee, so the Dice score did not match their current work.

Need to determine whether to only compute Dice score on verified assignments or not.

[davidborland]

@hyi will add a new _user_intermediate.tif file to the _whole subvolume to track user edits so an up-to-date Dice score can be calculated as users save their edits.

[hyi]

@davidborland I have added the work for this in this branch https://github.com/RENCI/ninjato/tree/add-intermediate-volume. Please test it out to see if it works well. I'll hold off merging the PR until you have a chance to test it out.

[davidborland]

We figured out that assignments are only updated with verified changes from other overlapping assignments only if annotation_done metadata is not true. Therefore, reviewers may not see verified changes when verifying an assignment, causing verified changes to be overwritten. Proposed changes to fix this:

1. Apply verified changes to all active assignments that have not been verified yet.
2. Calculate a diff between the beginning state of the assignment (already saved per assignment) and the users edits, and only update the whole volume with voxels the user has actually edited.

[davidborland]

Most recent comment now covered by #274