Toggle modes

[bnmajor]

This PR contains quite a few changes. To summarize:

Component refactoring

• At a high level: we used to create a separate component for the labeling mode and guided mode where each would create their own slide viewer with their own collection of components (for example: labeling dialog for labeling mode, keyboardshortcuts and filmstrip for guided mode). We now create one component, ActiveLearningGlobalComponent, which manages components that are reused regardless of mode (slide viewer, labeling dialog, information panel, opacity slider) as well as the visibility of those that are mode-specific (merge confirmation, predictions filmstrip).
• The slide (image) viewer used to be managed by two separate components (one for each "mode"): ActiveLearningInitialLabels and ActiveLearningContainer. This has been refactored to now only create one slide viewer for all modes called the ActiveLearningSlideViewer component.
• The ActiveLearningKeyboardShortcuts component has been removed completely and the ActiveLearningLabeling component is now re-used in both the labeling and guided mode.

UI and Feature Changes

These changes should match the mockup as closely as possible.

• There are now three options in the toolbar that allow the users to change modes: Labeling, Guided and Review. The review mode is disabled in this PR.
• Moved the image name in guided mode to a dialog now re-used across labeling and guided mode.
• In guided mode the predictions layer toggle as well as the retrain button have been moved to the labeling dialog.
• The predictions layer toggle is now available in labeling mode after the first training has been run.
• The static, text-only keyboard shortcuts dialog has been removed from the guided mode and replaced with the labeling dialog which now supports changing category name and color without changing modes.

Questions and Concerns

1. MERGING: We currently support the option to merge categories after the initial labeling step. This could become quite expensive though, as merging categories requires changing the values of (almost) every superpixel for each slide in order to keep the predictions annotation in sync with these changes.
   1. For every image there is a prediction annotation and for every superpixel on that annotation there is a value associated with it to represent the category that was predicted for it, so each value that has changed needs to be updated.
   2. This could become quite expensive. However, since this is not likely to be a common operation this may not be a concern.
2. DELETING: There is no obvious way to represent a deleted category in the predictions annotations. For labels we have a special "default" category that represents un-labeled superpixels, so deleting a category is as simple as replacing the superpixel values for the deleted category with the default "0". Since every superpixel for a prediction annotation has a prediction associated with it this concept of the absence of a prediction does not exist.
3. SYNCHRONIZING CHANGES: The previous two concerns hinge on the idea that the predictions should be kept in sync with the labels when users merge or delete categories. If we do not want that behavior then we face some additional work:
   1. All of the existing logic assumes that the two are in sync and often relies on comparing indices. I believe that this could be re-worked with some thought (i.e. making comparisons between labels and predictions based on name rather than index, although it would be even better if we had some kind of unique id for mapping labels to predictions instead).
4. Should the labeled superpixels count change as predictions are labeled?

[manthey]

As the UI switches between images, it jumps around a lot (the label list vanishes and comes back). Also, somehow I got this to occur:

[bnmajor]

As the UI switches between images, it jumps around a lot (the label list vanishes and comes back).

There was an unnecessary conditional for rendering the list of categories. This has been removed so there should be no more jumpy labeling dialog.

Also, somehow I got this to occur:

I can't reproduce this exact behavior but my best guess is that somehow the UI was in a state of constantly trying to re-compute how many prediction chips to display. I've refactored and simplified this logic so hopefully this solves this specific issue.

As to general jumpiness I am not seeing anything else in my testing, but if you are please let me know specifics and I will try to reproduce and resolve the issue!

[manthey]

Trying this branch again, I don't see the category counts change as I label superpixels. I tried a reload, and I got a display where I see and can't retrain. I am using two images.

[manthey]

I'm still seeing an issue where I take a two image project and start it, after creating categories before superpixels are generated, I then can seemingly label the superpixels but the labels don't save so I can't do training. Refreshing the interface resolves the issue.

The other occasional issues I see are when toggling between modes and going back to Guided mode, sometimes it seems like the wrong image is present (or at least the bounding box isn't aligned with the superpixel thumbnail). Similarly, after retraining once it appeared to show the wrong image (bounding box outside of the image bounds).

Third, when I type a label category hotkey on the last guided patch on the bottom, it was failing to advance to the next panel of patches. But I can't reliably reproduce this.

If it is easier, we can merge this in and see if there are still issues with mosaic view branch and resolve them there.

[bnmajor]

I'm still seeing an issue where I take a two image project and start it, after creating categories before superpixels are generated, I then can seemingly label the superpixels but the labels don't save so I can't do training. Refreshing the interface resolves the issue.

Just to note here: When @manthey and I last met to review these issues we did not see this behavior and I have still been unable to reproduce. The bug may still exist but it may be better to open this as a separate issue with any relevant details for reproducing the bug if it appears again.

The other occasional issues I see are when toggling between modes and going back to Guided mode, sometimes it seems like the wrong image is present (or at least the bounding box isn't aligned with the superpixel thumbnail). Similarly, after retraining once it appeared to show the wrong image (bounding box outside of the image bounds).

I was able to produce this bug once. I have added in an additional guard that I believe solves the issue, but I was unable to reproduce the bug a second time (with or without the fix) so I would appreciate some additional testing.

Third, when I type a label category hotkey on the last guided patch on the bottom, it was failing to advance to the next panel of patches. But I can't reliably reproduce this.

Despite seeing this bug first hand with @manthey I have been unable to reproduce and debug. I will open a separate issue for this to hopefully track and resolve shortly.

Additional fix: It was possible to begin to edit the hotkey for a category and, depending on how the input was selected, attempts to edit would be ignored. This issue has been resolved.

[bnmajor]

@manthey The lost labels bug seems to be corrected now, I'd love to confirm you are also no longer seeing issues!

Edit: The automatic "next page" for labeled superpixel chips should also be fixed now.

[manthey]

Awesome! I can no longer replicate the issues I had be seeing.