Add Ability for User to Remove Their Doodles

[venuswku]

Main Features

• Using Tap tool to select the doodle(s) to remove.
  • This is an example of how to select, unselect, and remove a doodle. I added instructions on how to use the Tap tool in case users forget or never used it before.
  • I added a Selection1D stream for each doodle plot (_draw has doodles that are drawn with the currently chosen segmentation class and line width, _drawn has the rest of the doodles) to check if doodles are tapped/selected in the _set_remove_doodles_ability() subscriber. Please test selecting and removing doodles if you have time! You might find new test cases that I've never tried out before.

    # doodler > components.py > DoodleDrawer class
    def __init__(self, class_color_mapping, **params):
    # ...
    # Create a custom widget (allows dynamically setting disabled property) for the remove_doodles parameter.
    self._remove_doodles_button = pn.widgets.Button.from_param(
        parameter=self.param.remove_doodles,
        name='Remove selected doodle(s)',
        button_type='default', disabled=True
    )
    # For each DynamicMap containing doodles, create a Selection1D linked stream
    # and attach it to the DynamicMap to see if at least one doodle was selected.
    self._draw_selection_stream = hv.streams.Selection1D(source=self._draw)
    self._drawn_selection_stream = hv.streams.Selection1D(source=self._drawn)
    # Add a subscriber that enables/disables the ability to remove doodles depending on whether a doodle was selected.
    self._draw_selection_stream.add_subscriber(self._set_remove_doodles_ability)
    self._drawn_selection_stream.add_subscriber(self._set_remove_doodles_ability)
    # ...
    def _set_remove_doodles_ability(self, index: Optional[List[int]] = []):
    """
    Enables/disables the button for removing doodles depending on whether doodles are selected.
    """
    # Enable the remove_doodles button if at least one doodle is selected.
    if index: self._remove_doodles_button.disabled = False
    # Else disable the button if no doodles are selected.
    else: self._remove_doodles_button.disabled = True

• Clicking Remove selected doodle(s) button to confirm removing.
  • This is an example of how to remove more than one doodle from different segmentation classes.
  • The Remove selected doodle(s) button is only clickable when at least one doodle is selected, so you can see the button is greyed out whenever no doodles are selected.
  • I added the remove_doodles event parameter so that the _remove_doodles() function could be called whenever an event is triggered (by pressing the _remove_doodles_button).

    # doodler > components.py > DoodleDrawer class
    # ...
    remove_doodles = param.Event(label='Remove selected doodle(s)', doc='Button to remove the selected doodle(s)')
    # ...
    def __init__(self, class_color_mapping, **params):
    self._accumulated_lines = []  # List of dataframes
    # ...
    # Create a FreeHandDraw linked stream and attach it to the DynamicMap/
    # The DynamicMap plot is going to serve as a support for the draw tool,
    # and the data is going to be saved in the stream (see .element or .data).
    self._draw_stream = hv.streams.FreehandDraw(source=self._draw)
    # ...
    @param.depends('remove_doodles', watch=True)
    def _remove_doodles(self):
    """Removes any selected doodles when the button for removing doodles is enabled and clicked.
    """
    # If the user is allowed to remove selected doodles (button is enabled)...
    if not self._remove_doodles_button.disabled:
        if self._draw_selection_stream.index:
            # Remove the selected doodle(s) from the draw stream.
            remaining_doodles_data = self._draw_stream.data.copy()
            for col, all_doodles_vals in remaining_doodles_data.items():
                remaining_doodles_data[col] = [one_doodle_vals for i, one_doodle_vals in enumerate(all_doodles_vals) if i not in self._draw_selection_stream.index]
            self._draw_stream.event(data=remaining_doodles_data)
            # Plot the non-removed doodles by sending the draw pipe the modified list of dataframes.
            remaining_doodles_dataframes = [element.dframe() for element in self._draw_stream.element.split()]
            self._draw_pipe.event(data=remaining_doodles_dataframes)
        if self._drawn_selection_stream.index:
            # Remove the dataframe that corresponds to each selected drawn doodle.
            self._accumulated_lines = [doodle for i, doodle in enumerate(self._accumulated_lines) if i not in self._drawn_selection_stream.index]
            # Plot the non-removed drawn doodles by sending the drawn pipe the modified list of dataframes.
            self._drawn_pipe.event(data=self._accumulated_lines)
        # Disable the button once the doodles are removed and no doodles are selected.
        self._remove_doodles_button.disabled = True

  • Different methods are used to remove doodles from the _draw and _drawn plot because _draw stores doodles in the FreehandDraw stream and _drawn stores doodles in _accumulated_lines.
• Note: Remember to switch back to the Freehand Draw tool to continue doodling!
  • If no lines are appearing on the image when you're doodling, it might be because the Tap tool is still active after removing doodles.
  • Clicking the Tap tool while the Freehand Draw tool is active will automatically unselect the Freehand Draw tool and vice versa. You can see this happening in the GIFs above.
  • It's hard to distinguish if the user is doodling a dot with the Freehand Draw tool or selecting an empty image area with the Tap tool, so I think that's why Bokeh doesn't allow both tools to be active at the same time.
  • Switching between the tools might be confusing for first-time users. I've tried to programmatically set the Tap tool as inactive after removing doodles by using the hooks plot option (similar to this example but slightly modified since finalize_hooks is renamed to hooks now), but nothing happened so I got rid of my code. Feel free to fix my implementation:

    # doodler > components.py > DoodleDrawer class
    def __init__(self, class_color_mapping, **params):
        # ...
        _shared_plot_opts = {
            'selected': [], 'hooks': [self._inactive_tap_tool],
            'selection_line_color': '#000000', 'selection_line_width': 5, 'nonselection_alpha': 1
        }
        # ...
        self._draw = hv.DynamicMap(self._update_draw_cb, streams=[self._draw_pipe]).apply.opts(
            color=self.param.line_color, line_width=self.param.line_width
        ).opts(active_tools=['freehand_draw'], **(_shared_plot_opts))
        # ...
        self._drawn = hv.DynamicMap(self._drawn_cb, streams=[self._drawn_pipe]).apply.opts(
            color='line_color', line_width='line_width'
        ).opts(tools=['tap'], **(_shared_plot_opts))
    # ...
    def _inactive_tap_tool(self, plot, element):
        """
        Sets all tap tools as inactive for the given plot, which unselects the Tap tool whenever doodles are removed
        and each DynamicMap's callback returns a new plot (user doesn't need to remember to switch back to the Freehand Draw tool).
        """
        plot.state.toolbar.active_tap = None

Other Considered Implementation Ideas

• The Freehand Draw tool has an in-built feature for removing doodles by selecting a doodle and pressing the BACKSPACE (Windows) or DELETE (Mac) key. You actually don't need to switch to the Tap tool all!
  • You can try it out in the docs too! You can't see in the GIF below but I just pressed the BACKSPACE/DELETE key to remove the doodle that I selected. No other tools are needed.
  • Should I mention this hidden feature in the instructions? I excluded since it might be confusing if only doodles in the _draw plot can be removed this way. Also, users probably don't even know that the doodles are split into two different types of plots (I didn't know until I looked closely at the code and read about how HoloViews streams work). If they try to select a doodle that they didn't know was in the _drawn plot, then they'll probably think there's a bug since the Tap tool isn't active and nothing gets selected.
    • This is an example of how the dark blue doodles in the _draw plot could be selected and removed with just the Freehand Draw tool but the yellow doodles in the _drawn plot can't. The yellow doodles can only be selected with the Tap tool.
    • P.S. Message me if this isn't clear enough. Or you can just ignore all this since I didn't mention any of this in the instructions anyway.
  • Instructing users to enable the Tap tool before selecting any doodle seems more consistent for the user experience, but users are welcome to use this in-built shortcut if they know which doodles belong to the _draw plot. As a reminder, doodles with the currently selected segmentation class and line width are in the _draw plot (e.g. dark blue doodles in the very first GIF above).
• I originally wanted to implement the ability to edit doodles too, but I decided to not because there might be some inconsistencies for the user experience again.
  • _drawn is currently a Contours plot, which is useful for placing all the previously drawn doodles in a single plot. However, HoloViews only has CurveEdit and PolyEdit streams for editing plot elements.
  • Even if I'm able convert the _drawn Contours plot into a Curve or Polygon plot, there's currently no way to interactively edit (e.g. moving and deleting vertices) the Freehand Draw doodles in the _draw plot. Removing them and redrawing is the best option right now. Do we want to close this issue or look further into this?

[2320sharon]

This branch works on all the tests I've tried so far. Great work!

[venuswku]

Thank you so much for testing Sharon!!! You're an angel I swear 😄

[dbuscombe-usgs]

Fantastic work @venuswku !!