[BUG] Unable to clip ROI on dbscanClumped output

[lukasfue]

Describe the bug I clipped to a small ROI of my data to run DBSCAN on, so that it would run faster as I was just testing parameters. However, after DBSCAN ran, it was clear that it ran on all of the data, ignoring the ROI. I tried to clip the dbscanClumped output itself to an ROI and nothing happened.

To Reproduce Steps to reproduce the behavior:

1. Clip to ROI
2. Run DBSCAN
3. Visualize DBSCAN clusters
4. See that it DBSCANed all the data rather than just the RIO data
5. See that you aren't able to clip the dbscanClumped output

Expected behavior Ability to both run DBSCAN on ROI and to make an ROI of the DBSCAN output.

[csoeller]

This is may feel like a bug but is how currently the pipeline is supposed to work in VisGUI, i.e. all functions work on the current data source. F8 (clip to ROI) inserts an x-y filter into something called the 'output' filter which you can see in its own GUI box. All functions work on the datasource upstream of the output filter. You can work around this by inserting a TableFilter into the current recipe and put the x-y filter into that. Since it is difficult to do this manually I have made a plugin to do this from the current selection, see https://github.com/csoeller/PYME-extra/blob/master/PYMEcs/experimental/selectROIfilterTable.py.

@David-Baddeley will be the one to discuss with if the current layout of the pipeline in VisGUI needs tweaking or possibly more explanation (although I think this behaviour is in the docs somewhere).

[barentine]

I wonder if there's an intuitive way to do something in the GUI allowing users to move filters from the output filter upstream. Maybe a right-click option to move to the ~input filter we add by default?

[csoeller]

That and/or if feasible a copy/paste action of filter settings could be very useful, if it could be done using the available wx gui tools.

[lukasfue]

Is there a reason that functions work on the datasource upstream of output filters? I understand one could work around it, but it seems unnecessarily complicated unless that is something people want/use often. I'm just thinking if one makes an output filter it is probably likely that they only care to perform following functions on the filtered output. Or if they don't, they could easily remove the filter. I'm sure there's something I'm not thinking of, but just wanted to share my thoughts.

[David-Baddeley]

As it stands, the output filter is a hangover from the previous fixed pipeline, it is slated for removal once we have a suitable ROI replacement. In any case, ROI handling needs a bit of a work-over (see also #711, #461) as it's not particularly intuitive. Unfortunately, with the flexibility we have in the pipeline it's a bit hard to know what the right way to handle ROIs is. Propagating the ROI settings back to the initial filter is a potential option/workaround, but also has some significant drawbacks, including:

• moving the ROI selection upstream makes it a heavy-weight operation, potentially forcing lots of stuff to be recomputed when you change the ROI.
• the upstream filter is not guaranteed to be there or have a consistent name/location - it depends on which pipeline you use to open the data
• you generally won't want the same ROI between different images - if you want to save a pipeline to use on all images of a certain type, you probably don't want to save a cell-specific ROI into that pipeline
• it's unclear how to handle different datasources in this case (e.g. localisations and drift, separately analysed channels)

[David-Baddeley]

removing the bug label as the behaviour is technically expected, but leaving open (at least until we have a "What should ROIs do" discussion issue).