Breaking out feature functionality

[0x00b1]

Hi, @jluethi (and @haesleinhuepf): I started writing a Napari plug-in for extracting standard image processing features (e.g. equivalent to the type of features extracted by CellProfiler). Would either (or both) of you be open to working together on this? The scope is much larger, e.g. ~1,200 features, so I'd understand if you want to keep something relatively simple! However, it would be awesome if we could collaborate on this.

[haesleinhuepf]

Hey @0x00b1

great to hear from you! I'm working on extending features regionally by exploiting neighborhood-relationships of cells, which also results in large numbers of features. Happy to join forces!

Cheers, Robert

[0x00b1]

@haesleinhuepf Awesome. What's your e-mail? I'll share the Google Doc I started.

I have the following neighborhood features:

location_object_neighborhood_angle
location_object_neighborhood_closest_0_distance
location_object_neighborhood_closest_0_object_index
location_object_neighborhood_closest_1_distance
location_object_neighborhood_closest_1_object_index
location_object_neighborhood_closest_2_distance
location_object_neighborhood_closest_2_object_index
location_object_neighborhood_neighbors
location_object_neighborhood_touching

Are you doing something more exotic?

How do you feel about the name napari-features?

[jluethi]

Sounds very useful indeed! I'd be interested to follow along, but not sure how much I'll work on feature extraction in general. napari-features sounds like a nice name for it!

@haesleinhuepf Would it make sense to merge the regionprops feature measurements into this? We've had some internal discussions and will probably set up single repositories per plugin at some point in the future.

Regarding feature measurements: I've always found DistanceToXClosest with X more like 10 or 20 and NumberOfObjectsWithinXDistance measurements somewhat useful as well :)

[0x00b1]

Regarding feature measurements: I've always found DistanceToXClosest with X more like 10 or 20 and NumberOfObjectsWithinXDistance measurements somewhat useful as well :)

Interesting!

[haesleinhuepf]

What's your e-mail? I'll share the Google Doc I started.

Awesome! I'm robert.haase@tu-dresden.de

Are you doing something more exotic?

Oh yes. Beyond features like distances to neighbors and numbers of neighbors, I can offer three different kinds of neighborhoods (and I'm working on the fourth):

• touching neighbors (and neighbors of neighbors etc)
• n-nearest neighbors
• proximal neighbors (within a given radius)

On those, you can compute statistical properties such as min/mean/max/mode/standard deviation of any kind of feature. Thus, you can for example compute the mean and standard deviation of the aspect_ratio of cells within a neighborhood with a given radius; or the mean of the aspect ratio plus two times standard deviation in this region or the number of cells with aspect ratio above the mean plus two times standard devation in the region. ;-) For analysing tissues such stuff is quite useful.

Btw, you find these operations in the napari-pyclesperanto-assistant

@haesleinhuepf Would it make sense to merge the regionprops feature measurements into this? We've had some internal discussions and will probably set up single repositories per plugin at some point in the future.

I seems that napari will become an ecosystem of many tiny plugins. I'm ok with that. However, I'd assume that end-users would prefer more complex and well organized plugins such as a "feature analysis toolbox" which comes with many useful tools as a collection: Feature extraction, visualization, analysis, plotting. Furthermore, such toolboxes have to be made by teams which increases quality and the bus factor. That's why I liked the combination here in this repository. But if you want to get rid of the scikit-image feature extraction GUI, don't worry. I can ship it as independent plugin, sure. Let me know what you'd prefer.

[0x00b1]

@haesleinhuepf Very cool Robert!

It seems that napari will become an ecosystem of many tiny plugins.

I have been thinking about this non-stop for the past week. We should talk soon. I would love to hear your thoughts. Frankly, I'm stuck trying to find the correct granularity so I'm open to all suggestions. This is especially challenging since, as far as I know, there is no concept (or planned concept), around sharing named scalar's across plugins.

But if you want to get rid of the scikit-image feature extraction GUI, don't worry. I can ship it as independent plugin, sure. Let me know what you'd prefer.

I like the magicgui widget!

(cc: @sofroniewn you might be interested in this conversation).

[sofroniewn]

I have been thinking about this non-stop for the past week. We should talk soon. I would love to hear your thoughts. Frankly, I'm stuck trying to find the correct granularity so I'm open to all suggestions. This is especially challenging since, as far as I know, there is no concept (or planned concept), around sharing named scalar's across plugins.

Indeed, I'd love to join a convo around this too - it's definitely tricky. Another relevant repo to look at is https://github.com/tlambert03/proto-skimage-widgets from @tlambert03

I seems that napari will become an ecosystem of many tiny plugins. I'm ok with that. However, I'd assume that end-users would prefer more complex and well organized plugins such as a "feature analysis toolbox" which comes with many useful tools as a collection: Feature extraction, visualization, analysis, plotting. Furthermore, such toolboxes have to be made by teams which increases quality and the bus factor.

I think for now I'm ok with some level of proliferation, folks trying different things and seeing what works.

We can also try and adopt some form of plugins within plugins model, where there could be "meta-like-plugins" say a napari-image-analysis which provided additional hookspecs for things like feature extraction, segmentation, registration etc.

I do like the idea though of being fairly explicit - is this plugin providing actually analyses or is it trying to organize and categorize analyses that could be provided by anyone.

All good things though!!

[tlambert03]

It seems that napari will become an ecosystem of many tiny plugins.

I don't see this as a long-term goal, as much as a short-medium term "restriction" if you will. I agree that end users ultimately want things to be complex and well organized... but what I don't think we want to see is a number of complex plugins that have largely overlapping functionality and varying degrees of organization. Instead, it would be awesome if napari could continue to do for interoperability between plugins what it is trying to do for common data types with layers. That is, if we can codify what it means for something to provide feature extraction or segmentation, etc... (in terms of types), then it needn't be up to a single monolithic workflow plugin to achieve that organized complexity. Instead, it can be a community conversation around the napari framework...

I agree with @sofroniewn that we definitely want to see folks trying different things and seeing what works, (that was a big motivation for the dock widget hookspec). But a much more valuable long-term vision is interop between plugins, and that will take time and conversation.

[haesleinhuepf]

I have been thinking about this non-stop for the past week. We should talk soon.

Maybe we append a feature-topic specific meeting right after a napari community meeting? For example, next week Wed June 9th 18:30 CEST? Otherwise, I'm pretty flexible in the CEST evenings and the image.sc island is always open.

Instead, it would be awesome if napari could continue to do for interoperability between plugins what it is trying to do for common data types with layers.

Yes, awesome! The magicgui table widget was already a super cool step in that direction :-)

But a much more valuable long-term vision is interop between plugins, and that will take time and conversation.

I need to get used to such long-term considerations. One decade of consecutive 0.5-1.5 year work contracts prevented me yet from thinking like this :-)

[sofroniewn]

Maybe we append a feature-topic specific meeting right after a napari community meeting? For example, next week Wed June 9th 18:30 CEST? Otherwise, I'm pretty flexible in the CEST evenings and the image.sc island is always open.

I have a conflict after the meeting, but agreed we should meet - maybe we move to zulip for scheduling

[jluethi]

I have a conflict after the meeting, but agreed we should meet - maybe we move to zulip for scheduling

@sofroniewn I'd be very interested to join this conversation as well! We're currently discussing in the lab about how we want to organize the plugins we're developing for data reading, visualization, classification etc. Some insights into the longer term ideas regarding organization, the potential for "meta-plugins" etc would be very helpful. I'll follow along on zulip then regarding what is scheduled

[jni]

Cool conversation!

1) tiny plugins are ok for the moment, we can bundle later as needed. But where it makes sense to join forces, earlier is better than later to avoid duplication of effort. It seems that feature computation (per segment, shape, vector, or location/point) is a good place to start to coordinate.

2) re "as far as I know, there is no concept (or planned concept), around sharing named scalar's across plugins", @0x00b1 do you mean per-region features? There's a little work to be done on the napari side to unify our feature handling across different layers. It's very high on my list. I think the goal is for properties to be "dataframe-like", and any "itemised" layer (points, vectors, shapes, tracks, labels — anything but Image I guess! :joy:) shares the same format. Then there's possibly some issues with namespacing of column names but otherwise what issues do you foresee?

Looks like the next meeting is not gonna be Oz friendly but I'm super-interested in this stuff! :blush: