Gather requirements from Scivision community

[ots22]

The aim of this issue is to collect some high-level requirements or use-cases where pixelflow could be helpful. Open additional issues based on the replies here. Please leave a comment about your use case or requirements with as much detail as you like.

[Tonks684]

I'm currently using CellProfiler to compute 200+ morphological, textual and intensity features using an intensity image and an instance mask. I had to create the pipeline in the CellProfiler GUI first and then separate to my Python-based workflow I had to extract the features for 54 subsets of my data. I think having a wrapper around CellProfiler features would also be a very good use case as keep everything in Python. Some of the simpler features such as Area/Shape etc come from skimage.measure. regionprops so this aligns with what Oliver has already suggested.

[evangeline-corcoran]

Recreate and enhance efficiency calculating seed size and shape metrics from instance segmentation masks.

For 2D images

• General shape metrics ('label', 'bbox', 'centroid', 'area', 'equivalent_diameter', 'major_axis_length', 'minor_axis_length', 'orientation', 'solidity') can be extracted with existing pixelflow 'features'

For 3D images

• label, bbox, and centroid can be obtained from existing set of features
• Need to include PoreSpy regionprops_3D function in features to get 3D shape metrics

For both

• Additional custom function for predict which valve seeds sit in and collecting valve specific measurements, based on existing R script to do this

[acocac]

For geospatial/satellite images:

• Retain geolocation information/metadata (coordinate reference system, image/scene ID, temporal components, among others).
• Landscape metrics for ecological analysis

[lupinthief]

I've also used MSPA before and found it useful: https://isprs-archives.copernicus.org/articles/XLVIII-4-W1-2022/427/2022/

[lupinthief]

For my iceberg work I'm using EFDs (https://github.com/hbldh/pyefd) a lot and also dynamic time warping for distances between curves (currently using this: https://github.com/wannesm/dtaidistance, though shapeDTW claims to do better at matching based on local rather than global features: https://github.com/MikolajSzafraniecUPDS/shapedtw-python). For DTW, what is really lacking is being able to handle global invariances (translation, rotation, scale) in tandem with DTW with the consequence that for the geospatial context it is tricky as the fact that objects are geographically distant from each other results in large differences even if their curves are identical. Key requirements for shape descriptors for me are being able to have them translation and rotation invariant, while reflection and scale invariance is a nice option to have but not needed for my particular use-case.

[evangeline-corcoran]

Requirements fitting into 3 main areas were identified during discussion with BAS team on E&S Workstream 2.3:

Georeferenced Data Processing (similar to what Alejandro has proposed above)

• Ability to propagate spatial metadata through all stages of analysis, so that i.e. when retrieving centroids of detect objects you also get the spatial coordinates
• Could potentially wrap Shapely/Geopandas
• Ability to input/output vectors and polygons instead of masks, and to convert inputs into vectors, geojson or netcdf files.

Shape Analysis

• Ability to output shape descriptors that can be used in downsteam analysis of shape similarity i.e. k-means

Abundance, Distribution and Morphometry of Wildlife

• Area of Walrus masks
• Size and shape of stranded whales i.e. length
• Current pixelflow functions can provide some relevant measurements for these use cases but would be good to expand

[Trotts]

I've briefly mentioned this to @IFenton on Slack, but also posting here after the Workstream meeting (sorry I wasn't able to make it live).

My work is on automating benthic taxa detection/ID, but downstream these detections are used for things such as community distribution analysis and size estimation. All our benthic images contain 3 laser points forming a triangle which are a set distance apart, so we have a pixel to metric conversion scale available.

Our images are also very similar to the AMI data e.g. bounding boxes, though aren't clean background and likely larger in size. Perhaps sometime down the line it would be good to see if Pixelflow could help here?

[lupinthief]

SOAPs

[lupinthief]

Provide results only for a sub-region(s) of the image, either through a GUI interface to define ROI manually or by passing bboxes/polygon masks some possible functionality here: https://github.com/hi-paris/deepdespeckling