Project-MONAI / MONAI

AI Toolkit for Healthcare Imaging
https://monai.io/
Apache License 2.0
5.66k stars 1.03k forks source link

Transforms support points and bounding boxes. #3375

Closed fluentgcc closed 2 years ago

fluentgcc commented 2 years ago

Is your feature request related to a problem? Please describe. There is a lot of other task except segmentation of 3d medical image, such as keypoints regression, bbox detection, which needs to transform points and box along with image spacial transformations. Also, when doing segmentation task, sometimes we have some extra infomation to spacial crop and sampling in order to balance samples, e.g using extra center to to crop. It will be useful that all transforms support points and bboxes.

Describe the solution you'd like For transforms add two other augments to transform points and bboxes:

class RandRotated(RandomizableTransform, MapTransform, InvertibleTransform):
    def __init__(
        self,
        keys: KeysCollection,
        point_keys:: KeysCollection,
        bbox_keys:: KeysCollection, 
        ...

Where point should be Nx4 (in 2d case may be Nx3) in shape, for three axis value and one class value, (x,y,z,class). And box should be Nx7 , (x,y,z,w,h,d,class).

wyli commented 2 years ago

This is a good point, we'll implement relevant ideas soon, pls see also https://github.com/Project-MONAI/MONAI/pull/324 https://github.com/Project-MONAI/MONAI/discussions/3354, cc @Can-Zhao @Project-MONAI/core-reviewers

wyli commented 2 years ago

initial support available in https://github.com/Project-MONAI/MONAI/tree/dev/monai/apps/detection/transforms