3D image classification / MONAI integration

[zhaoedf]

Support 3D image classification. See comment: https://github.com/PyTorchLightning/lightning-flash/issues/886#issuecomment-958956679

[dmarx]

One of these might be what you're looking for:

• https://lightning-flash.readthedocs.io/en/latest/reference/pointcloud_object_detection.html
• https://lightning-flash.readthedocs.io/en/latest/reference/pointcloud_segmentation.html
• https://lightning-flash.readthedocs.io/en/latest/reference/graph_classification.html

If none of those fits your needs, could you provide some additional details on the problem you are working on? What format are your "3D image" inputs? What kind of classifications are you trying to achieve? Can you maybe link to an example of your task in practice, like an academic publication or blog post working through a problem of the kind you are hoping to see supported?

[zhaoedf]

One of these might be what you're looking for:

• https://lightning-flash.readthedocs.io/en/latest/reference/pointcloud_object_detection.html
• https://lightning-flash.readthedocs.io/en/latest/reference/pointcloud_segmentation.html
• https://lightning-flash.readthedocs.io/en/latest/reference/graph_classification.html

If none of those fits your needs, could you provide some additional details on the problem you are working on? What format are your "3D image" inputs? What kind of classifications are you trying to achieve? Can you maybe link to an example of your task in practice, like an academic publication or blog post working through a problem of the kind you are hoping to see supported?

i mean, can i use any 3d classfication model like resnet18-3d in ImageClassifier?

[zhaoedf]

One of these might be what you're looking for:

• https://lightning-flash.readthedocs.io/en/latest/reference/pointcloud_object_detection.html
• https://lightning-flash.readthedocs.io/en/latest/reference/pointcloud_segmentation.html
• https://lightning-flash.readthedocs.io/en/latest/reference/graph_classification.html

If none of those fits your needs, could you provide some additional details on the problem you are working on? What format are your "3D image" inputs? What kind of classifications are you trying to achieve? Can you maybe link to an example of your task in practice, like an academic publication or blog post working through a problem of the kind you are hoping to see supported?

i mean, can i use any 3d classfication model like resnet18-3d in ImageClassifier?

no need, i find a way, you close this issue.

[dmarx]

Glad to hear it! You should consider sharing your solution in case someone with the same problem finds this in the future. VIdeo isn't my domain, but I can contribute this much:

# import flash; print(flash.__version__)  # 0.5.1
from flash.video import VideoClassifier
VideoClassifier.available_backbones()

 'csn_r101',
 'efficient_x3d_s',
 'efficient_x3d_xs',
 'i3d_r50',
 'r2plus1d_r50',
 'slow_r50',
 'slow_r50_detection',
 'slowfast_16x8_r101_50_50',
 'slowfast_r101',
 'slowfast_r50',
 'slowfast_r50_detection',
 'x3d_l',
 'x3d_m',
 'x3d_s',
 'x3d_xs']

[ethanwharris]

@zhaoedf We don't currently support 3D models within the image classifier. We have been interested in having a 3DImageClassifier task (ideally including integration with MONAI). I think this notebook from @Borda would make a great starting point: https://www.kaggle.com/jirkaborovec/brain-tumor-classif-lightning-monai-resnet3d

I'll leave this issue open as a feature request for 3D image classification :smiley:

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.