TransOrga

Organoid research plays an important role in drug screening and disease modeling. Obtaining accurate information about organoid morphology, number, and size is fundamental to this research. However, previous methods relied on fluorescence labeling which can harm organoids or have problems with accuracy and robustness. In this paper, we first introduce Transformer architecture into the organoid segmentation task and propose an end-to-end multi-modal method named TransOrga. To enhance the accuracy and robustness, we utilize a multi-modal feature extraction module to blend spatial and frequency domain features of organoid images. Furthermore, we propose a multi-branch aggregation decoder that learns diverse contexts from various Transformer layers to predict the segmentation mask progressively. In addition, we design a series of losses, including focal loss, dice loss, compact loss and auxiliary loss, to supervise our model to predict more accurate segmentation results with rational sizes and shapes. Our extensive experiments demonstrate that our method outperforms the baselines in organoid segmentation and provides an automatic, robust, and fluorescent-free tool for organoid research.

Environment

pytorch 1.10.0 scikit-image 0.20.0 numpy 1.23.5 opencv-python 4.7.0.72 scikit-learn 1.2.2 monai 1.1.0 d2l 0.17.6 timm 0.6.12 einops 0.6.0

Dataset

The dataset for model training and validation/testing is openly available here: https://osf.io/xmes4/ Please download the dataset and put them under /dataset

Preprocess

The frequency-level images of https://osf.io/xmes4/ is SR_result and the edge images is edge.

Please unzip these files and place the folders in the root directory, like

'TransOrga/Dataset',

'TransOrga/SR_results',

'TransOrga/edge_results'.

If you want to utilize your data: Please first obtain the frequency-level images using SRNET.py

(Please change the folder dirs as your owns.)

Test

Our pretrained model is here .

1. Download the pretrained model and put it in 'checkpoints/'.

2. create 'log_results/' as the output folder.

3. run test.py to obtain the results.

Train

You can download the official pretrained vit-base model here.

1. dataset.py contains how to load the data. (Please change the folder dirs as your owns.)
2. run train.py

Result

ACC organoid	Model	Dice ↑	mIoU↑	Precision↑	Recall↑	F1-score↑
SegNet	0.798	0.664	0.579	0.803	0.630
A-Unet	0.884	0.791	0.671	0.952	0.783
OrganoID	0.848	0.736	0.622	0.866	0.716
Ours	0.913	0.840	0.791	0.903	0.843