search

WinstonHuTiger / mamba_mae

The official Pytorch implementation for the paper, Learning Brain Tumor Representation in 3D High-Resolution MR Images via Interpretable State Space Models
https://arxiv.org/pdf/2409.07746
Other
3 stars 0 forks source link

About pre-trained checkpoints #2

Open jianjun0407 opened 3 days ago

jianjun0407 commented 3 days ago

Thank you for your sharing. I have tried your method, and now I have encountered some problems, please advise me. First of all, I downloaded your open source three model parameters, and I visualized the MAE part using the code: visualize.py. The results are as follows ( test image, masked test image, network output image, and the masked part of the test image is backfilled with network prediction results): 11111111111

From the test results, checkpoint-999_patchSize_16.pth cannot reconstruct the test image. I think this is the problem I use, so I would like to ask: (1) What data processing needs to be done for the test image sent into the network? (2) Referring to what you wrote in the readme, my current data processing process is: first crop the foreground area, then resize it to [160,160,160], and then normalize it by min-max, and then multiply it by 255. Finally, the processed data is used to call visualize.py. May I ask if this is the right way to process the image? Given the current test results, where do you think my problem occurred?

222222

WinstonHuTiger commented 3 days ago

May I ask what data you used and it seems the masking ratio is not right. I am not sure that the model can reconstruct the image for masking ratios other than 0.75.

jianjun0407 commented 1 day ago

Thank you for your attention, and I will repeat the above operations in detail below, mainly including data processing, testing and visualization process. Please also take a look at my current processing and any problems: (1) I used the data of the BraTs2023 challenge, which should be similar to the data of the BraTs2022 challenge you used. In the following, take the "BraTs-GLI-00000-000" data as an example. (2) Refer to the readme, this section of data processing requirements: 222222

My approach is to crop out the foreground area of the image, then resize to [160,160,160], then use the maximum-minimum normalization method, zoom to [0,1], then multiply by 255, and finally save the processed data in xxx.npy format.

(3) When visualizing, The "checkpoint-999_patchSize_16.pth" model is used, with mask_ratio=0.75. Here is the code for the visualization section, along with the results : 44444444 55555

Please help me to look at the use of this checkpint where my current problem occurs? Or, what about the.NPY process you suggest for converting match data to code use ?

WinstonHuTiger commented 1 day ago

Sorry, I might made a mistake on the dataset description there.
It should be brats2021 dataset, but 2022 version and 2021 version are the same. I can share with you the file where we store our preprocessing records.

Here is one example visualization of the reconstruction in EGD dataset using the provided checkpoint.

image

I think you have to store the file in nii.gz's format.

jianjun0407 commented 1 day ago

Thank you for your reply, I think my problem may appear in the image coordinate system： (1) My image, which is opened in itk-snap as shown in the following figure, has different positions in the cross, sagittal and coronal planes from your example image above : 999999999

(2) Can you upload the code for the data processing part? For example, if you take data from the brats2021 challenge and read in the .NIi.gz data, what do you need to do to start training with main_pretrain.py? Specifically, I see that the dataLoader part of main_pretrain.py reads the code as follows, which converts the data of brats2021 challenge to the data that can be processed by the following code. The code of this part can be uploaded ? thank you~

88888888