search

Thanos-DB / FullyConvolutionalTransformer

Official implementation of The Fully Convolutional Transformer for Medical Image Segmentation
https://chaitanya-kaul.github.io/
112 stars 13 forks source link

cannot reproduce tensorflow version's result #12

Closed kingo233 closed 1 year ago

kingo233 commented 1 year ago

Hi!I ran the jupyter notebook and just changed the location of dataset and batch size.And I still can not reproduce the result. Here is my notebook and result: main_ACDC_92.84.md

Can you give a docker image or a big zip package contains code and data that I can reproduce the result ?

chaitanya-kaul commented 1 year ago

Hi,

I do not think it is possible to provide you with the data, as we don't own the dataset. I think that may cause problems with guidelines of the dataset as it is open source and hence it needs to be downloaded from its source. As we aren't owners of the dataset, we would violate the terms and conditions of the datasets by distributing it as we aren't authorised to do so. This is the reason we provide the jupyter notebook with our model training from scratch to 92.84 dice to show the results of our paper are fully reproducible. What we should however be able to provide are the pre trained models for the result on the ACDC dataset that you're trying to reproduce. I think we should still have that on our servers somewhere. Thanos is busy trying to make the BMVC deadline at the moment but once he's done he can find it and update the readme with a link to model weights. Our servers have crashed a few times since the paper has been published so the data might be gone, but we've done a K-Fold CV that suggests that the results are pretty consistent every time we run the model so I'm pretty sure we can definitely provide weights that are around 92.84 dice either way by retraining the model. Would that be something you'd be interested in? The docker container we used that will reproduce our environment is a part of the readme file that comes with our code and can be pulled from docker hub. We used that exact same environment on our GPU cluster to get our results. As to not being able to reproduce our results, what dice score are you getting with our model? We've seen dice ranging from 92.6 to 92.9 (when we did cross validation) so as long as it's something between these values the results are still fine as that difference is just because of stochasticity of neural networks in general depending on what random vectors the weights are initialised with at first. If the values are considerably different, please let us know. It might be version issues with tensorflow (we've seen different results based on different tensorflow versions ourselves). Hope this helps!

Best, Chaitanya

kingo233 commented 1 year ago
截屏2023-04-18 18 25 21

I don't know why I can't reproduce your result...my dice is so low...And I just changed the location of dataset and batch size.

Thanos-DB commented 1 year ago

Hi kingo233,

Here (https://github.com/Thanos-DB/FullyConvolutionalTransformer/issues/2) you can find the weights for the given notebook. Here (https://github.com/Thanos-DB/FullyConvolutionalTransformer/issues/6) you can find the split we used. As Chaitanya mentioned none or our trainings were this low. Are you using the docker file we provide?

BR Thanos