search

tianbaochou / NasUnet

170 stars 45 forks source link

The result of different model on NERVE dataset shows unusual similarity(Different from the results in the paper) #31

Open ProfessorHuang opened 4 years ago

ProfessorHuang commented 4 years ago

Thank you for your inspring work I tried your code on the NERVE dataset, but the result of Unet and NasUnet is very strange and similar. I run the code

python train.py --config='../../config/baseline/nerve.yml' --model='unet'

to train the unet on NERVE dataset and the result is image

python train.py --config='../../config/nas_unet/nas_unet_nerve.yml' --model='nasunet'

to train the nasunet and the result is image

It seems the DSC of two models are both about 0.77 and mIOU are both 0.983 However, the result in your paper is image

Why the performence of Unet is better than expected and the performence of NasUnet is worse than expected? Why the performence of two model are so similar, am I doing something wrong? Looking forward to your answer.

ProfessorHuang commented 3 years ago

Hello Andy, I just run the code in this repository. The code has no bug, but I can't reproduce the result. If you have some problems, you can open an issue. Good Luck! :-) Best Huang

AndyWorks96 commented 3 years ago

Ok,very Thanks!Good luck!

---Original--- From: "ziyan huang"<notifications@github.com> Date: Sat, Dec 19, 2020 11:08 AM To: "tianbaochou/NasUnet"<NasUnet@noreply.github.com>; Cc: "Comment"<comment@noreply.github.com>;"AndyWorks96"<2450157016@qq.com>; Subject: Re: [tianbaochou/NasUnet] The result of different model on NERVE dataset shows unusual similarity(Different from the results in the paper) (#31)

Hello Andy, I just run the code in this repository. The code has no bug, but I can't reproduce the result. If you have some problems, you can open an issue. Good Luck! :-) Best Huang

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

innocent-oss commented 1 year ago

Thank you for your inspring work I tried your code on the NERVE dataset, but the result of Unet and NasUnet is very strange and similar. I run the code

python train.py --config='../../config/baseline/nerve.yml' --model='unet'

to train the unet on NERVE dataset and the result is image

python train.py --config='../../config/nas_unet/nas_unet_nerve.yml' --model='nasunet'

to train the nasunet and the result is image

It seems the DSC of two models are both about 0.77 and mIOU are both 0.983 However, the result in your paper is image

Why the performence of Unet is better than expected and the performence of NasUnet is worse than expected? Why the performence of two model are so similar, am I doing something wrong? Looking forward to your answer.

can you share the steps you followed for implementation?

umaiskhan42 commented 9 months ago

This can be due to when you search a model you have to copy the architecture in geno_searched.py file in models, only this way you can use your seached achitecture for evaluation. from util.genotype import Genotype

''' Search on Pascal voc '''

enhance

NAS_UNET_V1_En = Genotype(down=[('down_conv', 0), ('down_dil_conv', 1), ('down_dep_conv', 0), ('down_dep_conv', 1)], down_concat=range(2, 4), up=[('cweight', 0), ('up_cweight', 1), ('conv', 0), ('up_cweight', 1)], up_concat=range(2, 4))

enhance + sharing

NAS_UNET_V1_En_sh = Genotype(down=[('down_cweight', 0), ('down_cweight', 1), ('down_dep_conv', 0), ('down_cweight', 1)], down_concat=range(2, 4), up=[('dep_conv', 0), ('up_cweight', 1), ('cweight', 0), ('up_cweight', 1)], up_concat=range(2, 4))

enhance

NAS_UNET_V2_En = Genotype(down=[('down_dep_conv', 0), ('down_dil_conv', 1), ('down_cweight', 0), ('down_dep_conv', 1), ('down_dep_conv', 1), ('down_dep_conv', 0)], down_concat=range(2, 5), up=[('identity', 0), ('up_dep_conv', 1), ('cweight', 0), ('up_cweight', 1), ('conv', 2), ('up_cweight', 1)], up_concat=range(2, 5))

NAS_UNET_V2 = Genotype(down=[('down_conv', 1), ('down_dep_conv', 0), ('down_cweight', 1), ('down_dil_conv', 0), ('down_dil_conv', 1), ('down_conv', 0)], down_concat=range(2, 5), up=[('identity', 0), ('up_cweight', 1), ('identity', 2), ('up_cweight', 1), ('cweight', 3), ('up_conv', 1)], up_concat=range(2, 5))

NAS_UNET_V3 = Genotype( down=[('down_dil_conv', 1), ('down_cweight', 0), ('down_cweight', 0), ('down_cweight', 1), ('down_cweight', 0), ('conv', 3), ('down_cweight', 0), ('conv', 4)], down_concat=range(2, 6), up=[('cweight', 0), ('up_cweight', 1), ('conv', 2), ('up_cweight', 1), ('up_cweight', 1), ('conv', 3), ('up_cweight', 1), ('conv', 4)], up_concat=range(2, 6))

enhance + no sharing

NAS_UNET_V3_En_sh = Genotype(down=[('down_dep_conv', 0), ('down_cweight', 1), ('conv', 2), ('down_cweight', 1), ('identity', 3), ('down_cweight', 1), ('down_dil_conv', 1), ('conv', 3)], down_concat=range(2, 6), up=[('cweight', 0), ('up_conv', 1), ('cweight', 2), ('up_conv', 1), ('cweight', 3), ('up_conv', 1), ('cweight', 0), ('up_cweight', 1)], up_concat=range(2, 6))

NAS_UNET_NEW_V3 = Genotype(down=[('down_dep_conv', 0), ('down_cweight', 1), ('down_conv', 1), ('max_pool', 0), ('max_pool', 1), ('cweight', 2), ('down_dil_conv', 0), ('down_dil_conv', 1)], down_concat=range(2, 6), up=[('dep_conv', 0), ('up_conv', 1), ('shuffle_conv', 0), ('up_cweight', 1), ('identity', 2), ('up_cweight', 1), ('dil_conv', 3), ('up_cweight', 1)], up_concat=range(2, 6))

NAS_UNET_NEW_V2 = Genotype(down=[('down_dil_conv', 1), ('down_dep_conv', 0), ('max_pool', 0), ('down_conv', 1), ('down_conv', 1), ('down_dil_conv', 0)], down_concat=range(2, 5), up=[('identity', 0), ('up_dil_conv', 1), ('identity', 0), ('up_dil_conv', 1), ('dil_conv', 3), ('up_cweight', 1)], up_concat=range(2, 5))

NAS_UNET_NEW_V1 = Genotype(down=[('down_dil_conv', 0), ('down_conv', 1), ('max_pool', 1), ('down_conv', 0)], down_concat=range(2, 4), up=[('conv', 0), ('up_dil_conv', 1), ('conv', 2), ('up_cweight', 1)], up_concat=range(2, 4))

NASUNET = NAS_UNET_V2

here you need to paste the genotype that you searched.

for example

NEW=Genotype(down=([.......................})