how to reproduce the result for cityscapes

SHMCU commented 4 years ago

Thanks for the valuable contribution! I am trying to reproduce the results for cityscapes, I used the configuration file cityscapes_spnet101.yaml The validation mIoU can only reach to ~73.x %. my experiment runs on 4 GPUs, sync_bn is enabled.

The only thing I changed is the batchsize, i am using 12 sample in a batch.

Could you give me some hints?

houqb commented 4 years ago

Thanks for your interests in this paper. I have tried spnet50 on Cityscapes and the mAP score is 79.5 on the validation set (w/o hard example mining). Please check your configuration.

BTW, I have updated some code. So, you can reclone it.

Kind regards, Qibin (Andrew)

On Wed, Jul 8, 2020 at 5:23 AM SHMCU notifications@github.com wrote:

Thanks for the valuable contribution! I am trying to reproduce the results for cityscapes, I used the configuration file cityscapes_spnet101.yaml The validation mIoU can only reach to ~73.x %. my experiment runs on 4 GPUs, sync_bn is enabled.

The only thing I changed is the batchsize, i am using 12 sample in a batch.

Could you give me some hints?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/Andrew-Qibin/SPNet/issues/13, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADNCN42ZBE3RWSPFTT3HOFLR2OG65ANCNFSM4OTU57GQ .

SHMCU commented 4 years ago

In your configuration, do you set use_apex=True ?

Thanks, SHMCU

SHMCU commented 4 years ago

Hi Qibin,

Could you share your configuration file for this spnet50 achieving mIoU 79.5 on Cityscapes val set?

Thanks, Hai

dongzhang89 commented 3 years ago

I have the similar question.

cm0561 commented 2 years ago

Thanks for the valuable contribution! I am trying to reproduce the results for cityscapes, I used the configuration file cityscapes_spnet101.yaml The validation mIoU can only reach to ~73.x %. my experiment runs on 4 GPUs, sync_bn is enabled.

The only thing I changed is the batchsize, i am using 12 sample in a batch.

Could you give me some hints?

I have the same problem ,can u give me your wechat number so we can discuss the problem, my wechat is lz_609492668

SHMCU commented 2 years ago

If it is trained on multiple GPUs, did you enable the sync_batchnorm ? On Sunday, November 14, 2021, 06:11:28 AM PST, cm0561 @.***> wrote:

Thanks for the valuable contribution! I am trying to reproduce the results for cityscapes, I used the configuration file cityscapes_spnet101.yaml The validation mIoU can only reach to ~73.x %. my experiment runs on 4 GPUs, sync_bn is enabled.

The only thing I changed is the batchsize, i am using 12 sample in a batch.

Could you give me some hints?

I have the same problem ,can u give me your wechat number so we can discuss the problem, my wechat is lz_609492668

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

cm0561 commented 2 years ago

Thanks for your repaly , now I train the model with only one gpu and later I will train the moel with multiple GPUs. can u give me some other advice, thanks anyway!

------------------ 原始邮件 ------------------ 发件人: "Andrew-Qibin/SPNet" @.>; 发送时间: 2021年11月15日(星期一) 凌晨3:38 @.>; @.**@.>; 主题: Re: [Andrew-Qibin/SPNet] how to reproduce the result for cityscapes (#13)

If it is trained on multiple GPUs, did you enable the sync_batchnorm ? On Sunday, November 14, 2021, 06:11:28 AM PST, cm0561 @.***> wrote:

Thanks for the valuable contribution! I am trying to reproduce the results for cityscapes, I used the configuration file cityscapes_spnet101.yaml The validation mIoU can only reach to ~73.x %. my experiment runs on 4 GPUs, sync_bn is enabled.

The only thing I changed is the batchsize, i am using 12 sample in a batch.

Could you give me some hints?

I have the same problem ,can u give me your wechat number so we can discuss the problem, my wechat is lz_609492668

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

cm0561 commented 2 years ago

Do u have a wechat,can i have your wechat so we can discuss the model in other problem,my wechat number is lz_609492668

------------------ 原始邮件 ------------------ 发件人: "Andrew-Qibin/SPNet" @.>; 发送时间: 2021年11月15日(星期一) 凌晨3:38 @.>; @.**@.>; 主题: Re: [Andrew-Qibin/SPNet] how to reproduce the result for cityscapes (#13)

If it is trained on multiple GPUs, did you enable the sync_batchnorm ? On Sunday, November 14, 2021, 06:11:28 AM PST, cm0561 @.***> wrote:

Thanks for the valuable contribution! I am trying to reproduce the results for cityscapes, I used the configuration file cityscapes_spnet101.yaml The validation mIoU can only reach to ~73.x %. my experiment runs on 4 GPUs, sync_bn is enabled.

The only thing I changed is the batchsize, i am using 12 sample in a batch.

Could you give me some hints?

I have the same problem ,can u give me your wechat number so we can discuss the problem, my wechat is lz_609492668

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

SHMCU commented 2 years ago

What is the batchsize if you are training on one GPU. If it is larger than 12, then it should be fine. On Sunday, November 14, 2021, 05:31:38 PM PST, cm0561 @.***> wrote:

Thanks for your repaly , now I train the model with only one gpu and later I will train the moel with multiple GPUs. can u give me some other advice, thanks anyway!

------------------ 原始邮件 ------------------ 发件人: "Andrew-Qibin/SPNet" @.>; 发送时间: 2021年11月15日(星期一) 凌晨3:38 @.>; @.**@.>; 主题: Re: [Andrew-Qibin/SPNet] how to reproduce the result for cityscapes (#13)

If it is trained on multiple GPUs, did you enable the sync_batchnorm ? On Sunday, November 14, 2021, 06:11:28 AM PST, cm0561 @.***> wrote:

Thanks for the valuable contribution! I am trying to reproduce the results for cityscapes, I used the configuration file cityscapes_spnet101.yaml The validation mIoU can only reach to ~73.x %. my experiment runs on 4 GPUs, sync_bn is enabled.

The only thing I changed is the batchsize, i am using 12 sample in a batch.

Could you give me some hints?

I have the same problem ,can u give me your wechat number so we can discuss the problem, my wechat is lz_609492668

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

cm0561 commented 2 years ago

I train with only one gpu and the batchsize can only be 1 , it will be an error of "cuda out of memory" if I set it 2, and I have a another problem even i set the batchsize 1 , when the model procress the eval it still have the error of "cuda out of memory", my gpu is 1080ti which have 11G memory!

------------------ 原始邮件 ------------------ 发件人: "Andrew-Qibin/SPNet" @.>; 发送时间: 2021年11月15日(星期一) 上午10:46 @.>; @.**@.>; 主题: Re: [Andrew-Qibin/SPNet] how to reproduce the result for cityscapes (#13)

What is the batchsize if you are training on one GPU. If it is larger than 12, then it should be fine. On Sunday, November 14, 2021, 05:31:38 PM PST, cm0561 @.***> wrote:

Thanks for your repaly , now I train the model with only one gpu and later I will train the moel with multiple GPUs. can u give me some other advice, thanks anyway!

------------------ 原始邮件 ------------------ 发件人: "Andrew-Qibin/SPNet" @.>; 发送时间: 2021年11月15日(星期一) 凌晨3:38 @.>; @.**@.>; 主题: Re: [Andrew-Qibin/SPNet] how to reproduce the result for cityscapes (#13)

If it is trained on multiple GPUs, did you enable the sync_batchnorm ? On Sunday, November 14, 2021, 06:11:28 AM PST, cm0561 @.***> wrote:

Thanks for the valuable contribution! I am trying to reproduce the results for cityscapes, I used the configuration file cityscapes_spnet101.yaml The validation mIoU can only reach to ~73.x %. my experiment runs on 4 GPUs, sync_bn is enabled.

The only thing I changed is the batchsize, i am using 12 sample in a batch.

Could you give me some hints?

I have the same problem ,can u give me your wechat number so we can discuss the problem, my wechat is lz_609492668

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

SHMCU commented 2 years ago

Okay, try to train with batchsize larger than 12. You can either choose to shrink the width of the layers or train at lower resolution or change to a smaller backbone.11G memory is not enough to play with segmentation models. You can try some other CV tasks but not these dense prediction task.Batch size 1 will definitely not work. Cause it makes the batchnorm layer fails. Think about how to compute variance if there is only one sample. On Sunday, November 14, 2021, 10:21:02 PM PST, cm0561 @.***> wrote: