search

eccv22-ood-workshop / ROBIN-dataset

ECCV 2022 Workshop: A Benchmark for Robustness to Individual Nuisances in Real-World Out-of-Distribution Shifts
http://www.ood-cv.org/
46 stars 6 forks source link

Request for a further delay of challenge ending #32

Open vtddggg opened 2 years ago

vtddggg commented 2 years ago

Hi @DTennant ,

We have receive the delay of the end to Oct. 1, 2022. However, Oct. 1 is the National Day in China, which means most people may not have enough time working for the challenge. Can we ask for a further delay of ending?

Thanks!!

DTennant commented 2 years ago

Hi,

The decision for extending the deadline to Oct. 1 is agreed upon by all organizing committee members, and it can't be further extended to ensure that the leaderboard won't be overfitted. We do realize that Oct. 1 is a national holiday in China, most of the committee members are Chinese, we are very sorry for the inconvenience.

Best regards, Bingchen Zhao

vtddggg @.***> 于2022年9月30日周五 11:22写道：

Hi @DTennant https://github.com/DTennant ,

We have receive the delay of the end to Oct. 1, 2022. However, Oct. 1 is the National Day in China, which means most people may not have enough time working for the challenge. Can we ask for a further delay of ending?

Thanks!!

— Reply to this email directly, view it on GitHub https://github.com/eccv22-ood-workshop/ROBIN-dataset/issues/32, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADY4MVOUV3S3PO4CK6SUARLWAZMIHANCNFSM6AAAAAAQZK6SHY . You are receiving this because you were mentioned.Message ID: @.***>

wujiekd commented 2 years ago

Hi, The decision for extending the deadline to Oct. 1 is agreed upon by all organizing committee members, and it can't be further extended to ensure that the leaderboard won't be overfitted. We do realize that Oct. 1 is a national holiday in China, most of the committee members are Chinese, we are very sorry for the inconvenience. Best regards, Bingchen Zhao vtddggg @.> 于2022年9月30日周五 11:22写道： … Hi @DTennant https://github.com/DTennant , We have receive the delay of the end to Oct. 1, 2022. However, Oct. 1 is the National Day in China, which means most people may not have enough time working for the challenge. Can we ask for a further delay of ending? Thanks!! — Reply to this email directly, view it on GitHub <#32>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADY4MVOUV3S3PO4CK6SUARLWAZMIHANCNFSM6AAAAAAQZK6SHY . You are receiving this because you were mentioned.Message ID: @.>

I think the current leaderboard is grossly overfitted. Can you publish the best performance of the current leaderboard instead of hiding it.

DTennant commented 2 years ago

Hi,

Removing a score from the leaderboard is a feature built into the CodaLab server, and we cannot change or disable it, this feature can also help the participant remove submissions that violate the rules. Please be assured that we will manually check the top-5 submission to ensure the performance on the leaderboard is reproducible, so the competition will still be fair.

wujiekd @.***> 于2022年9月30日周五 19:50写道：

Hi, The decision for extending the deadline to Oct. 1 is agreed upon by all organizing committee members, and it can't be further extended to ensure that the leaderboard won't be overfitted. We do realize that Oct. 1 is a national holiday in China, most of the committee members are Chinese, we are very sorry for the inconvenience. Best regards, Bingchen Zhao vtddggg @.

> 于2022年9月30日周五 11:22写道： … <#m4778521410651212218> Hi @DTennant https://github.com/DTennant https://github.com/DTennant https://github.com/DTennant , We have receive the delay of the end to Oct. 1, 2022. However, Oct. 1 is the National Day in China, which means most people may not have enough time working for the challenge. Can we ask for a further delay of ending? Thanks!! — Reply to this email directly, view it on GitHub <#32 https://github.com/eccv22-ood-workshop/ROBIN-dataset/issues/32>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADY4MVOUV3S3PO4CK6SUARLWAZMIHANCNFSM6AAAAAAQZK6SHY https://github.com/notifications/unsubscribe-auth/ADY4MVOUV3S3PO4CK6SUARLWAZMIHANCNFSM6AAAAAAQZK6SHY . You are receiving this because you were mentioned.Message ID: @.>

I think the current leaderboard is grossly overfitted. Can you publish the best performance of the current leaderboard instead of hiding it.

— Reply to this email directly, view it on GitHub https://github.com/eccv22-ood-workshop/ROBIN-dataset/issues/32#issuecomment-1263473003, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADY4MVJCXYSFHOBKUJOIYNLWA3HXJANCNFSM6AAAAAAQZK6SHY . You are receiving this because you were mentioned.Message ID: @.***>

wujiekd commented 2 years ago

Hi, Removing a score from the leaderboard is a feature built into the CodaLab server, and we cannot change or disable it, this feature can also help the participant remove submissions that violate the rules. Please be assured that we will manually check the top-5 submission to ensure the performance on the leaderboard is reproducible, so the competition will still be fair. wujiekd @.> 于2022年9月30日周五 19:50写道： … Hi, The decision for extending the deadline to Oct. 1 is agreed upon by all organizing committee members, and it can't be further extended to ensure that the leaderboard won't be overfitted. We do realize that Oct. 1 is a national holiday in China, most of the committee members are Chinese, we are very sorry for the inconvenience. Best regards, Bingchen Zhao vtddggg @. > 于2022年9月30日周五 11:22写道： … <#m4778521410651212218> Hi @DTennant https://github.com/DTennant https://github.com/DTennant https://github.com/DTennant , We have receive the delay of the end to Oct. 1, 2022. However, Oct. 1 is the National Day in China, which means most people may not have enough time working for the challenge. Can we ask for a further delay of ending? Thanks!! — Reply to this email directly, view it on GitHub <#32 <#32>>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADY4MVOUV3S3PO4CK6SUARLWAZMIHANCNFSM6AAAAAAQZK6SHY https://github.com/notifications/unsubscribe-auth/ADY4MVOUV3S3PO4CK6SUARLWAZMIHANCNFSM6AAAAAAQZK6SHY . You are receiving this because you were mentioned.Message ID: @.> I think the current leaderboard is grossly overfitted. Can you publish the best performance of the current leaderboard instead of hiding it. — Reply to this email directly, view it on GitHub <#32 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADY4MVJCXYSFHOBKUJOIYNLWA3HXJANCNFSM6AAAAAAQZK6SHY . You are receiving this because you were mentioned.Message ID: @.>

I'm glad you got back to me. We got it. Thank you ~