search

CityBrainChallenge / KDDCup2021-CityBrainChallenge-starter-kit

75 stars 40 forks source link

Why do we need '--metric period =200' for evaluation? #32

Open HaoZhouGT opened 3 years ago

HaoZhouGT commented 3 years ago

Hi can you guys explain the setting of --metric_period 200 for evaluation? Does it mean we don't care the delay or throughput of the network after 2000s? Why not using --metric_period 359 for the whole period?

Kanstarry9T commented 3 years ago

--metric_period=200 means that we currently evaluate your solution every 200 seconds for scoring. You can set this parameter for your own needs.

HaoZhouGT commented 3 years ago

--metric_period=200 means that we currently evaluate your solution every 200 seconds for scoring. You can set this parameter for your own needs.

Hi Kan, do you imply we can change this setting? I assume this is not modifiable. Also I found only by adding --metric_period=200 we can get the same result as the server does.

Kanstarry9T commented 3 years ago

Yes, you can modify --metric_period, for example, --metric_period=100 means that evaluate your solution locally every 100 second, but please be notified currently, the scoring system evaluate your solution every 200 seconds. Also if you have any further questions, please join our Q&A session 06/02/2021 9:00PM (Beijing time) or 9:00 AM (Atlanta, EDT) https://bluejeans.com/895107273/0765