search

CityBrainChallenge / KDDCup2021-CityBrainChallenge-starter-kit

77 stars 40 forks source link

Some questions about the final stage #41

Open Enoch2090 opened 3 years ago

Enoch2090 commented 3 years ago

Some questions about the final stage:

  1. The contest homepage claimed "large-scale cloud computing platform is provided" in the final stage. Will CBEngine be accelerated in the final stage? It spends about 10 minutes to run a whole episode on our own platform, and triple the time on the official platform. It is extremely important to know the speed of the final benchmark platform as it determines the complexity of algorithm we choose.
  2. We know that the online evaluation flow data is unknown in the final stage, but will the vehicle flow be deterministic or random? i.e. will two submissions of the same agent have exactly the same outcome, or will they only have statistically close outcomes?
  3. Will the data structure and definition of actions, observations, rewards, infos, dones change in the final stage?
  4. Will the vehicles in next round have different type, like different speed limit/move strategies?
  5. According to the tutorial(API functions - Simulation Initialization), there is a thread_num parameter in the initialization of the engine. We've tested with this parameter, and either using the engine in the docker or outside the docker(see Enoch2090/city_brain_kit), this parameter just doesn't work. We are running the engine on AMD R7-5800X, so there's enough core to support multi-threading. htop command further shows that the engine can't fully occupy more than 1 core at the same time, pls fix this.
  6. If possible, please give a detailed list of what libraries you will provide in the next version of docker image, pip freeze > requirements.txt would be fine.
Kanstarry9T commented 3 years ago
  1. For final stage, we will provide a computing platform with 3000 CPU cores, participants can run multiple simulator in parallel;
  2. We will provide a set of traffic scenarios, and will evaluate your solution over all scenarios;
  3. only minor change to the current data structure, we will add more information to the current data;
  4. The vehicle behavior model will keep same, only the OD and volume will change;
  5. we will support multiple thread (thread_num greater > 1) for the third stage (final phase).
namdre commented 3 years ago

@Kanstarry9T

  1. Will the new scenarios include new road networks?
  2. Will we be able to run some or all the new scenarios locally? If yes, where can we obtain them?
Kanstarry9T commented 3 years ago
  1. As for new scenarios, we will use the same road network that is used in qualification phase (but with some minor modifications)
  2. In the final phase, you are encouraged to create your own sample traffic data, you can find the traffic_generator.py script in the /data folder of starter-kit that will be released sooner.
behrisch commented 3 years ago

Concerning 3. I am now missing the route and t_ff information for the vehicles. Is it possible to re-enable them?

Kanstarry9T commented 3 years ago

The route and free-flow travel time (total travel time at speed limit for the full trip), t_ff information are removed intentionally to add more challenges in the final phase, which are usually not accessible in real world traffic.

behrisch commented 3 years ago

But it contradicts your statement above that there will be no significant changes and you will add more info

Kanstarry9T commented 3 years ago

By "add more information", we mean that we will add more sample definition (data structure) of observations (i.e., traffic states representation), sorry for the misleading.