CARLA as a car simulator for humans and AI?

[fbuchinger]

Dear CARLA-Team, I discovered your project this weekend and I'm really impressed by CARLAs status and feature set. For me it's a pity that it will "only" be used by researchers in the autonomous driving area.

Have you thought about turning CARLA into general purpose car simulator that can be used to improve the driving skills of humans AND autonomous driving vehicles? For me, this could be a win-win situation for both groups:

• Humans (e.g. driving school students) can profit from the simulation of dangerous situations (arbitrary crossing of pedestrians)
• the AI in CARLA could be improved by learning from human player data (e.g. how would an unexperienced or very experienced driver behave in a certain situation?), allowing for more realistic simulations.

As you mention in your paper, there are already a few commercial car simulator games out there (I'm surprised that you didn't mention City Car Driving ;-) But they suffer from limited customizsation possibilities (e.g. no new towns/missions) and a lack of sensor data, which is important for autonomous driving engines.

In turn, I believe that an "AI-only" car simulator would also suffer from the lacking human feedback. Its realism might deterioriate over time if there isn't a constant influx of human player data.

Luckily, there aren't so many features missing for turning CARLA into a "human playable" car simulator:

• enable Joystick/steering wheel support in CARLA Server (poor performance when using python client)
• cleanup HUD display to make car control for human beings easier (e.g. hide pure debugging-related metrics like fps and coordinates).
• define some basic missions that human players have to achieve
• introduce a score system that punishes the player when he commited a traffic rule violation, define game-over-scenarios (e.g. car crash)
• ability to record and upload driving data from human players

How well would that align with the project roadmap?

[felipecode]

Hello @fbuchinger Thanks a lot for your interest in CARLA and giving this super nice feedback text to us. I think your ideas are great and to some extent aligned with the road-map. But, unfortunately, probably not in a near future.

You are right, currently CARLA has poor support for human players. By adding the possibility for humans also to play in a "gamified way", we could eventually engage more the community. We could even add the posibility for humans to play and contribute to CARLA in form of data. Also, In the future, is for sure a interesting thing to compare machine learning with humans in our benchmarks. However, this is something we unfortunately don't have time to work now. Luckily, I think this is something the community could easily engage since it does not seem to require any core change in CARLA system. There already some people working on that. @sandman on issue #88 for instance.

Again, thanks a lot for your post.