Idea: stay on player if gunshots or vehicles are heard

[adamdecaf]

Looking through the code relating staying/rotating to the next player I was wondering if it's possible to capture sounds and use those as feedback to stay on the current player.

• If there's sufficient movement and gunshots are heard stay on the player (increment slower, e.g. by 0.1)
• If there's movement and first person vehicle sounds are heard stay on the player (increment slower)

I don't think the bot currently captures sounds to process, but being able to watch a firefight or someone in an attack chopper for longer would be exciting.

[cetteup]

I recon it's possible to some degree. The biggest issue (apart from my complete lack of experience with audio) would probably be CPU/GPU usage. As it is, the spectator is pretty lightweight. The most CPU intensive thing is probably the constant OCR-ing of screenshots.

Focusing more on active/interesting players is absolutely something I want to move towards, but I fear audio processing is out of scope for now.

P.S.: I previously started building an image-classifier dataset to enable the bot to determine what's on screen. Idea being that it would stay longer on a helicopter and instantly skip prone snipers.

[adamdecaf]

I don't think there's enough changes on the screen to detect rifle blasts either. Enemies could be off screen as well.

[cetteup]

The image-classifier would have just looked for e.g. what vehicle/class is on screen. The assumption being that an attack helicopter is more interesting to watch (and should thus get more screen time) than a prone sniper.

For that purpose, one screenshot every few seconds would probably be enough. The current afk detection takes more big screenshots than that and fails miserably when looking at water due to the reflections, so this would be an improvement in that sense. But training the classifier would be a lot of work and running it would likely increase the amount of compute resources required to run the bot.