srobo / competition-simulator

A simulator for Student Robotics Virtual Competitions
https://studentrobotics.org/docs/simulator/
MIT License
8 stars 2 forks source link

Validate running competition matches for SR2024 #410

Closed PeterJCLaw closed 4 months ago

PeterJCLaw commented 10 months ago

Once we reach the point where we're ready for SR2024 (i.e: https://github.com/srobo/competition-simulator/milestone/9) we should validate that the full stack of competition mode running matches works. Likely it will do as it's checked in CI, however in SR2023 there were a few things which we found during the pre-virtual-competition run-through, so it's good to do a manual shake-down.

PeterJCLaw commented 9 months ago

Here's the results from a test match I've recorded using the same dummy robot code in all zones. The simulation ran at between 0.8× and 1.3×, however it's notable that this robot doesn't us its sensors at all.

dancer.py ``` python from sr.robot3 import Robot def set_power(R: Robot, power: float) -> None: m0, m1 = R.motor_board.motors m0.power = power m1.power = power def set_rotate(R: Robot, power: float) -> None: m0, m1 = R.motor_board.motors m0.power = power m1.power = -power def dance(R: Robot) -> None: while True: set_rotate(R, 0.5) print('sleep', R.zone) R.sleep(1) set_power(R, 1) print('sleep', R.zone) R.sleep(1) set_rotate(R, -0.5) print('sleep', R.zone) R.sleep(1) set_power(R, -1) print('sleep', R.zone) R.sleep(1) dance(Robot()) ```

Recording at 1920×1080:

https://github.com/srobo/competition-simulator/assets/336212/22de951c-d31a-400b-9225-ae9986f3ff59

match.json (manually constructed) ``` json { "match_number": 13, "arena_id": "Simulator", "teams": { "ABC": {"zone": 0}, "DEF": {"zone": 1}, "GHI": {"zone": 2}, "JKL": {"zone": 3} }, "duration": 20, "recording_config": { "resolution": { "height": 1080, "width": 1920 }, "quality": 100 } } ```

Animations (note that these appear to suffer from the known lighting issues https://github.com/srobo/competition-simulator/issues/393, contrary to my prior comments there that the latest webots didn't reproduce that issue): match-13.zip

I've also run the same arena for a full duration and that seemed similar in terms of performance, though I haven't done a full video render for that.

PeterJCLaw commented 9 months ago

Here's a full match recording for the robot as above (dancer), a simpler robot which just spins on the spot, plus two instances of a robot which moves & runs vision (though doesn't do anything with the result). This recorded at a similar speed to the previous match, though was very slow for the first second or so.

simple-drive-and-see.py ``` python from sr.robot3 import Robot R = Robot() while True: print("I see {} marker(s):".format(len(R.camera.see()))) R.motor_board.m0.power = .5 R.motor_board.m1.power = .5 R.sleep(2) R.motor_board.m0.power = 0 R.motor_board.m1.power = 0 R.sleep(.7) print("I see {} marker(s):".format(len(R.camera.see()))) R.motor_board.m0.power = -.5 R.motor_board.m1.power = .5 R.sleep(0.3) ```

https://github.com/srobo/competition-simulator/assets/336212/15a74b3d-1d5e-4a88-b365-2b994eb7fd26

Animations: match-42.zip

PeterJCLaw commented 9 months ago

Based on this I'm confident that the simulator is ready to run a virtual competition this year, at least from a purely technical point of view. It'd be great to improve the animations and finesse the lighting that we didn't manage to improve last year, however IMO those are enhancements rather than blockers.

PeterJCLaw commented 9 months ago

Running four dancer.py robots in a simulation on Windows 10 (same hardware as above, but not recording nor running a match) I'm seeing the same simulation speed -- 0.8×-1.3×, averaging around 0.95×. This suggests that something other than OS is what dictates performance.

PeterJCLaw commented 4 months ago

We ran the friendlies a few weeks back, they were fine. We've also now run the virtual competition matches this evening, that was fine.