Closed Uriopass closed 3 years ago
skairunner
commented
3 years ago As far as I can tell, these steps would be required:
(1) Ability to generate and load test scenarios (w/map, vehicle, and success conditions) (2) Framework to headlessly start & run a scenario and check for completion (3) Adding this as part of the CI/CD
Uriopass
commented
3 years ago I agree on your points except for CD (3), this is merely an indicator of how well the car AI performs but shouldn't be required for CI to pass. Random might be used for car AI which would lead to flaky tests, also car AI is floating-point heavy which leads to further reproducibility problems. (note: I tried once to have reproducible runs for easier debugging but it was so hard I gave up.)
However, this applies to complicated tests (lots of cars doing hard interactions). I agree that some simple tests (<= 2 cars, simple goals) could be used to test for regressions.
About scenarios (1), I'm happy I got Lua working and I think it could be a cool application, and implementing this would further extend the modding possibilities which are non-existent at the moment.
(2) shouldn't be too hard either since the core engine egregoria is separated from the render module wgpu-renderer so having a headless client using the mods module for running scenarios shouldn't be too hard.
skairunner
commented
3 years ago We don't need to make it necessarily fail the build, but could make an informational webhook kinda like some implementations of coverage
On Mon, Aug 10, 2020, 16:12 Pâris DOUADY notifications@github.com wrote:
I agree on your points except for CD (3), this is merely an indicator of how well the car AI performs but shouldn't be required for CI to pass. Random might be used for car AI which would lead to flaky tests, also car AI is floating-point heavy which leads to further reproducibility problems.
However, this applies to complicated tests (lots of cars doing hard interactions). I agree that some simple tests (<= 2 cars, simple goals) could be used to test for regressions.
About scenarios (1), I'm happy I got Lua working and I think it could be a cool application, and would further extend the modding possibilities.
(2) shouldn't be too hard either since the core engine egregoria is separated from the render module wgpu-renderer so having a headless client using the mods module for running scenarios shouldn't be too hard.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/Uriopass/Egregoria/issues/19#issuecomment-671379639, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA22VOZVHN5C35JKFAU4B33R7755XANCNFSM4PWOHGIA .
Uriopass
commented
3 years ago Ah ! Interesting, I'm not sure how this information could be presented but then I definitely agree.
Uriopass
commented
3 years ago Got some work done, a basic headless runner, a scenario player in wgpu-renderer and some lua bindings for the world to modify it. See the test script here
Uriopass
commented
3 years ago Got scenarios up and working, even in the wgpu-renderer accessible from the GUI. Not entirely complete but closing. (Not running on CI yet)
When trying to enhance the car AI, I wish there was some sort of automated "car scenarios" test where I could check if a) No collisions happened b) No gridlocks happened c) How fast they went to their objective
For example, two cars in front of each other wanting to go forward, this is tough and cannot be resolved at the moment but maybe someday they could both chose to go right to avoid each other.
Another example is two cars going perpendicular to each other, where one must stop to let the other pass, this works at the moment but if I had an automated test to prove it it would be great.