flatland-association / flatland-rl

The Flatland Framework is a multi-purpose environment to tackle problems around resilient resource allocation under uncertainty. It is designed to be a flexible and method agnostic to solve a wide range of problems in the field of operations research and reinforcement learning.
https://www.flatland-association.org/
MIT License
25 stars 8 forks source link

enable a redis container in github workflow and configure test_service.ipynb to use it #65

Open hagrid67 opened 11 months ago

hagrid67 commented 11 months ago

Objectives Re-enable test_service.ipynb in tox notebooks to test the evaluation service

Todos

Minimal testing requirements N/A - this change enables an existing test to run

Additional context checks.yml notebooks have recently been enabled by Jeremy / @hagrid67 in PR #61 @fnberta would you mind taking a look at enabling redis in the workflow please? (Or I'll take a look in a few days)

fnberta commented 11 months ago

Yes, I can look into it. But do we really want to require a redis instance to be running just to run the checks in our build pipeline?

hagrid67 commented 11 months ago

It seems that running Redis in github CI is well established.

I think we have plans to offer the evaluation service as a key feature of flatland, so that people can publish their benchmarks. We discussed at the last dev meeting starting work by the end of the year; Manuel is doing some planning. So I'd suggest that it's worth putting the effort into building some tests around it. The existing evaluator uses redis; this comes from the AICrowd architecture.

The notebook is one I built 2-3 years ago to run a simple test to start an evaluator service and then run a client against it. It "manually" starts and stops processes in the background so I'm sure it could be improved; eg we might want to run Redis in one container, and the evaluator in another, and then run the client in the foreground.

Until then, it seems fairly low effort to get this existing test working on github CI. It seems to run OK on my local Ubuntu 22.04 with redis running.