Add a microgrid (API client) mocking tool for users to easily test actors and apps

[leandro-lucarella-frequenz]

What's needed?

We need to be able to test examples in the examples directory as part of the CI so we don't break them accidentally (or forget to update them when there are API changes), and allow users to do the same for their apps and actors.

Proposed solution

Create a microgrid mock, or a microgrid API client mock.

The simplest first version should just run an example for few seconds and check if there it didn't crash.

Use cases

No response

Alternatives and workarounds

Test using a real microgrid API, but that would be overkill.

Additional context

No response

[jakub-toptal]

@leandro-lucarella-frequenz got this script to do the job locally:

#!/bin/bash
python examples/sdk_usage_example.py &
pid=$!
sleep 5

if ps -p $pid > /dev/null
then
   echo "SDK Usage example running properly"
   kill -9 $pid
   exit 1
else
   echo "SDK Usage example exited unexpectedly"
   exit 0
fi

It runs the sdk example and checks if it is still running after 5 seconds. If so, the test passes, otherwise it fails.

What do you think? I could add an additional step after name: run tox in .github/workflows/ci.yaml.

[leandro-lucarella-frequenz]

But how can this work if it can't connect to the microgrid API?

[jakub-toptal]

It would connect to the sandbox microgrid, which is currently the case for sdk_usage_example and this would detect if the example works or not. This assumes that the test would always be able to connect to the sandbox microgrid.

But if we want to completely mock the microgrid then yes, this approach will not work and it would require more work on the python side.

[leandro-lucarella-frequenz]

Yes, this issue is about the mock. The sandbox won't be available in the CI.

[jakub-toptal]

@leandro-lucarella-frequenz I realized that we already have some Microgrid API mock: https://github.com/frequenz-floss/frequenz-sdk-python/blob/v0.x.x/tests/test_microgrid/mock_api.py that is being used in some tests.

It seems that run the examples against that. The question is how this should be structured in the CI. One way to do that could be to:

• modify the current example to read Microgrid HOST and PORT from environment variables instead
• add a separate script that would setup the Microgrid API mock, generate some random data and expose it on some localhost and some PORT
• add step(s) in the CI that for each example (or perhaps only once for all examples) would start the Microgrid API mock and then run all examples using that

[leandro-lucarella-frequenz]

Aha, interesting. A few observations:

• I think we should provide a higher level mock for running the examples, we should mock the whole MicrogridApi and pass use MicrogridApiMock in the examples instead of the regular MicrogridApi.
• It should be provided in frequenz.sdk.test instead of in the tests directory, as users should be able to use it too.
• Internally (in MicrogridApiMock) we could still use a mocked grpc service, the point of doing it at that level would be to let users easily choose between the real and the mocked object in tests/examples.
• With this approach we don't need to care about host/port, not setting up any scripts.

[llucax]

I guess this one is done, this is the MockMicrogrid? @shsms

It might need some cleanup and maybe exposing it in another way, but the class is there I guess.

[shsms]

I guess it can be turned into a proper mock that listens on a port, etc, if that's necessary. Getting the sandbox ready might be a better option though.

[llucax]

No, no, the point of this was to not listen to any port, but just be a real mock (we already had that and it is very problematic, because we get errors about ports being already used and we can't run tests in parallel).

But this is about the API, true, but there is also the MockMicrogridClient, which is doing something similar. I guess it is at a different level, as it is not mocking the API itself, but the client, but I think that was the intention of this issue. I guess I didn't understand very well the nomenclature when I wrote this issue.

I mean, the end goal of this issue is for SDK users to be able to test actors or apps without the need to connect to a proper API (sandbox or not), we could do that at different levels (microgrid object, like MockMicrogrid, or the microgrid API client).

In any way, maybe we should leave it open because right now is all internal and not exposed to the user, so there is still work to be done to be made generally usable.