CLI: support flex-model and flex-context with our new dynamic Schema

[nhoening]

Discussed in https://github.com/FlexMeasures/flexmeasures/discussions/916

^{Originally posted by **kyros32** November 29, 2023} Hello Flexers, **1) I was wondering, how can I use SoC data --soc-at-start from a sensor?** **Instead of this:** ``` flexmeasures add schedule for-storage --sensor-id 2 --consumption-price-sensor 1 \ --start ${TOMORROW}T07:00+01:00 --duration PT12H \ --soc-at-start 50% --roundtrip-efficiency 90% ``` Something like this, so SoC is pulled dynamically everytime I force rescheduling via CRON for example or API call. ``` flexmeasures add schedule for-storage --sensor-id 2 --consumption-price-sensor 1 \ --start ${TOMORROW}T07:00+01:00 --duration PT12H \ --soc-at-start sensor-id 14 --roundtrip-efficiency 90% ``` I could fill it as a variable for in a call, but I think, why not to use an associated sensor with it. More simplex... Any help much appreciated. Robert

[nhoening]

I believe after #897, we'd pass a sensor on the CLI like this: --soc-at-start sensor:14. The Marshmallow schema might need a little adaptation, but that is the gist. We might support this in the flex-model as well, btw, and that would be where our main attention goes for this.

[nhoening]

I'm not sure, but I don't think #897 worked on the CLI. So this is where the focus of this issue would probably lie, see my last comment.

We could check that the CLI supports this format for all flex-model attributes which support passing in a sensor.

[Ahmad-Wahid]

@victorgarcia98, I believe you wanted to store soc-at-start for the sensor in the attributes, right ?

[victorgarcia98]

I think we need to narrow a bit the scope of the issue. Perhaps, it would be nice to have a 15min-30min session with @Flix6x .

Currently, we are only using a power sensor for scheduling a storage device but, in the near future, on the transition of scheduling on the Asset instead of a sensor, we could include a SOC sensor as well. This way, the scheduler itself could save the planned SOC values and the users could save the realized SOC values in a different source, similar to what we do for the power sensor.

Nonetheless, this requires quite a lot of changes and we could achieve a similar functionality by handling this at the command level. In the case of the API, users should be able to fetch the latest values and build the flex-model with that.

[nhoening]

We worked on more expression in the flex-model recently, here. That is a great basis.

Now, this issue should focus on letting users express the flex-model and flex-context in the CLI as well.

Sadly, passing JSON on CLI is not nice, so we would support all the attributes one-by-one verbatim, as we have started (e.g. --soc-at-start is a CLI argument.

But their values should be strings supported by the new schemas in the PR above. The CLI command should basically allow all the fields in flex-context and flex-model, built them into a dict and let the Schema validate the user input.

I think it is not really hard.

[Flix6x]

I'd still want to argue for a JSON solution here:

1. I find it too cumbersome to support these parameters one by one.
2. Explicitly modelling every flex-model/flex-context field as a CLI option goes against the idea of defining and registering your own schedulers, which may work with their own flex-model/context schemas.
3. We already use JSON representations in the API and, soon, in the DB (similar to how we store reporter configurations under data source attributes), so this could streamline the dev/user experience.

In cli.data_add.add_report we already have logic to parse JSON (and YAML) from file and even to allow editing on the fly. In add_sensor we also have logic to parse JSON passed as a string, which facilitates CLI usage in scripts (bypassing local file storage). We might even streamline this experience by defining our own JSONField that accepts a bare JSON string or a file path (editing would remain toggled by a separate flag).