Reuse of MLFLOW_TRACKING_URI

[PaulNdrei]

Is it possible to set MLFLOW_TRACKING_URI at Magma actions variables and then use it in the templates/repos that call Magma reusable workflow?

[ankush13r]

Don´t know if it's possible, but we can try it. Do you think if in our group everyone is gonna use the same endpoint?

[ankush13r]

ChatGpt Answer: Variable Precedence

When the same variable is set in both repository A and B, the following precedence rules apply:

Inputs Passed During Call: Inputs provided when calling the reusable workflow from repository B will override default values set in repository A.
Secrets Passed During Call: Secrets passed from repository B will override any secrets or environment variables defined in repository A.
Environment Variables: If the same environment variable is set in both repositories, the value from repository B (caller) will take precedence.

To summarize:

Inputs and secrets passed from the caller workflow in repository B override those defined in the reusable workflow in repository A.
Environment variables set in the caller workflow (repository B) override those set in the reusable workflow (repository A).

If chatgpt is not wrong so it's possible and it's a great idea!

[PaulNdrei]

It's also interesting to do it with the other "static" variables, and it wouldn't be necessary to force the users to add those variables to the vars section.

Vars:

GPFS_DATA_REGISTRY_PATH GPFS_MODELS_REGISTRY_PATH GPFS_SINGULARITY_IMAGE_REGISTRY_PATH and of course MLFLOW_TRACKING_SERVER_URL

Let's keep this issue open. In the meantime, we have decided when we are going to work on it.

[PaulNdrei]

We have to reconsider this because if we want this to be used by other groups or externals, we cannot set base variables in this repository.

[ankush13r]

I've tried this and didn't work for me. When you call a reusable workflow it only calls the workflow without using env variables from the repository of reusable workflow.

[PaulNdrei]

Ok, then, a solution to avoid researchers manually adding this static variable could be synchronizing the repository vars once a researcher creates a new repository. Maybe would probably be possible to achieve via Webhooks and https://github.com/jpoehnelt/secrets-sync-action because of the lack of Enterprise Edition.

[PaulNdrei]

A workaround could be adding the variables to a file (values encrypted with base64), one vars file per bsc group.

We will decide in the short-medium term if we will work on this issue.

Note: This only is suitable for variables not for secrets.