ETL: support organization id based reporting

[soxofaan]

Discussed with @JanssenBrm: we want to introduce an "organization" concept in the ETL API resource reporting, so that users that are part of multiple organizations/projects can explicitly specify the desired organization that resource usage should be reported under.

• user specifies organization_id as job option (well known feature of batch jobs, also possible for sync jobs) during job creation time
• organisation_id should be included in ETL resource reporting request (not supported yet)
• ideally, organization_id should be checked for validity at job creation time. ETL endpoint to be added

[jdries]

and why not using the 'plan' option for this? That's part of the openEO API, so easier to standardize?

[soxofaan]

and why not using the 'plan' option for this?

In our discussion, @JanssenBrm mentioned also another thing on the roadmap that feels more suited for the "plan" field: distinction between different kind of credits: free, commercial, ..., . The "organization" concept matches a lot worse with how "plan" is currently described in the spec, e.g. orgs will be specific per user, while plans are expected to be declared globally in the capabilities doc.

[soxofaan]

this ticket is currently blocked: this feature first has to be implemented in ETL API : https://jira.vito.be/browse/MKTP-379

[rahmandawibowo-vito]

Hi @soxofaan, I'm picking up the ETL-API implementation for this ticket https://jira.vito.be/browse/MKTP-379. I'm just want to confirm the changes made on ETL-API to make this working:

1. The OpenEO needs to check if the job can be executed based on org ID using the new endpoint (link https://etl-dev.terrascope.be/docs/#/organizations/OrganizationsController_getOrganizationPermissions).
2. If the OpenEO sends an org ID on the reporting request (sync job or batch job), the ETL-API will try to deduct credits based on the org ID (even though user ID is also present on the payload). The deduction will be a success only if the user is the member of the org. The org ID should be the same with the one on the 1st point.

This also implies that whenever the organization based reporting is being used by OpenEO, these two points needs to be strictly followed to ensure a correct flow. This is because mixing up between the user and organization permission request can lead to unintended effect (e.g. the job can be executed on VITO org, but when deducting credits will be using other org that doesn't have sufficient credits).

Do you perhaps have some comments on this?