Budget handling behaviour

[soxofaan]

When creating a batch job, there is a field budget described as:

Maximum amount of costs the request is allowed to produce. The value MUST be specified in the currency of the back-end. No limits apply, if the value is null or the back-end has no currency set in GET /.

Some questions and discussion points:

• What if a backend does not check the budget?
  • E.g in the geopyspark backend implementation we currently do not check this budget field for example and blindly execute the job (as long as the user has credits). The main problem is that we can not reliable predict the cost of a job, so it's currently counter-productive to put hard blockers in place here.
  • Is not checking the budget an API violation?
  • should there be a backend capabilities entry about support for budget? E.g. In the python client we support the budget field when creating a job, but it's confusing when that is not being honored by the backend. It would be better if we can warn/error about that client-side.
• In the context of hard to predict costs, is it ok to cancel a job when at the time it goes over budget (assuming cost can be tracked real time)? This would practically mean that the job will fail (so no results) and the user will be charged with the cost? Or should the budget be checked before actual processing, so that the user does not risk being charged for something that is killed by the backend?

[soxofaan]

cc @HansVRP

[HansVRP]

Perhaps it would be better to limit the job runtime instead of the job cost?

We have seen that runtime is a pretty good indicator of job costs, however it scales differently for every process graph.

Usually with some initial trial and error you do have an idea of how long your job is allowed to last. When it runs longer it is either unoptimized, there is a scaling issue or a problem in fetching the information.

The required information is already available since we do track the job status...

[soxofaan]

Note that this ticket is about the existing "budget" field in the openEO API, which is strictly about currency/credits, so it's a bit out of scope to involve run time. Moreover, with run time you have the same problems as originally stated: how to predict it, and if it can not be predicted, is it ok to abort a job that exceeds a limit and still charge for it?

FYI related discussion about run time limits:

• https://github.com/Open-EO/openeo-api/issues/387

[m-mohr]

Related to #544