Open-EO / openeo-python-client

Python client API for OpenEO
https://open-eo.github.io/openeo-python-client/
Apache License 2.0
156 stars 42 forks source link

Job manager: trigger on_job_done for finished job from all active states #430

Closed StijnCaerts closed 1 year ago

StijnCaerts commented 1 year ago

Right now, the on_job_done() method will only be triggered when the cached state is running and the job status is finished. But it might be possible that the job manager was stopped and resumed later and jobs in another active state (created, queued) finished in the mean time. For example, this means that jobs going from queued to finished will not trigger the on_job_done() method.

This PR changes the behaviour of the job manager to trigger the on_job_done() method on the transition from any active state to the finished state.

soxofaan commented 1 year ago

won't this trigger on_job_done all the time once "finished" is reached?

maybe change detect reaching "finished" state with:

              if (
                    df.loc[i, "status"] != "finished"
                    and job_metadata["status"] == "finished"
                ):
StijnCaerts commented 1 year ago

It shouldn't make a difference, as the active jobs are filtered before iterating. Jobs already in the finished state will not be considered. https://github.com/Open-EO/openeo-python-client/blob/a1a2ec3f36eb755f8a10cde850c830b8cd7df86c/openeo/extra/job_management.py#L404-L412

soxofaan commented 1 year ago

thanks!