search

vantage6 / v6-on-kubernetes-PoC

2 stars 0 forks source link

Reconciling run_id and task_id concept with the K8S-based node architecture #19

Closed hcadavid closed 1 month ago

hcadavid commented 3 months ago

The K8S-v6-node is not reporting the status back to the server properly (after a failure or a completed task), as the concepts of job_id and run_id still need to be reconciled with the identifiers we are using for the PODs. This can probably be fixed by adding the missing identifier into the job/POD metadata so it can retrieved when getting the results, but I'm still a bit confused with these concepts: Task, Job (used in V6 and k8s), and Run :)

https://github.com/vantage6/v6-on-kubernetes-PoC/blob/dcddc6567cab2f188bc255fb46ffef40352736e7/integration_poc/container_manager.py#L738-L745

@bartvanb and @frankcorneliusmartin, if you have the chance to take a look at the 'integration poc' could you give us a hint on this?

hcadavid commented 3 months ago

Update: the job PODs metadata now includes both IDS:

https://github.com/vantage6/v6-on-kubernetes-PoC/blob/770a741ac1e70cca539befe0351b8f59f2480267/integration_poc/container_manager.py#L233-L267

However, the identifiers on the node are not consistent with the ones on the server. For example, when the node sends an update with run_id=7 and job_id=7, the server logs reports run_id=7 and job_id=1:

2024-07-26 14:44:01 - websockets     - INFO     - A container for job_id=1 and run_id=7 in collaboration_id=1 on node_id=3 has a new status=completed.