Display Nr of waiting/ready jobs to execute in the experiment summary or experiment treeview/graph

[kinow]

In GitLab by @jberlin on Feb 9, 2023, 15:14

Based on what was discussed in issue 893 we need a visual hint to indicate the current the maximum concurrent jobs setting defined for the experiments and indicate how many are currently waiting, a sample message proposed by Dani to indicate could be:

"Some ready jobs from platform:{platform_affected} are not being submitted due to the experiment total_jobs and max_waiting_jobs constraints, they will be submitted once current jobs finish their execution"

We need to find a proper place in the GUI to indicate this (a label in the treeview or the graph)

@mcastril @dbeltran @mgimenez

[kinow]

In GitLab by @mcastril on Feb 10, 2023, 21:22

Independently of the message what we decided was to show an indicator pointing always the number of active and total tasks.

[kinow]

In GitLab by @bdepaula on Feb 27, 2023, 08:40

In case we need some inspiration for this, here's the Cylc UI (Vue.js) showing some of the tree & graph view elements.

And about the statuses, it would be interesting to do a deeper comparison of task/job statuses used in AS, ecFlow, and Cylc. Tasks in Cylc have the statuses “waiting” (waiting on prerequisites) and “preparing” (job being prepared for submission) (see other task statuses here. Not sure if AS & ecFlow have the same statuses.

There are also task modifiers to show that something could not start due to the queue limits:

[kinow]

In GitLab by @jberlin on Feb 27, 2023, 09:10

Hi @bdepaula , nice explanation, this is totally new in this version or there were before an UI ? this new version allows the user to manipulate the experiment such as change statuses of jobs, edit configs, etc ? thanks!

[kinow]

In GitLab by @bdepaula on Feb 27, 2023, 10:02

Hi @jberlin

this is totally new in this version or there were before an UI ?

The Vue.js UI and controlling the workflow via the UI are features available in Cylc since mid 2021, I believe.

I started working on the Vue.js UI for Cylc back by the end of 2018, and we had a working prototype in April 2019. The 8.0a1 (alpha1) was released in 2019, the first beta 2 years later, in Q1 2021.

At that time you could already manipulate experiments/workflows and change statuses, start, stop, pause, via mutations in GraphQL. Since then they have been adding more features to the UI and I think they are now preparing 8.1 (or it could have already been released, I think).

[kinow]

In GitLab by @mcastril on Mar 1, 2023, 12:26

@bdepaula I think our READY would be equivalent to preparing. We also have HELD and PREPARED (READY to be HELD).

Not sure about the purpose of Runahead.

[kinow]

In GitLab by @mcastril on Mar 1, 2023, 12:27

In this case we aim for a simple status bar showing the global status and the limits.

[kinow]

In GitLab by @bdepaula on Mar 6, 2023, 16:44

I couldn't find a list or table with the possible statuses for a job in Autosubmit. Do you know if we have one in our docs, @dbeltran ?

@mcastril I tried mapping the statuses among AS, cylc, and ecFlow. I will review it some other day when I have spare time (need to dig into the code/docs to confirm if queued means the same in both tools).

Docs:

• Cylc: https://cylc.github.io/cylc-doc/stable/html/user-guide/running-workflows/task-job-states.html#task-job-states
• ecFlow: https://ecflow.readthedocs.io/en/latest/ug/user_manual/running_ecflow/status_of_a_task.html
• Autosubmit: https://earth.bsc.es/gitlab/es/autosubmit/-/blob/master/autosubmit/job/job_common.py#L27-39 (source code)

cylc	ecFlow	autosubmit
status	status	status
	unknown	unknown
waiting		waiting
preparing
		prepared
submitted	submitted	submitted
		queueing
queued	queued
running	active	running
succeeded	complete	completed
failed	aborted	failed
submit-failed
		skipped
		delayed
expired
held	suspended	suspended
runahead

(I will move this to another issue in Autosubmit: https://earth.bsc.es/gitlab/es/autosubmit/-/issues/966)

[kinow]

In GitLab by @mcastril on Mar 24, 2023, 13:04

Thanks for this @bdepaula . I don't think we have this documented, but @dbeltran can clarify it.

• Preparing in Cylc would match to READY in Autosubmit.
• I think QUEUING in Autosubmit would match queued in Cylc and ecFlow.
• Is it HELD a job hold (queued but hold in SLURM?). In that case, it would be equivalent to HELD in Autosubmit