cylc / cylc-admin

Project planning for the Cylc Workflow Engine.

https://cylc.github.io/cylc-admin/

GNU General Public License v3.0

5 stars 13 forks source link

2021 Cylc Meetings #139

Closed hjoliver closed 2 years ago

hjoliver commented 2 years ago

Jan 2021

Cylc 7.8.7/7.9.2 released

deployed at MO and NIWA, no probs

Event-driven datastore PR: almost done

rose suite-run migration:

cylc install, reinstall, run1/2/3 dirs, rose-suite.conf plugin
cylc clean
done by end of Jan, if not before

editable mutations
- only thing missing is testing
delta-driven gscan
- need to check workflow "unregister" comes through as a delta
- DS: workflow status flicker hopefully fixed by event-driven PR
- if WF started from UI, need to trigger an immediate scan (config 5 sec delay at the moment)
message triggers (almost done?)
- choose ellipsis demo, with mods
hierarchical gscan
- a bit longer, needs deltas
- (needs more to the datastream? DS)

TODO for cylc-8.0b0 (by end of Feb):

finish existing PRs

UIS

minimal mutation-driven services: start stopped flows
(authorization, if time)

Scheduler

finish existing PRs
finish task state rationalization: queued as xtrigger (plus some trivial renaming?)
- queued as xtrigger
- (need to consider behaviour of multiple queues ... global queues)
- (ditch the trivial changes)
(delta-driven TUI? - nope; beta-1+)

Docs?

HO to consider minimal requirements for beta?

Other:

JR has time on Cylc 8 now :tada:

hjoliver commented 2 years ago

Feb 2021

Cylc 7?

no problems?

Cylc 8

Admin

applying for project funding extension next FY
possible 10 week internship at NIWA (CS Masters student)
other?

Done, or as good as, since last meeting

editable mutations
- just tests to do
cylc install, cylc reinstall, cylc play, cylc clean ( and rose-suite.conf plugin)
event-driven datastore
removed obsolete queued task state (and centralize queue logic)
- revisit sorted queues etc. before 8.0
other?

Outstanding problems and issues

display task output messages (reactivity?) https://github.com/cylc/cylc-ui/pull/530
- added delta contains no outputs, so no initial object to subsequently be reactive
deltas in gscan https://github.com/cylc/cylc-ui/pull/543
hierarchical gscan (blocked by item above) https://github.com/cylc/cylc-ui/pull/502
UIS: start stopped flows
- DS to look at
rose-suite-run plugin slightly broken on re-install (TP on it)
intelligent host select not working (needs fixing soon, but post-beta is fine)
5 most recent failed tasks gscan tooltip:
- scheduler datastore to maintain a separate FIFO list/queue to update on submit-failed or failed events - DS
other?

TODO beta release

(finish all above)
Docs:
- "What's new in Cylc 8"?
- What's still missing, to be done by 8.0
- Exactly how can the beta can be used (e.g. remote spawning not well tested yet?)
other? ("execute user scripts in a subshell"??)

Next meeting

hash out priorities and assign issues for 8.0, and for beta-1

hjoliver commented 2 years ago

March

Main topic: Cylc 8.0b0 TODO list

(updated)

Also:

what exactly should gscan (and its tooltips) display? (in Cylc 7 it is a summary of the current scheduler task pool, which makes less sense for Cylc 8)
- show n=0 (active + unhandled failed) in icon list, 5-most-recent in tooltips
should we allow cylc play --re-run? (without cylc clean first ... probably not)
- agreed not, stick with the plan to extend cylc clean to prep for in-place re-run
confirm what to do if the UI misses deltas or receives bad deltas
- agreed: it should not happen, but if it does start from scratch is required. Freeze the UI, alert the user, and advise to reload the browser
queued state indicators in the UI occasionally not removed when the task is de-queued?
- need to go back and investigate, it seemed to be a back-end problem
user scripts in a subshell
- DM has approved, let's get it in for beta if possible
gscan deltas probs:
- UIS sometimes sends out-of-order deltas that can't be applied, leaving incorrect state ("missing family node"...)
- delta representation of non-running workflow transitions (registered, deleted)
- OS noticed some data incorrectly appearing in the "added" deltas?

"Question" labeled issues:

https://github.com/cylc/cylc-flow/issues/3763
- keep processed config in the log dir, not in the top level
https://github.com/cylc/cylc-flow/issues/4112
- comms timeout: not critical to change; let's see what info we get if there's a network issue; OS: could look a proper TCP timeout and relay ping-like info to the user
https://github.com/cylc/cylc-flow/issues/4108
- global Jinja2 config: agreed option 2
https://github.com/cylc/cylc-flow/issues/4039
- (didn't discuss)

Beta release timeline:

end of March at latest, frequent updates thereafter
get delta gscan in if possible (decide by early next week)

hjoliver commented 2 years ago

April

Cylc 7

Problems?

Cylc 8

UI

GScan Deltas https://github.com/cylc/cylc-ui/pull/543
- pending workflow rename/move/remove GraphQL delta
- (and viewing source workflows?)
- gscan performance: wait for deltas to be in
- and erroneous workflow added deltas
Hierarchical GScan https://github.com/cylc/cylc-ui/pull/502
- pending GScan deltas :point_up:
Table view status https://github.com/cylc/cylc-ui/issues/631

Packaging

proposal: meta-package https://github.com/cylc/cylc-flow/pull/4158
- use conda environment.yml files (examples of different recipes for different site needs: spawners, authenticators etc.)
- may need to rename cylc-flow back to cylc, but think through the implications! (before 8.0!!)
- (conda outputs PR done)
close Packaging end-game issue? https://github.com/cylc/cylc-admin/issues/27

Documentation

things that don't work as expected, e.g. cylc hold
- 2-column comparison of how to do X in Cylc 7
terminology: time to completely excise "suite"
see also glossary (OS) on cylc-doc

8.0b1 release?

content: minimal authorisation, ...?
- (cylc install; logging of shutdown)

Priorities pre-8.0.0:
1. bug fixes
2. deployment testing
3. cross-user and authorization - fine-grained via graphql middle-ware; in-progress MH
4. performance testing
5. basic graph and static vis
6. edit-run (CLI and UI) - prob broadcast
7. documentation
8. task hold!

Other

GH actions DNS problems
- 2 different DNS configs for GHA runners, only one is compat with cylc! (socket.gethostbyname()??) - (also problem with MacOS - the way that python socket lib binds to the BSD system socket module) ... can we stop using that, post platforms - OS to take a look.

Next meeting

1st Wednesday of May?

hjoliver commented 2 years ago

May

Cylc 8 beta deployment delays at our sites

NIWA:
- PAM: "In order to use PAM you need root privileges to configure the PAM module you’re targeting to login and we’d need to modify images and automation to make the changes persist." (Or ... use the existing JupyterHub instance on the system)
- Integration with existing JupyterHub instance: "I think we can set up multiple spawners with https://github.com/jupyterhub/wrapspawner there’s an issue about specifying cmd, but no examples."
Met Office:
- can't use conda (glibc old)
- installing on Monsoon (same platform, internet access)
- quite reliant on pre-compiled binary versions of some libraries
- post to Discourse
- (manylinux1 check is in now, with Python 3.7)
- OS: distributed spawner - UIS is currently assuming hub is on localhost?
- DM: want to switch on J-Hub internal SSL, needs support in UIS?
(BOM?)

Work in progress

authorization (MH): anything we need to discuss here?
- read, modify, execute? (modify is dangerous)
task hold
- (trigger etc. is ok for ghost tasks)
- (support globs because it's more general than hold)
- (what about trigger foo.*??)
- (solution: globs ONLY apply to n=0)
runhead pool -> runhead-limited indicator flag
- (consider representing as a queue - cycle point priority queue)
- (but: user sets it with "runahead limit")
xtriggered indicator flag, as interim measure? (before https://github.com/cylc/cylc-ui/issues/331)
- (multiple flags?)
- (maybe we can go straight to the full solution? - xtriggers in schema, including status? - DS to follow-up)
intelligent host select (TP)
- approaching POC (a week ... or more?)
"expanded workflow status deltas": looks like DS has nailed this, but we need to review it https://github.com/cylc/cylc-uiserver/pull/212
- OS: doesn't deal with the added deltas yet (DS: when workflow goes from running to stopped; or installed, or deleted; OS: for testing, to see the generated deltas, can we fake the hub?)
- DS to look at the seemingly spurious workflow status added deltas showing up
- UI: 9 PRs in progress, most are fairly simple, except for gscan deltas & hierarchical
UIS: load balanced spawner (OS)

TODO

graphql subscriptions at the scheduler (DS ready to do it)
(major items reminder: https://github.com/cylc/cylc-admin/wiki/Cylc-8.0-TODO-List)
thoughts on prioritization from here on?

Other

(Check on scheduler reload, we get a new "initial data burst" - check for the reload flag in the delta)
Altair update

hjoliver commented 2 years ago

June

Sites

any new Cylc 8 issues?
- possible gscan-caused UI "stability issues", waiting for gscan deltas
- tasks appearing beyond end of workflow
- DM concerns about partially-satisfied prereqs
- ignore pre-initial for start-tasks

Quick summaries of what we've each been working on, main issues and problems, etc.

authorization
- can we resolve remaining questions now?
  - access group names and mapping to commands and queries
  - agreed on CONTROL name
  - agreed to NOT allow per-workflow permissions
  - JR: timed-out privileges would be desirable
  - BK: J-hub has concept of inactivity (kill on inactivity)
  - MO Ops: too easy to hit the pause button
  - behaviour if config file permission are too loose
  - implementation options A, B, or C??
  - Option B: middleware! ... just config to go; what to do in the UI for unauthorised actions; tidying, maybe by end of month
  - DM: less essential than it was because of J-Server

gscan deltas
- up for review now: OS and HO to do
merging subscriptions
- issue going up on combining fragments

workflow status deltas: done?
multiple subscriptions per request issue: done?
...

intelligent host select
- PR up, failing on some GH tests, not failing locally, very nearly there (needed for 8.0, otherwise degraded from rose host-select)
cylc review status
- suite to workflow, nearly ready to go in
- one outstanding issue, but it's bodgeable

load balanced spawner done! but UIS can't use because it has to be on the hub host! .. which will be achieved by jupyter server
jupyter server run Jupyter lab and Cylc UIS under J-server, with exist auth ... couple of PRs to J-server. Launch UIS without going via the Hub (and opens browser tab)... for standalone (hub-less) use! ... J-S used to be part of Jupyter lab; solve issues like support for internal SSL, because J-S supports that; only catch is you don't get multi-user for this
universal ID
- RD to look at centralising parsing of the workflow arg, as a prereq to this

(ICM Cylc support?)
...

task hold -done - awaiting final review
targeted clean
- safety if user symlinks present, TBD

done: remove runahead pool, expose runahead limiting (c.f. queued, held; note not done with xtriggered)
done: start from specified tasks (SoD)
some smaller stuff e.g.:
- done: fix manual triggering of queued tasks (SoD)
- done: inherited parameter env fix
- NIWA end of FY Cylc and UMP project reporting and start of FY proposals :grimacing:

8.0.0 release

end of June?
- DM: end of Sept at the latest for 8.0 pushing out to new users; end of Feb for default version, production ready
exactly what do we need to finish first?
- Cylc 8.0 TODO List

AOB?

What should HO present the UM Workshop next week?

Quick Summary Of Changes
Expected 8.0 release date
What's still missing, future work

hjoliver commented 2 years ago

July

Apologies:

Cylc 7 next release

minor bug fix(es) - GUI and cylc review
need to avoid messing with Cylc 8 run-dirs

Cylc 8.0b2

almost done?
9 open issues, most closed by current PRs? https://github.com/cylc/cylc-flow/milestone/93

Cylc 8.0

What's still TBD?

BUG: cylc reinstall nukes scheduler work dir
- MH: easy to fix (skip log, work, share)
optional/expected outputs
intelligent host select (~done?)
- still in review with OS
- (w/o platform from platform groups - follow-up)
authorisation (~done?)
- documentation added (auto-gen)
- ready for review!
- OS fixing an error on his JupyterServer branch
integer flow labels (~done)
- (HO almost review-able)
infer run number (~done)
- ready for review
UI: hierarchical gscan (~done)
- ready for review
- possible performance improvements as follow-up
- sorting bug pending review (separate fix PR)
UI: table view (~done?)
- missing multi-jobs
- and sorting
- and columns not configurable
a bunch of ~smaller PRs
anything else?
- 94 Open Issues :scream: https://github.com/cylc/cylc-flow/issues?q=is%3Aopen+is%3Aissue+milestone%3Acylc-8.0.0
- VC next week OS, DM, HO
- need to decide any that are critical, and bump the rest
JupyterServer changes will probably be good to go!
- new release of J-S thanks to OS
Universal ID?
- not a breaking change
- CLI implementation easy, pushing it throughout the system
Graph view is 8.x
Static graph view essential??
- tide us over: sed, cylc graph --ref, dot => PNG. About an hour's work!
RELEASE DATE?
- "September hard deadline"? (If possible!) If not, a lot of beta users with heavy use
documentation!
- review issues
- Quick Start Guide: simple as possible

AOB?

UMP contract
"semi-absolute triggers"? (Yikes) https://github.com/cylc/cylc-flow/issues/4316
post-8.0 stuff in progress
- GraphQL subscriptions at the scheduler
next meeting?
- DM are from 18 August
- ~Tuesday 17th August, same time
JR: a week in August to do a POC, at BOM
MO has some include/exclude-at-startup suites, is SOS-specific. Is impossible to support with SoD? Need to obsolete it, or say "unsupported" and point to a migration page. Probably easily down with "optional outputs". Note an 8.0 blocker
put links to migration guide in certain error messages

hjoliver commented 2 years ago

August

Sites

any new Cylc 8 issues?

Quick summaries of what we've each been working on, main issues and problems, etc.

Learn about Jupyter Server and review Oliver's PR's
- (also corrects deficiencies, e.g. internal SSL, distributed UISs, which we won't have to reinvent ourselves; has a magic switch to go between single user and hub use - for using J-hub on the front multi-user)
- OS have contributed a change to J-server
Review Mel's Authorization PR's
- ready to go in
- OS J-server to go in first then MH to rebase
- Let's get J-server in quickly
Trying to start using milestones for UI, focusing on 0.6 and avoiding adding more things, or work on 1.0 issues
0.6
- Table view
- (priority)
- Propagate status in the GScan hierarchy
- hunt down any tracebacks and errors we might see in the UI
- TODO test UI master memory footprint

8.0b2 release.
- auto-release actions worked great
- now have unix builds incl. mac os
- TODO: can we delete the earlier meta-package releases? and turn the meta package into a stub?
Jupyter Server work
Trying and failing to keep up with reviewing

Centralised reg parsing & infer run number ready to go in: one-stop for CLI, infers runN
Conditional prerequisite bug
- in progress
Cylc 7
- better handling of Cylc 8 workflows
  - stops Cylc messing with Cylc 8 workflows
- store cycle point TZ in DB
  - (back port)
  - TODO NEED A Cylc 7 RELEASE FOR THIS - HO

required and optional outputs
- proposal
- PR cylc-flow#4343
- (Cylc 7 back-compat!)
integer flow numbers (already discussed, but not finished off and reviewed)

Finished Auth work
greyed-out UI mutations for not-authorized
Review Jupyter Server
Getting Auth working with Jupyter Server

IHS From Platforms
- ready to go
Cylc Rose Fixes
- Work with list, validate, graph, config
- Template var handling bug
Documentation - Various
Cylc Report timings
- Cylc 7: Cause mysterious (probably pandas version related), PR up.
- Cylc 8: Most obvious failure merged. Ticket raised to look for other failures.
- We have at least one user at MO.
Warnif StopCP > FinalCP

8.0.0 release

In September
Next UIS release should be all the multi-user multi-server functionality
- another beta release with J-server

What's still TBD?

Discuss

https://github.com/cylc/cylc-flow/issues/3704
~~https://github.com/cylc/cylc-flow/issues/3293~~
cylc/cylc-doc#420 (DM: I propose we leave behaviour as is and document clearly)

AOB?

Open Questions (any time left over)

https://github.com/issues?q=is%3Aopen+is%3Aissue+archived%3Afalse+label%3Aquestion+org%3Acylc

Next videoconference

hjoliver commented 2 years ago

09 September

Sites

any new Cylc 8 issues?
graphql.error.located_error.GraphQLLocatedError https://github.com/cylc/cylc-uiserver/issues/244 (BK: seeing this everyday while working on the UI)

Quick summaries of what we've each been working on, main issues and problems, etc.

GScan improvements (performance & maintainability) and propagate states - https://github.com/cylc/cylc-ui/issues/721
- might need Vue3 and ditch vuetify for performance
- punt this to 1.1
Deltas callbacks to fix deltas applied to families - https://github.com/cylc/cylc-ui/pull/749
- good to go
- UIS on reload will fix the sync (start from scratch) - causes a blink in the UI (PR up to fix this)

Reviewing.
Reviewing.

Reviewing (mainly optional outputs)
cylc clean - docs + improve logging
- PR coming
A couple of bugfixes

Optional outputs https://github.com/cylc/cylc-flow/pull/4343
- new approach to family triggers and output "optionality"
- refactored workflow timeout code (there are 3 timeouts now)
- ongoing management of conflicts with the integer flow label branch :nauseated_face:
Some reviewing and merging

Continuing Auth work working with Jupyter Server - good to go; and the associated UI PR
- (and reviewing auth in general, with OS)
Reviewing

Cylc Rose
- No issues against 0.3
- 2 Issues against 1.0
- retire rose site is blocked by https://github.com/cylc/cylc-flow/issues/4108) (tagged 8.x, but the Cylc-rose pr is tagged 1.0, which is inconsistent)
- ROSE_VERSION/CYLC_VERSION
Cylc Flow
- IHS From Platforms (still in review)
- cylc stop --kill bug
- Remove RunN if target gone
- Wrapper Scripts
Documentation
- Platform examples
- Example Conda Builds
- Host ⇒ Platform Upgrader

8.0b3

8.0.0 release

In September (which is, this month :scream: )
- :tada: hashtag yesplease
Next UIS release should be all the multi-user multi-server functionality
- another beta release with J-server

What's still TBD?

cylc-flow

cylc-ui

cylc-uiserver

Discuss

Lets concentrate 100% on:

get current PRs merged
bump all issues that can reasonably be bumped
hammer the system to find any problems
full docs revision

"Release candidate" - once major features are frozen (once current PRs are in)
UI post 1.0 can do a continuous release process to get new features in

NOTE: - one more Cylc 7 release required, for cylc review compat

AND: final beta release with Jup-Server, optional outputs, and distributed spawner...

AOB?

Open Questions

Next videoconference

meetings every 2 weeks from now, for those can attend, every second one can be smaller

hjoliver commented 2 years ago

22 September

Sites

any new Cylc 8 issues?

Quick summaries of what we've each been working on, main issues and problems, etc.

Propagate state in GScan
Deltas callbacks
Fixing production build problem found by Hilary
Documenting UI Development

UM Partnership contract - can BK be replaced :sob:
presenting Cylc 8 to NIWA HPC and Data Science management and teams
"optional outputs" merged to master
reviewing and testing ...

Finalising auth ui-server work (mainly fixing tests and refactoring)
Adding tests and responding to feedback on ui disabling unauthorised buttons
Couple of reviews
Met Office related admin

TP Done:

Documentation
Holiday
Make booleans line continuation chars in Graph language Waiting:
Reviews of platform from platform group

TODO

Cylc 7 - one more release required, for cylc review compat

cylc-flow

8.0b3 open issues
8.0rc1 open issues
8.0 open issues
final beta release with Jupyter Server, optional outputs, and distributed spawner...
"Release candidate 1" - once major features are frozen (once current PRs are in)

cylc-ui

0.6 milestone
1.0 milestone
UI post 1.0 can do a continuous release process to get new features in

cylc-uiserver

Next UIS release should be all the multi-user multi-server functionality
- another beta release with J-server
0.6 milestone
1.0 milestone

Priorities

Lets concentrate 100% on:

(avoid new developments as far as possible?)
get current PRs merged
bump all issues that can reasonably be bumped
hammer the system to find problems
full docs revision

AOB?

Open Questions

Next videoconference

meetings every 2 weeks from now, for those can attend, every second one can be smaller

hjoliver commented 2 years ago

08 October

Apologies

David Sutherland, (DM & MH 30 mins late)

Sites

any new Cylc 8 issues?

Quick summaries of what we've each been working on, main issues and problems, etc.

Triaging issues & writing docs
Fix for "Wrong status in Toolbar" - https://github.com/cylc/cylc-ui/pull/795
Fix for "UI reporting delta not in the lookup" - https://github.com/cylc/cylc-ui/pull/749
Created issues in each conda-forge repo to remind others to update maintainers list in the next release
Assigned Hilary admin role in each Element chat I am listed as admin (I cannot demote my own user, someone will have to change it)

Catching up.
Reviewing.

Allow implicit tasks in Cylc 7 back-compat mode
cylc clean improvements/fixes
A bunch of small stuff

(various reviews)
(4 days leave)
finished off integer flow labels and fixes related to triggering tasks beyond the scheduler window
discussions with NIWA and UM Partnership management on backfilling Bruno's position

Reviewing
Finishing Auth Documentation
Responding to review comments.

TP Done

Platforms from Platform groups now in!
Prevent recursive installation
Polling bug
Assorted bugfixes/documentation/reviewing Priorities
Reviewing as asked
Closing my own PRS
Add checks to Rose 1 to prevent it damaging installed Cylc workflows

TODO

cylc-flow

8.0b3 open issues
8.0rc1 open issues
8.0 open issues
final beta release with Jupyter Server, optional outputs, and distributed spawner...
"Release candidate 1" - once major features are frozen (once current PRs are in)

cylc-ui

0.6 milestone
1.0 milestone
UI post 1.0 can do a continuous release process to get new features in

cylc-uiserver

Next UIS release should be all the multi-user multi-server functionality
- another beta release with J-server
0.6 milestone
1.0 milestone

Meeting Notes

beta-3 release: let's get authorization merged
mutations bug on start-up (mutations menu does not appear) - problem with XSRF token, see the Issue, BK to look
UIS memory leak? middleware objects get created but never destroyed.Are we using Graphene middleware properly? One middleware object per request, so less of a problem now that we have persistent subscriptions, but still needs fixing (still have auth requests etc.). OS: may be able to reuse the same object each time.
BK: will post reminders for others to take over pypi and conda admin before he leaves
Vuetify is blocking migration to Vue 3, which would allow us to drop vuex, for simpler code and better reactivity. Vuetify itself to be updated for Vue3 but has been delayed, now ETA Q1 2022.
OS and BK: need to complete the job of purely datastore-driven views (currently still need initial burst deltas for constructing the view)
gscan performance is an issue with many workflows running - frequent updates all of which cause DOM reflow. This is worse on the "propagate states" work but performance not great on master either. BK: can we have the UIS construct the tree and send the result to the UI?
HO: will meet with new guy AC early next week

AOB?

HO: demote preparing from task state to badged waiting?

Open Questions

Next videoconference

hjoliver commented 2 years ago

21 October

Apologies

(BK)

Sites

Any new Cylc 8 issues?

Quick summaries of what we've each been working on, main issues and problems, etc.

reviewing!

lots of PR reviewing and merging
investigating cross-user hub
UMP contract (truncated "tranche 3" + "tranche 4")
lining up the BK -> AC handover

Authorization follow-up questions to discuss
Reviewing
UI Familiarisation
Auth docs

TP Done

CYLC_W_NAME => CYLC_W_ID [and lots of follow up]
2 Small UI changes (mostly training)
Assorted documentation & small fixes

In progress

Differentiate between Cylc config not set and config not a config.
Avoid recursive installation
Stop Rose 2019.x fiddling with Cylc 8 installs.
Cylc 8 timezone default == UTC

upgraded graphql-ws in the UIS: -160/+16 code lines :+1:
understands the delta processing bug and has a fix in the works
- will need quick reviewing once the PR is up

TODO

cylc-flow

8.0b3 open issues
8.0rc1 open issues
8.0 open issues
final beta release with Jupyter Server, optional outputs, and distributed spawner...
"Release candidate 1" - once major features are frozen (once current PRs are in)

cylc-ui

0.6 milestone
1.0 milestone
UI post 1.0 can do a continuous release process to get new features in

cylc-uiserver

Next UIS release should be all the multi-user multi-server functionality
- another beta release with J-server
0.6 milestone
1.0 milestone

AOB?

New team member AC has been indoctrinated by BK. First cylc-ui issues to start on?

A few of easier reviews and changes?
Tableview + finish datastore-driven view framework

Reminder of BK's final docs and notes:

Open Questions

Next videoconference

~2 weeks, day time NZ?
let's go back to Element VC, not Teams (we are an open source project :-)

Meeting Notes

HO: UIS spawning by hub requires J-Hub "admin user" status, or target UIS running already
- upcoming changes in J-Hub and J-server may help us address this without modifying the hub code
- try a custom hub service (that can be permitted to do just what we need)?
- not a super high priority right now because single-user use is now much easier
- worth posting a question to the JupyterX forum?
MH working on the xsrf issue, then authorization config validation
beta-3 release by end of next week if possible. Needed:
- DS delta processing fix
- MH cylc-ui PR for greying out unauthorized operations
- MH to look at the mutations not loaded issue too
- (all cylc-ui 0.6.0 issues if should be done)
but note many of us are taking leave during this time: NZ 4-day weekend, most UK people taking several days to a week off

hjoliver commented 2 years ago

02 November

Sites

any shocking new Cylc 8 issues?

Quick summaries of what we've each been working on, main issues and problems, etc.

OS

Reviewing.
Preparing task state and related fixes.
Rose CLI: Bash -> Python

RD

Remove ping mutation
Investigating the variations in GraphQL response formats causing traceback (see cylc-uiserver#242)
Reduced functional tests flakiness on GH Actions

HO

project admin: UM P "tranche 3" contract executed (and invoiced) at NIWA, "tranche 4" in the works
AC induction
UI triggering and "set-outputs" bug fix PRs
reviewing
testing:
- "preparing" job state problem (re)discovered
- delta log jam in the UI?

MH

Holiday
Reviewing
_xsrf token problem
continued follow up auth in ui and docs work
TP

Outstanding assigned issues

[PR up] Prevent nested installations
[PR up] Add changed in version ... to spec notes & Review CLI and Schema Docs
[PR up] Cylc Config Item Not Found
[Draft PR] Cylc Rose accidentally run twice
[Better fix this...] Review content of Log/flow-config directories
[Blocked by rose work] Investigate wrappers and conda activation (PR Up and in draft)

TODO

cylc-flow

8.0b3 open issues
8.0rc1 open issues
8.0 open issues
final beta release with Jupyter Server, optional outputs, and distributed spawner...
"Release candidate 1" - once major features are frozen (once current PRs are in)

cylc-ui

0.6 milestone
1.0 milestone
UI post 1.0 can do a continuous release process to get new features in

cylc-uiserver

Next UIS release should be all the multi-user multi-server functionality
- another beta release with J-server
0.6 milestone
1.0 milestone

Priorities

AOB?

Open Questions

Next videoconference

hjoliver commented 2 years ago

18 November

Any important new issues?

Cylc 8?
Do we need another Cylc 7 bug fix release?:
- https://github.com/cylc/cylc-flow/pull/4518
- https://github.com/cylc/cylc-flow/pull/4516
- yes we do, some sites operational with Cylc 7 for a good while yet
- let's get the PBS long job ID fix in too (DM has a hack fix for it already)

Quick summaries of what we've each been working on, main issues and problems, etc.

OS

8.0b3 release
Rose Python CLI: Bash -> Python

RD

Updating/fixing GH Actions release process
Working with Tim on disallowing nested installs - cylc-flow#4442
Working on type/attribute errors in UI server - cylc-uiserver#271
- Question: cylc-flow#4523

HO

mostly: marathon doc upgrade for Cylc 8

Questions:

aborting a retry sequence in Cylc 8, without cylc reset
(SoD with cycle offset errors in the graph?)

MH

Getting the user who executes the command in the UI logged in the workflow log cylc-flow#4522 and cylc-uiserver#288
Removing suite.rc/flow.cylc symlink cylc-flow#4506.
Reviewing

Questions:

Scrap configurability of comms timeout? https://github.com/cylc/cylc-flow/issues/4112,
- bump config issue to 8.x; issue is must be configure on remote platforms as well, but that's OK
- leave the --comms-timeout CLI option, it actually works
Are we happy for other users' ui-servers, available in pulldown menu in UI, to be configurable? https://github.com/cylc/cylc-uiserver/issues/273
- yes (user decides who else's UIS they want look at if authorized)
- also need access to other hubs (probably another browser tab)

AC

multiples views driven by the UI shared datastore alone
tableview PR: let's get is merged now; further improvements as follow-up

TP

Still in progress

Reviewing Oliver's Rose CLI → Python change.
No recursive install - waiting on Ronnie's finessing
Only Run Cylc Rose once - nearly done
Cylc 7 Bug
Done
Fixed UIS bug for Dave and min weird groups config (probably a common issue here)
Review of Config Spec internal docs - painful, but less painful than I feared.
Messy CYLC_WORKFLOW_NAME/ID follow up 😞
Questions

Can we review where we are at with the wrapper script changes?

DS

TODO

8.0b3 release

new conda outputs new documenting
cylc-uiserver dependent on graphql-ws, BK and HO are maintainers of the conda-forge package. We're two releases behind (need merge). Should ping the pypi package owners to see if they are willing to take it on
need to update admin release process docs with graphql-ws

cylc-flow

cylc-ui

1.0 milestone
UI post 1.0 can do a continuous release process to get new features in

cylc-uiserver

1.0 milestone

Priorities

Open RC1 Questions

https://github.com/cylc/cylc-flow/issues?q=is%3Aissue+is%3Aopen+label%3Aquestion+milestone%3Acylc-8.0rc1

AOB?

Open Questions

Next videoconference

30 Nov: 8.0r1 issue triage; and decide if we release in December.

hjoliver commented 2 years ago

30 November

Sites

any new Cylc 8 issues?

TODO

Particular issues (?):

cylc-flow

final beta release with Jupyter Server, optional outputs, and distributed spawner...
"Release candidate 1" - once major features are frozen (once current PRs are in)
8.0rc1 issues

cylc-ui

1.0 milestone
UI post 1.0 can do a continuous release process to get new features in

cylc-uiserver

Next UIS release should be all the multi-user multi-server functionality
- another beta release with J-server
1.0 milestone

Open Questions

Priorities

(What are your priorities?) (Are there any blockers you need to discuss?)

Maintenance, admin & boring stuff.
Universal ID

In progress: client/server + GraphQL refactor to reduce unexpected traceback: cylc-flow#4529
Upcoming: cylc install should respect max scan depth: cylc-flow#4363

(off sick for a few days :grimacing: ... well, not really "off", but sick)
bulk docs upgrade for Cylc 8 ... then beta3 release announcement

Reviewing
Passing auth user to cylc flow for logging (https://github.com/cylc/cylc-uiserver/issues/289 identified in process)
Started work on passing configurable ui server access. (https://github.com/cylc/cylc-uiserver/issues/273)

Cylc Tutorial/Resources
Rose issues marked rc1. Functional first, then documentation.

Ditch the filter button
Table view merged (driven by shared store)
Table view new PR
- Expand jobs
- filter toolbar pinned

AOB?

Next videoconference

hjoliver commented 2 years ago

8 December Cylc-UI Meeting

1. Table View (and general UI) MVP for 8.0rc1

Latest state of the table view: https://github.com/cylc/cylc-ui/pull/861

2. Begin prioritizing further UI development

OS from Element chat:

Ok, here's my attempt at triage. We can probably agree a path forward before the meeting, have a skim through, see what issues you think should be higher up the pecking order.

I think the main priority right now is to get what we have working properly and fill in the obvious functionality gaps to make it usable. My picks:

Addressing user feedback (will be plenty before long).

Multi-user functionality.

Finishing off GScan, Tree & Table views (mutations, polish, performance).

Finishing off the data model (& any changes to the view interface).

Improving the mutation editor.

Implement the view toolbar.

Preserve user's tabs.

Then we can start implementing the more ambitious features e.g:

Graph view.

Offline data.

N-Window.

I've been through the UI issues (wrote up a few things that were missing). Here's a quick (and scary) overview of the main things:

Create views off the data store

Close this issue https://github.com/cylc/cylc-ui/issues/803

Multi-user functionality

Jupyter Hub & spawning other users UIServers.

Allow switching user/hub via the boxes in the top left. (requires a new configuration in the UIS).

Views

View interface

Create a standard toolbar component for views to display search, filter and control icons - https://github.com/cylc/cylc-ui/issues/471

Preserve user's tabs - https://github.com/cylc/cylc-ui/issues/662

Allow users to configure their default views?

Cylc "views" should be plugable things that can be easily developed by others.

Any more improvements to make before we start writing more views.

Table view improvements https://github.com/cylc/cylc-ui/issues/631

Graph view

GScan improvements

Auto-collapse workflows with only one run.

Propagate states up the tree.

https://github.com/cylc/cylc-ui/issues/721

https://github.com/cylc/cylc-ui/pull/736

Add mutations to Cylc objects.

Add clean mutation at the UIS.

Add warning icons - https://github.com/cylc/cylc-ui/issues/632

Broadcast view - https://github.com/cylc/cylc-ui/issues/463

Etc...

AOTF

Improve the mutation editor forms (consider ditching Vuetify for this).

Auto-complete cycle points etc - https://github.com/cylc/cylc-ui/issues/602

XTriggers

Display xtriggers in tree (& table?) view(s).

Includes retry triggers.

https://github.com/cylc/cylc-ui/issues/131

https://github.com/cylc/cylc-ui/issues/331

Testing

"@cypress/vue" - https://github.com/cylc/cylc-ui/issues/610

Updating UI without updating the UIS.

The UIS supports multiple UIS versions, but how to install them.

A simple admin command to unpack the latest release from GH releases?

UI needs to detect a new version is available https://github.com/cylc/cylc-ui/issues/565

Selection

Allow selection within the views.

Allow keyboard control over selection (up down left right) ala tui.

Allow selection in one view to be transferred to another.

https://github.com/cylc/cylc-ui/issues/434

https://github.com/cylc/cylc-ui/issues/534

n-window

Allow the n-window to be controlled on a per view basis.

The workflow service will need to widen the data window to the largest requested number.

https://github.com/cylc/cylc-ui/issues/864

Data model

Revisit sorting - https://github.com/cylc/cylc-ui/issues/694

We could consider simplifying the subscription interface. https://github.com/cylc/cylc-ui/issues/862

Keep subscriptions alive for a short while to allow quickly switching back: https://github.com/cylc/cylc-ui/issues/863

How do we handle future schema changes?

Mutations are the same for all workflows irrespective of the Cylc version they are running with because of the multi-workflow GraphQL endpoint.

Can we improve this to handle schema changes more elegantly.

Offline Data

Render the n=1 window from the database when the scheduler is down.

Allow "cylc reivew" style n=max working with DB data.

Log files

Push static graph data into the data store when requested.

JS packaging - https://github.com/cylc/cylc-ui/issues/412

Consider packaging the graph view when developed so we can use it in the docs?

3. Conclusions

Immediate priorities:

Addressing user feedback (will be plenty before long).
Multi-user functionality.
- Launching another users UIS.
- Switching between UIS'es and Hub's via the GUI.
Finishing off GScan, Tree & Table views (mutations, polish, performance).
- Make mutations available in all places.
- Improve GScan sorting.
- Propagate workflow states up the tree in GScan.
- Auto-collapse nodes in GScan where appropriate.
- Improve GScan load times for larger numbers of workflows.
- Implement the view toolbar.
- Final spit n' polish & misc improvements.
Improving the mutation editor.
Preserve user's tabs.

Longer term priorities:

Graph view.
Offline data.
N-Window.
Any data model improvements that are required.

wxtim commented 2 years ago

16/17 December (UK/NZ respectively)

Christmas Leave/Apologies?

RD, DS, AC

Any new Cylc 8 issues?

Work Done/Priorities / Blockers

DS - on leave

Universal ID

RD - on leave

Deployed and documented 8.0b3 for users on NIWA HPC
- used and adapted wrapper from TP's umerged PR
Reviewing

Reviewing
Tidying passing auth user from uiserver to cylc-flow.
- should be read to merge now
Working on configurable access to others ui servers (https://github.com/cylc/cylc-uiserver/issues/273).

Chipping away at Cylc Docs where possible.
Pushing Rose & Cylc-Rose towards rc1/1.0.0.
Review/Rewrite Rose Docs: rose suite-run → cylc validate; cylc install; cylc play.
Foolishly Poking @hjoliver and @oliver-sanders about Cylc Tutorial Workflows (and tutorial docs?).
- decision made
https://github.com/cylc/cylc-flow/issues/4528 - Probably trivial - any violent objections?
- no objections

AC - on leave

TODO

can we release 8.0rc1 before end-of-year?

Possible blockers:

universal ID
task prerequisites outside beyond n=0
(change in hub/server config file path)

Decision: release rc1 in early January. It should have universal ID, task prereqs for the UI, and fixed tutorials

extant issues

cylc-flow

cylc-ui

1.0 milestone

cylc-uiserver

1.0 milestone

AOB?

Open Questions

HO to check the old rose symlink config issue - does NIWA still need it? DM says no way to do it with Cylc 8 at the moment.

Next video conference

mid Jan

hjoliver commented 2 years ago

EOY