cylc / cylc-flow

Cylc: a workflow engine for cycling systems.
https://cylc.github.io
GNU General Public License v3.0
329 stars 93 forks source link

event handler calls don't look in share/bin #6240

Closed dwsutherland closed 2 months ago

dwsutherland commented 2 months ago

Description

Event handlers can't find scripts deployed to share bin, i.e. [event]handlers don't have $CYLC_WORKFLOW_SHARE_DIR/bin in PATH ..

This bin is meant to be the new location for all deployed code within a workflow, and is made available for job scripts, but event handlers may also need deployed.

Workaround is to deploy to $CYLC_WORKFLOW_RUN_DIR/bin and add exceptions to .cylcignore but this shouldn't be needed.

Reproducible Example

[sutherlanddw@w-ec-admin01 c3s]$ ls ~/cylc-run/c3s/run1/share/bin
cycle_obj_transfer.sh  ecox-archive.sh  icinga2.sh  icingaclient.py
[(('event-handler-00', 'succeeded'), 1) cmd] icinga2.sh 'succeeded' 'c3s/run1' '20240410T00/download_C3S_rolling_hindcasts' 'job succeeded'
[(('event-handler-00', 'succeeded'), 1) ret_code] 127
[(('event-handler-00', 'succeeded'), 1) err] /bin/sh: icinga2.sh: command not found

Expected Behaviour

Even handlers find script in share/bin

oliver-sanders commented 2 months ago

This bin is meant to be the new location for all deployed code within a workflow

The share directory is for resources that are created by tasks, that we want to persist and share with other tasks.

The share/bin directory is intended for executables that are generated by the workflow (e.g. if you have a make task that produces an executable, move it to share/bin to make it available to tasks).

The regular bin directory is for scripts and executables that are present in the workflow source.

We haven't considered the use case of a workflow event handler that uses an executable generated by tasks within the workflow.

dwsutherland commented 2 months ago

Yes, the issue is there maybe many handler scripts deployed from external repositories (via deploy tasks) that need to persist across reinstall.. And declaring them all in the .cylcignore for every workflow is undesirable.