search

kabouzeid / turm

TUI for the Slurm Workload Manager
https://crates.io/crates/turm
MIT License
121 stars 4 forks source link

Array job log files aren't found. #11

Open evanr70 opened 1 year ago

evanr70 commented 1 year ago

Thanks for the excellent tool. My only complaint is that array job log files aren't found correctly. turm looks for the same file for each array task, that being the coordinating task.

Happy to look at this myself if you can point me to the right place to work

image
kabouzeid commented 1 year ago

What did you pass for --output= in sbatch?

This is the relevant function which tries to figure out where the log file is located: https://github.com/kabouzeid/turm/blob/c169e1844574885246736dbde920ae0f77b121b2/src/job_watcher.rs#L127

evanr70 commented 1 year ago

#SBATCH -o /well/woolrich/users/ozn760/analyse-tdd/cluster_run/pipeline/logs/%j.out

kabouzeid commented 1 year ago

Ah yes, more undocumented slurm behavior---I will probably fix that. If you want a workaround for now, I would recommend to use "%A_%a.out" instead of "%j.out" for array jobs.

kabouzeid commented 1 year ago

Also seems to happen with --wrap when no explicit output file is specified. Will fix soon.

gvahe commented 3 weeks ago

@kabouzeid Thanks for great work. Just following up on this, has this been resolved? When no specific output format is requested for an array job, it still does not account the array idx when searching for stdout in turm.

kabouzeid commented 3 weeks ago

The fix should be easy enough and it's been on my todo list for ages. Sadly, I don't find the time to do it. If someone wants to open a PR for this, I'll be happy to merge.