search

aws-deadline / deadline-cloud

Multi-purpose library and command line tool that implements functionality to support applications using AWS Deadline Cloud.
Apache License 2.0
25 stars 28 forks source link

Bug: job_history not appended to from cli or python submission #448

Open Mathieson opened 6 days ago

Mathieson commented 6 days ago

Describe the bug

I recently learned there is a job_history folder, mentioned in the docs here.

This folder does not exist on my machine. I've been submitting jobs primarily via Python and a few times via the deadline cli. None of these jobs have ended up in the job_history folder.

Before realizing this feature existed, I wrote a Python context manager that creates a folder (optionally temporary) and allows me to dynamically assemble my job_bundle. I also made the folder in the .deadline folder, but by a different name, so I could keep track of submission history. My context manager was intended to create a temporary folder every time. However, I've never had it be temporary because I want to see the results of the dynamically assembled bundle for sanity and troubleshooting.

Discovering that this feature already exists but is not being called for these kinds of submissions prompted me to create the ticket here. I'd much rather keep my bundle creation temporary and have the history managed by Deadline.

Expected Behaviour

I expected my submitted jobs to appear here.

I'd expect it to be representative of the bundle after any additional processing that might happen inside Deadline's submission processing.

I don't know what is included in these job_history bundles since none have been created for me yet, but giving the sub-folder a legible name (whatever the name is that gets submitted by the deadline) was an extra challenge for me that I wasn't prepared to try to tackle. I ended up just naming each folder as a GUID. It makes it difficult to associate which job is which, beyond sorting by date last modified. I'd hope for it to name them in a way that is easier to associate with what was submitted. Maybe even include a link to the submitted job URL as it appeared on Deadline after submission.

This feels like it is getting more into feature request territory than bug report, but I wanted to still mention it since I don't actually know what this feature looks like even though it currently exists.

Current Behaviour

Nothing is logged.

Reproduction Steps

This should do it

Environment

At minimum:

  1. Operating system: Windows 11
  2. Output of python3 --version: Python 3.11.9
  3. Output of deadline --version: deadline, version 0.48.8
epmog commented 3 days ago

Hey Mathieson, thanks for creating the issue.

The only workflow that currently creates a job history entry is the gui submitter (either via CLI or launching the gui from the library). The rationale here is that this is only ever a interactive usage of the submission. For other uses of the cli/library (non-gui based) we currently don't do it, since the assumption was that you're scripting the submission (and thus don't need a copy of the bundle).

For the library usage we expose the functionality in the python library via create_job_history_bundle_dir, so you should be able to add that to any of your scripts if you'd like that functionality: https://github.com/aws-deadline/deadline-cloud/blob/41c64861a2e51fab159b84bfcbaca79d6422ea6a/src/deadline/client/job_bundle/__init__.py#L20-L28

>>> import deadline.client.job_bundle
>>> dir(deadline.client.job_bundle)
[..., 'apply_job_parameters', 'create_job_history_bundle_dir', 'datetime', 'deadline_yaml_dump', 'get_setting', 'glob', 'loader', 'os', 'parameters', 'read_job_bundle_parameters']

I do believe we've missed a use-case where we might want to create the history for the bundle, which is an interactive cli submission (non-gui), so I'm thinking that's what this issue should become. Perhaps users can opt to turn it off if they indicate it's non-interactive (like in a bash script). Thoughts?