VUIIS / dax

Distributed Automation for XNAT
MIT License
25 stars 24 forks source link

Dax relaunching failed jobs(memory and time limit) #408

Open KarthikMasi opened 1 year ago

KarthikMasi commented 1 year ago

Job failures because of time limit or memory limit can either use an automatic relaunch of the jobs with increased time and/or memory limit.

Other ideas that were pitched around were:

To assign memory and time requirements based on image resolution or size at build time.

To launch 5 or 10% of new assessor jobs for a project as a precursor and based on the results, launch rest of the jobs with an increased time and/or memory limit.

Thoughts?