search

i-VRESSE / bartender

Middleware web service to schedule jobs on various infrastructures
https://i-vresse-bartender.readthedocs.io
Apache License 2.0
0 stars 0 forks source link

Handle big output #104

Open sverhoeven opened 1 day ago

sverhoeven commented 1 day ago

When I have a job destination with a sftp/dirac filesystem and a job output of several Gigabytes. Then copying the output back could timeout

sverhoeven commented 1 day ago

This is also a problem for the re-* interactive applications. They would also take long time

sverhoeven commented 1 day ago

Also creating archives could timeout. A workaround could be to create archive of all files during job executation.