Download job output

[Peter9192]

Add endpoint for downloading contents of job folder as archive.

To do:

• [x] allow different archive formats
• [x] exclude input/output/stdout/stderr
• [x] cleanup archive after download

To test:

• Try the archive endpoint on http://127.0.0.1:8000/api/docs#/job/retrieve_job_stdout

refs #66

[Peter9192]

Tried using accept header to request specific output format. This works, but not through the openapi docs. Perhaps a more graceful option is better: try to respect requested header, but fallback to default if that doesn't work. Another alternative is to use a query parameter instead.

With respect to the exclude option: shutils.make_archive doesn't have that natively. Solution could be to copy content to a temporary directory excluding some files, and pack the tempdir.

[Peter9192]

Maybe we can explicitly add "accept" value to the FileResponse object

[Peter9192]

Using a query instead of accept header make it possible to copy the URL

[Peter9192]

can also use PyFileSystem with a file-like streamingresponse.

Then we can build the archive in-memory, walking over the directory contents including/excluding files as specified. Challenge will be to change the in-memory archive from write to read mode. Fallback option will be to write to tempfile on disk.

[Peter9192]

Maybe we can explicitly add "accept" value to the FileResponse object

Doesn't solve the issue with the openapi docs, it will still submit as application/json

[sverhoeven]

you could try to add binary content type application/octet-stream in openapi spec

[Peter9192]

you could try to add binary content type application/octet-stream in openapi spec

That works 🎉 I'll keep using the query param as I like that the URL is copy-able, but adding this additional info to the API is still useful :-)

[Peter9192]

@sverhoeven thanks for the helpful review! I addressed all comments, can you have another look?