search

PecanProject / pecan

The Predictive Ecosystem Analyzer (PEcAn) is an integrated ecological bioinformatics toolbox.
www.pecanproject.org
Other
202 stars 235 forks source link

Download multiple workflow files as a zip #2940

Open koolgax99 opened 2 years ago

koolgax99 commented 2 years ago

Description

Currently we can only download an individual file from the workflow using download.workflow.file(server, workflow_id=99000000031, filename="pecan.xml", save_as="local.pecan.xml")

But now we should be able to download all the files in the workflow directory as a zip. In this we would only specify the parameters like worflow_id and maybe save_as

Is your feature request related to a problem? Please describe. This will give users more functionality to download files from the pecan server. Some users might want multiple files from a workflow so currently they have to send the request multiple times. But now using only one request they can download all the files in that workflow.

Proposed Solution

Introducing a new function in pecan API's to download all files available in a workflow as a ZIP.

Alternatives Considered

If we don't wanna allow users to download all the files from a workflow, we can also allow users to download multiple files of their choice from a workflow using a single request. Eg. The function will take a list of files the user needs and then we can zip those files and give the users the zip containing all the desired files.

cc @istfer @robkooper

dlebauer commented 2 years ago

I like the idea. I don't recall what all the files in the workflow directory - does this include all of the model outputs or other files that could grow to and beyond GB sizes? I like the idea of allowing a user to specify a subset to include / exclude, not by file name but by category.

robkooper commented 2 years ago

The other thing to look at is in the different pecan.xml files in the workflow folder, we should probably try to filter out the database connection parameters.

koolgax99 commented 2 years ago

I like the idea. I don't recall what all the files in the workflow directory - does this include all of the model outputs or other files that could grow to and beyond GB sizes? I like the idea of allowing a user to specify a subset to include / exclude, not by file name but by category.

Yes i think the workflow directory may end up in GB sizes which would be a bad idea at this point of time. The Category wise inclusion/exclusion would be good too. But @dlebauer sir what's ur opinion on giving the list of files for downloading?

github-actions[bot] commented 1 year ago

This issue is stale because it has been open 365 days with no activity.