As it stands, it is very difficult (and in many cases impossible) to export or publish a template from a large study. However, often it is not necessary to copy every file in a study, just the structural elements. I.e., the nodes, the connections between them, the directory structure of jupyter notebooks, and code/config files (.py, .ipynb, *.txt etc.). Therefore I propose we extend the export/publish options to allow the user to filter which data types they would like to consider when cloning the study.
Expected behaviour
I expect to be able to export a study's contents, clone a study, or publish a study template, without necessarily copying large data files that are generated during the execution of the pipeline. Ideally, the user could filter (or otherwise somehow specify) which files/filetypes they would like to preserve with the copy.
Actual behaviour
Currently publishing/exporting a study tries to zip everything, including potentially (very) large intermediate files that are produced during pipeline execution.
Note: your environment was attached but will not be displayed
Long story short
As it stands, it is very difficult (and in many cases impossible) to export or publish a template from a large study. However, often it is not necessary to copy every file in a study, just the structural elements. I.e., the nodes, the connections between them, the directory structure of jupyter notebooks, and code/config files (.py, .ipynb, *.txt etc.). Therefore I propose we extend the export/publish options to allow the user to filter which data types they would like to consider when cloning the study.
Expected behaviour
I expect to be able to export a study's contents, clone a study, or publish a study template, without necessarily copying large data files that are generated during the execution of the pipeline. Ideally, the user could filter (or otherwise somehow specify) which files/filetypes they would like to preserve with the copy.
Actual behaviour
Currently publishing/exporting a study tries to zip everything, including potentially (very) large intermediate files that are produced during pipeline execution.
Note: your environment was attached but will not be displayed