RakipInitiative / ModelRepository

Joint project of EFSA, Federal Institute For Risk Assessment, DTU and ANSES to create a online model repository.
GNU General Public License v3.0
2 stars 0 forks source link

Feature: move all temporary data to workflow area #413

Closed schuelet closed 1 year ago

schuelet commented 2 years ago

We create temp folders to store our files like workspaces, plots, generated files and parameter data in the temp folder of the current system.

If possible, we move all those temp directories to some folder in the workspace of the workflow itself.

One problem I see: This might blow up the size of the workflow, making it more difficult to export it (is the loading time on startup affected?).

Should it prove to be unfeasible, we need to have another look at how we store temp data. The ScriptHandler and JSONHandler classes create a lot of data, and copy the relevant files to other temp folders. This reeds to be revised and optimized, some copy operations may be wasteful and superfluous. Special attention to combined models, since they might have the most optimization potential.

llavall commented 2 years ago

Indeed, the inner workflow-option only works well on the KNIME Server, since there the Workflow is not touched, but a copy is used as job and removed afterwards. Maybe we can have a folder for FSK stuff defined in the settings of FSK and there is then everything stored, For example also the .fsk-folder for R stuff. This would enable the posibility to just empty this folder or a relevant subfolder from time to time, without confliting the server files itself.

schuelet commented 1 year ago

resolved. Path to folder is set in FSK-Lab preference page. Property name for preferences.epf: fsk.path