search

genericworkflownodes / GenericKnimeNodes

Base package for GenericKnimeNodes
https://github.com/genericworkflownodes/GenericKnimeNodes
Other
15 stars 16 forks source link

Storing loaded Input data only supported in (single) File Input #152

Open jpfeuffer opened 7 years ago

jpfeuffer commented 7 years ago

Why do we only overwrite save/loadInternals for the single FileInput but not for the multi FileInputs node? For the single FileInput, the Input file is zipped and stored in the "Internal" directory of this node, such that it can be recovered when closing and reopening the workflow. But not if you use the multi FileInputs node (or the new DirectoryImporter). Do we want this? In all? In none of them? I would like to have it consistent at least. How about a checkbox? @aiche @temehi @rrahn @timosachsenberg

AlexanderFillbrunn commented 5 years ago

From what I see the code in the Input File node does not save the whole file but only the first 50 lines. In the execute method the first 50 lines are read in a class variable called data and in saveInternals that data is stored on disk. I think it may not have been done for the Input Files node because you'd need a summary of all read files, which can get quite big and is not very easy to interpret.

jpfeuffer commented 5 years ago

Ah I see it now. It is only the preview that you can see by RightClick -> "View first lines" of file. I think none of the Input nodes creates FileStores and I think it is ok. You do not want to copy all potentially huge input files into a temporary directory. And usually you ship them separate if you share workflows.