Sync only the report folder

SIMEXP / cbrain-plugins-psom

Boutiques JSON descriptors for Niak pipelines.

1 stars 6 forks source link

Sync only the report folder #22

Open poquirion opened 7 years ago

poquirion commented 7 years ago

The output folder can be pretty big (more than 600 GB for the 1570 GSP subjects), we should only sync the report folder on the cbrain portal.

glatard commented 7 years ago

In this case we should create two outputs for the task:

The report folder, that will be synchronized when the viewer is triggered.
The rest of the outputs (including the report folder), that will be synchronized only if the user explicitly requires it.

poquirion commented 7 years ago

Should we should add a step in niak to copy/move the report? Can/should that be done at another level, not sure of the best way to do that beyond the outputs folder we provide in the form.

glatard commented 7 years ago

We just need to register the report separately from the rest of the output files, by adding another element to the "output-files" array in the Boutiques descriptor. We can either move the report out of the complete results (if we don't want it to be registered and copied twice), or just leave it there for consistency. Unless the report is likely to grow, I think the second solution is the best.