By default, CBRAIN tasks create symbolic links to the CBRAIN DP cache for all the input files that the task need; that way inputs can be shared by multiple task without too much data duplication.
The symbolic links are created by the make_available() utility method in class ClusterTask.
Some stupid tasks though MUTATE their own input data, so creating a symlink doesn't work anymore, the input data of one task would perturb the input data of another one directly in the cache area.
I'd like a new option for make_available() where instead of creating a symlink, the method would make a full copy of the input data directly in the task workdir.
I'd also like this option to be invoked when a Boutiques descriptor for the new integrator contains a statement about making a full copy. This shouldn't be implemented as a module, but as an option in the custom section of the descriptor.
By default, CBRAIN tasks create symbolic links to the CBRAIN DP cache for all the input files that the task need; that way inputs can be shared by multiple task without too much data duplication.
The symbolic links are created by the make_available() utility method in class ClusterTask.
Some stupid tasks though MUTATE their own input data, so creating a symlink doesn't work anymore, the input data of one task would perturb the input data of another one directly in the cache area.
I'd like a new option for make_available() where instead of creating a symlink, the method would make a full copy of the input data directly in the task workdir.
I'd also like this option to be invoked when a Boutiques descriptor for the new integrator contains a statement about making a full copy. This shouldn't be implemented as a module, but as an option in the
customsection of the descriptor.