aces / cbrain

CBRAIN is a flexible Ruby on Rails framework for accessing and processing of large data on high-performance computing infrastructures.
GNU General Public License v3.0
71 stars 42 forks source link

[ADD] boutiques_dir_maker module #1267

Closed natacha-beck closed 2 years ago

natacha-beck commented 2 years ago

Some tools take expect that a file/directory input is placed input a parent directory.

For example we have the following Boutiques command-line:

apptool [PRECOMPUTED_INPUT]

with the following input in Boutiques:

{
  "description": "A directory that contain the precomputed data process by...",
  "id": "precomputed_input",
  "name": "Precomputed input",
  "optional": false,
  "type": "File",
  "value-key": "[PRECOMPUTED_INPUT]"
}

and in the cbrain:integrator_modules section:

"BoutiquesDirMaker": {
    "precomputed_input": "fake_parent_dir"
 }

In CBRAIN the user will select a userfile for example sub-n, the final command line will become:

apptool precomputed/sub-n
prioux commented 2 years ago

I have several comments to make about the design and implementation, but for the moment I'll point out that the module is not installed on the bourreau side so this cannot work.

MontrealSergiy commented 2 years ago

Yup, @natacha-beck you need to add soft links to Bourreau side. Does any consumers of derivative/precomputed bids require participant or any other home folder files? (May be ask Eric?)

MontrealSergiy commented 2 years ago

Natacha are we targeting Nibabies + Cabinet? I believe command line should become

apptool fake_parent_dir

do you have some examples of command lines and data for these pipelines to test?

prioux commented 2 years ago

I'm just seeing lots of places where the changes will make things crash...

natacha-beck commented 2 years ago

apptool fake_parent_dir

Natacha are we targeting Nibabies + Cabinet? I believe command line should become

apptool fake_parent_dir

do you have some examples of command lines and data for these pipelines to test?

Here it is, according to Eric email's:

We do not want to provide sub-n directly to NiBabies, we want to provide something like ./precomputed/sub-n (although the folder doesn't explicitly need to be named precomputed, it just has to have the sub-n "precomputed" results directly beneath it)

MontrealSergiy commented 2 years ago

@natacha-beck What concerns Eric comment are you sure he talks about the command line tool and not CBRAIN tool? My understanding that's command line tool remains the same, but inputs of CBRAIN integration my be different. In some cases as with SingleSubjectMaker we change input format of the command line tool in the CBRAIN. Do not you recall Eric was basically asking to generalize the SingleSubjectMaker onto many bids datasets (and Pierre's SingleSubjectMaker does not contain sub-xxxx subfolder in the command line). Thus, (unless sub-xxx has another sub-xxx/sub-xxx under it), sub-xxx should be dropped from the command line.

according to Eric email's:

We do not want to provide sub-n directly to NiBabies, we want to provide something like ./precomputed/sub-n (although the folder doesn't explicitly need to be named precomputed, it just has to have the sub-n "precomputed" results directly beneath it)
prioux commented 2 years ago

One more overall comment. I'd like the entire module to be renamed:

BoutiquesInputSubdirMaker

natacha-beck commented 2 years ago

I tried to have a better explanation for the overall module. I also improve the variable naming. I test it locally everething works as I expect. Please give it an other revision round.