adswa
commented
5 years ago I have a preliminary update on this: The failure occurs in https://github.com/poldracklab/fitlins/blob/c7cc798330a8fe8cea8ee545c64c7fb08d1a562b/fitlins/interfaces/nistats.py#L165-L180 during the third level model. The problem is that self.input.contrast_info for a reason I do not yet understand ingests entries from the participants.tsv file and turns them into contrasts.
This is how `self.input.contrast_info` looks like for a 3rd level model
```python (Pdb) self.inputs.contrast_info [{'entities': {'session': 'movie', 'task': 'avmovie'}, 'weights': [{'forrest_av_rating': 1}], 'type': 't', 'name': 'forrest_av_rating'}, {'entities': {'session': 'movie', 'task': 'avmovie'}, 'weights': [{'forrest_seen_languages': 1}], 'type': 't', 'name': 'forrest_seen_languages'}, {'entities': {'session': 'movie', 'task': 'avmovie'}, 'weights': [{'hearing_problems_current': 1}], 'type': 't', 'name': 'hearing_problems_current'}, {'entities': {'session': 'movie', 'task': 'avmovie'}, 'weights': [{'forrest_seen_dist': 1}], 'type': 't', 'name': 'forrest_seen_dist'}, {'entities': {'session': 'movie', 'task': 'avmovie'}, 'weights': [{'forrest_av_feeling': 1}], 'type': 't', 'name': 'forrest_av_feeling'}, {'entities': {'session': 'movie', 'task': 'avmovie'}, 'weights': [{'age': 1}], 'type': 't', 'name': 'age'}, {'entities': {'session': 'movie', 'task': 'avmovie'}, 'weights': [{'forrest_seen_count': 1}], 'type': 't', 'name': 'forrest_seen_count'}, {'entities': {'session': 'movie', 'task': 'avmovie'}, 'weights': [{'forrest_av_storydepth': 1}], 'type': 't', 'name': 'forrest_av_storydepth'}, {'entities': {'session': 'movie', 'task': 'avmovie'}, 'weights': [{'forrest_ad_known': 1}], 'type': 't', 'name': 'forrest_ad_known'}, {'entities': {'session': 'movie', 'task': 'avmovie'}, 'weights': [{'vision_problems_current': 1}], 'type': 't', 'name': 'vision_problems_current'}, {'entities': {'session': 'movie', 'task': 'avmovie'}, 'weights': [{'forrest_av_fatigue': 1}], 'type': 't', 'name': 'forrest_av_fatigue'}, {'entities': {'session': 'movie', 'task': 'avmovie'}, 'weights': [{'amplitude_.RIGHT': 1}], 'type': 't', 'name': 'amplitude_.RIGHT'}, {'entities': {'session': 'movie', 'task': 'avmovie'}, 'weights': [{'handedness': 1}], 'type': 't', 'name': 'handedness'}, {'entities': {'session': 'movie', 'task': 'avmovie'}, 'weights': [{'hearing_problems_past': 1}], 'type': 't', 'name': 'hearing_problems_past'}, {'entities': {'session': 'movie', 'task': 'avmovie'}, 'weights': [{'forrest_seen': 1}], 'type': 't', 'name': 'forrest_seen'}, {'entities': {'session': 'movie', 'task': 'avmovie'}, 'weights': [{'vision_problems_past': 1}], 'type': 't', 'name': 'vision_problems_past'}, {'entities': {'session': 'movie', 'task': 'avmovie'}, 'weights': [{'gender': 1}], 'type': 't', 'name': 'gender'}, {'entities': {'session': 'movie', 'task': 'avmovie'}, 'weights': [{'forrest_av_artist_count': 1}], 'type': 't', 'name': 'forrest_av_artist_count'}] ```where everything apart from the entry {'entities': {'session': 'movie', 'task': 'avmovie'}, 'weights': [{'amplitude_.RIGHT': 1}], 'type': 't', 'name': 'amplitude_.RIGHT'} stems from the participants.tsv file.
The following https://github.com/poldracklab/fitlins/blob/c7cc798330a8fe8cea8ee545c64c7fb08d1a562b/fitlins/interfaces/nistats.py#L165 will then go and assign weights of zero to the first contrast_info entry (as it should), but then in the following https://github.com/poldracklab/fitlins/blob/c7cc798330a8fe8cea8ee545c64c7fb08d1a562b/fitlins/interfaces/nistats.py#L169-L172 attempts to give an empty list as input to model.fit(), which results in the failure I observed.
I'm not sure whether the problem lies in the fact that contrast_info is given contrast information that I did not consciously specify anywhere in my model.json (and it is weird that this only happens on third, not second level), or in the fact that it will attempt to give an empty list as input. As for a temporary fix, I modified it like this:
for name, weights, type in prepare_contrasts(self.inputs.contrast_info, names):
# Need to add F-test support for intercept (more than one column)
# Currently only taking 0th column as intercept (t-test)
weights = weights[0]
if all(weights == np.zeros(len(names))):
continue
input = (np.array(filtered_files)[weights != 0]).tolist()
design_matrix = pd.DataFrame({'intercept': weights[weights != 0]})
effigies
tyarkoni
adelavega
yarikoptic
mgxd
(migrating this from pybids where I falsely posted it before) pinging @effigies (@yarikoptic promised me to give you a hydra account that I can finally share the data).
We've stripped our dataset to a bare minimum: 3 subjects, each with 2 runs, and a model file with only one event in X to reduce computation time. You can find the dataset on hydra in the directory
/data/movieloc/backup_store/publish-BIDSSaccon branchslim(please disregard the master branch for now - it is a couple of weeks behind and I don't want to push the currently rather messy state of it). The model (found undermodels/movel_v3_smdl_autocon_splitconv_oneevent.jsonis evoked withfitlins . Jan_28 'run' -m models/movel_v3_smdl_autocon_splitconv_oneevent.json --desc highpass --space 'MNI152NLin6Sym' -d $PWD -w 'Jan_28_wd' --n-cpus 3Click here to expand our model
```json { "name": "FEF_localizer", "Input": { "session": "movie" }, "Steps": [ { "Level": "run", "Model": { "X": [ "amplitude_.RIGHT" ] }, "Contrasts": [], "AutoContrasts": true, "Transformations": [{ "Name": "Split", "Input": ["amplitude_"], "By": ["trial_type"] }, { "Name": "Convolve", "Input": ["amplitude_.RIGHT"], "Model": "spm" }] }, { "Level": "subject", "AutoContrasts": true }, { "Level": "dataset", "AutoContrasts": true }] } ```The model fails on level 3 (dataset) with
after producing seemingly sensible output on the first (run) and second (subject) level. As far as I can judge, the files that the level 3 model should ingest have been created.
Here is the traceback
```python Traceback (most recent call last): File "/home/adina/Repos/nipype/nipype/pipeline/plugins/multiproc.py", line 69, in run_node result['result'] = node.run(updatehash=updatehash) File "/home/adina/Repos/nipype/nipype/pipeline/engine/nodes.py", line 473, in run result = self._run_interface(execute=True) File "/home/adina/Repos/nipype/nipype/pipeline/engine/nodes.py", line 1254, in _run_interface self.config['execution']['stop_on_first_crash']))) File "/home/adina/Repos/nipype/nipype/pipeline/engine/nodes.py", line 1176, in _collate_results (self.name, '\n'.join(msg))) Exception: Subnodes of node: l3_model failed: Subnode 0 failed Error: Traceback (most recent call last): File "/home/adina/Repos/nipype/nipype/pipeline/engine/utils.py", line 99, in nodelist_runner result = node.run(updatehash=updatehash) File "/home/adina/Repos/nipype/nipype/pipeline/engine/nodes.py", line 473, in run result = self._run_interface(execute=True) File "/home/adina/Repos/nipype/nipype/pipeline/engine/nodes.py", line 557, in _run_interface return self._run_command(execute) File "/home/adina/Repos/nipype/nipype/pipeline/engine/nodes.py", line 637, in _run_command result = self._interface.run(cwd=outdir) File "/home/adina/Repos/nipype/nipype/interfaces/base/core.py", line 369, in run runtime = self._run_interface(runtime) File "/home/adina/Repos/fitlins/fitlins/interfaces/nistats.py", line 174, in _run_interface model.fit(input, design_matrix=design_matrix) File "/home/adina/env/fitlins/local/lib/python3.5/site-packages/nistats/second_level_model.py", line 164, in fit raise ValueError('A second level model requires a list with at' ValueError: A second level model requires a list with atleast two first level models or niimgs ```Reproducibility sits in a bar somewhere and laughs its ass of, but I'm trying to also give an overview of custom changes @yarikoptic and I made to the fitlins sourcecode. I don't see any particular relevance of the changes we made to the issue at hand (mostly hardcoding quickfixes for issues that erose), but then again, I'm certainly not the one to judge what is of relevance and what not ;-) , and an attempt to rerun the model would fail without the additional space choice and the selection of only one of two identical bold files being returned.
pybids changes
```patch diff --git a/fitlins/cli/run.py b/fitlins/cli/run.py index 28dcdba..f64e14e 100755 --- a/fitlins/cli/run.py +++ b/fitlins/cli/run.py @@ -85,15 +85,15 @@ def get_parser(): g_bids.add_argument('--derivative-label', action='store', type=str, help='execution label to append to derivative directory name') g_bids.add_argument('--space', action='store', - choices=['MNI152NLin2009cAsym', ''], + choices=['MNI152NLin2009cAsym', '', 'MNI152NLin6Sym'], default='MNI152NLin2009cAsym', help='registered space of input datasets. Empty value for no explicit space.') diff --git a/fitlins/interfaces/bids.py b/fitlins/interfaces/bids.py index 919742f..a7b98a9 100644 --- a/fitlins/interfaces/bids.py +++ b/fitlins/interfaces/bids.py @@ -177,7 +177,7 @@ class LoadBIDSModel(SimpleInterface): selectors = self.inputs.selectors analysis = Analysis(model=self.inputs.model, layout=layout) - analysis.setup(drop_na=False, desc='preproc', **selectors) + analysis.setup(drop_na=False, desc='highpass', space='MNI152NLin6Sym', **selectors) self._load_level1(runtime, analysis) self._load_higher_level(runtime, analysis) @@ -198,25 +198,37 @@ class LoadBIDSModel(SimpleInterface): [...] - if len(preproc_files) != 1: - raise ValueError('Too many BOLD files found') + # ATM we could get multiple entries for the same file + # see https://github.com/bids-standard/pybids/issues/350 + if len(set(f.path for f in preproc_files)) != 1: + raise ValueError( + 'Too many (%d) BOLD files found: %s' + % (len(preproc_files), ', '.join(preproc_files)) + ) fname = preproc_files[0].path ```Do you have any idea what I am missing here to figure out why the dataset level does not work?