search

KamitaniLab / bdpy

Python package for brain decoding analysis (BrainDecoderToolbox2 data format, machine learning analysis, functional MRI)
MIT License
33 stars 22 forks source link

Failed feature prediction when the multiple weight files in directory containing special characters. #90

Closed kencan7749 closed 1 month ago

kencan7749 commented 1 month ago

When I predicted the feature such as features[34] from brain activity, I encountered an error.

Traceback (most recent call last): File "/home/nu/ken.shirakawa/projects/python_KS/python/share_analysis/repro_koide_majima/scripts/feature-decoding/featdec_fastl2lir_predict.py", line 238, in featdec_fastl2lir_predict( File "/home/nu/ken.shirakawa/projects/python_KS/python/share_analysis/repro_koide_majima/scripts/feature-decoding/featdec_fastl2lir_predict.py", line 158, in featdec_fastl2lir_predict y_pred = test.run() ^^^^^^^^^^ File "/home/nu/ken.shirakawa/projects/python_KS/python/share_analysis/repro_koide_majima/.venv/lib/python3.11/site-packages/bdpy/ml/learning.py", line 613, in run return np.concatenate(y_pred_list, axis=self.chunk_axis) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ValueError: need at least one array to concatenate

I identified the cause as the directory containing special characters ("[", "]"). I want to deal it with the same way as Features class in bdpy. I'd like to ask you to add this line https://github.com/KamitaniLab/bdpy/blob/b6179eab45f9142f568dab565ab02f82993f876a/bdpy/dataform/features.py#L281 here around https://github.com/KamitaniLab/bdpy/blob/b6179eab45f9142f568dab565ab02f82993f876a/bdpy/ml/learning.py#L552.

kencan7749 commented 1 month ago

It works just by modifying these lines https://github.com/KamitaniLab/bdpy/blob/b6179eab45f9142f568dab565ab02f82993f876a/bdpy/ml/learning.py#L555-L577 to these:

elif self.model_format == 'bdmodel': 
    if os.path.isfile(self.model_path):
        raise ValueError('BDmodel should be specified as a directory, not a file')

    # W: shape = (n_voxels, shape_features)
    if os.path.isdir(os.path.join(self.model_path, 'W')):
        model_path = self.model_path.replace('[', '[[]')
        W_files = sorted(glob.glob(os.path.join(model_path, 'W', '*.mat')))
    elif os.path.isfile(os.path.join(self.model_path, 'W.mat')):
        W_files = [os.path.join(self.model_path, 'W.mat')]
    else:
        raise RuntimeError('W not found.')

    # b: shape = (1, shape_features)
    if os.path.isdir(os.path.join(self.model_path, 'b')):
        model_path = self.model_path.replace('[', '[[]')
        b_files = sorted(glob.glob(os.path.join(model_path, 'b', '*.mat')))
    elif os.path.isfile(os.path.join(self.model_path, 'b.mat')):
        b_files = [os.path.join(self.model_path, 'b.mat')]
    else:
        raise RuntimeError('b not found.')

    model_files = [(w, b) for w, b in zip(W_files, b_files)]
else

The differences are https://github.com/KamitaniLab/bdpy/blob/b6179eab45f9142f568dab565ab02f82993f876a/bdpy/ml/learning.py#L561 and https://github.com/KamitaniLab/bdpy/blob/b6179eab45f9142f568dab565ab02f82993f876a/bdpy/ml/learning.py#L569, allowing special character by replacing "[" into "[[]". BTW, we can consider replacing glob.escape instead.

ShuntaroAoki commented 1 month ago

https://github.com/KamitaniLab/bdpy/commit/5a4dca06892a7dc0eb178640b201b9511835e3d0