Custom Scalars Error

[cornelius-eichner]

Dear AFQ developers,

We are currently running pyAFQ (python 3.8.8) in developmental data and would like to map custom tissue properties (qT1) to the reconstructed fibers. We followed the description on this website: https://yeatmanlab.github.io/pyAFQ/usage/scalars.html

When we try to incorporate the qT1 in the analysis pipeline, we get an error (attached):

The data are organized in a BIDS structure and the standard AFQ pipeline for FA and MD runs smoothly (AFQ config attached).

We tested and replicated this behavior in pyAFQ v0.9, v0.11, and v0.12.

Is there something we can do to circumvent this error? Thank you in advance for your help!

Best regards, Cornelius

AFQ Config:

    r1_scalar = ImageFile(
        suffix="R1_aligned",
        filters={"scope": "qmri"})

    # combine custom ROIs with default BundleDict ROIs
    #bundles = bundles + abd.BundleDict()

    tracking_params = dict(odf_model='CSD', directions='prob', n_seeds=3, step_size=0.5) 
    segmentation_params = dict(seg_algo='afq', prob_threshold=3)
    clean_params = dict(n_points=50, distance_threshold=3,min_sl=5)
    parallel_params = dict(n_jobs=8, engine='joblib', backend='loky')

    my_afq = GroupAFQ(
        bids_path,
        dmriprep='prepro',
        bundle_info = bundles,
        parallel_params = parallel_params,
        scalars=["dti_fa", "dti_md", "dti_rd", "dti_ad", r1_scalar],
        segmentation_params=segmentation_params,
        tracking_params=tracking_params,
        clean_params=clean_params)

    my_afq.export_all()

AFQ Error:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-2-663a074a9132> in <module>
----> 1 afq_config_synbrain_parallel_test_R1.afq_bids_run_all()

/data/pt_02568/software/anaconda3/lib/python3.8/site-packages/AFQ/afq_config_synbrain_parallel_test_R1.py in afq_bids_run_all()
     93         parallel_params = dict(n_jobs=8, engine='joblib', backend='loky')
     94 
---> 95     my_afq = GroupAFQ(
     96                 bids_path,
     97                 dmriprep='prepro',

/data/pt_02568/software/anaconda3/lib/python3.8/site-packages/AFQ/api/group.py in __init__(self, bids_path, bids_filters, preproc_pipeline, participant_labels, output_dir, parallel_params, bids_layout_kwargs, **kwargs)
    282                 self.valid_ses_list.append(session)
    283 
--> 284                 this_pAFQ = ParticipantAFQ(
    285                     dwi_data_file,
    286                     bval_file, bvec_file,

/data/pt_02568/software/anaconda3/lib/python3.8/site-packages/AFQ/api/participant.py in __init__(self, dwi_data_file, bval_file, bvec_file, output_dir, bids_info, **kwargs)
    109             plans = {  # Otherwise, do mapping first
    110                 "data": get_data_plan(kwargs),
--> 111                 "mapping": get_mapping_plan(kwargs),
    112                 "tractography": get_tractography_plan(kwargs),
    113                 "segmentation": get_segmentation_plan(kwargs),

/data/pt_02568/software/anaconda3/lib/python3.8/site-packages/AFQ/tasks/mapping.py in get_mapping_plan(kwargs, use_sls)
    249                 )
    250             else:
--> 251                 scalar.find_path(
    252                     bids_info["bids_layout"],
    253                     kwargs["dwi"],

/data/pt_02568/software/anaconda3/lib/python3.8/site-packages/AFQ/definitions/image.py in find_path(self, bids_layout, from_path, subject, session)
    132             self.fnames[session] = {}
    133 
--> 134         nearest_image = find_file(
    135             bids_layout, from_path, self.filters, self.suffix, session,
    136             subject)

/data/pt_02568/software/anaconda3/lib/python3.8/site-packages/AFQ/definitions/utils.py in find_file(bids_layout, path, filters, suffix, session, subject, extension)
    106         "subject", None
    107     )
--> 108     file_subject = bids_layout.parse_file_entities(nearest).get(
    109         "subject", None
    110     )

/data/pt_02568/software/anaconda3/lib/python3.8/site-packages/bids/layout/layout.py in parse_file_entities(self, filename, scope, entities, config, include_unmatched)
    440             config = list(set(config))
    441 
--> 442         return parse_file_entities(filename, entities, config,
    443                                    include_unmatched)
    444 

/data/pt_02568/software/anaconda3/lib/python3.8/site-packages/bids/layout/utils.py in parse_file_entities(filename, entities, config, include_unmatched)
     66 
     67     # Extract matches
---> 68     bf = make_bidsfile(filename)
     69     ent_vals = {}
     70     for ent in entities:

/data/pt_02568/software/anaconda3/lib/python3.8/site-packages/bids/utils.py in make_bidsfile(filename)
     97 
     98     patt = re.compile("[._]*[a-zA-Z0-9]*?(\\.[^/\\\\]+)$")
---> 99     m = re.search(patt, filename)
    100 
    101     ext = '' if not m else m.group(1)

/data/pt_02568/software/anaconda3/lib/python3.8/re.py in search(pattern, string, flags)
    199     """Scan through string looking for a match to the pattern, returning
    200     a Match object, or None if no match was found."""
--> 201     return _compile(pattern, flags).search(string)
    202 
    203 def sub(pattern, repl, string, count=0, flags=0):

TypeError: expected string or bytes-like object

In [3]: 

[36000]

Thank you @cornelius-eichner for using pyAFQ and making this bug report. I was able to reproduce this error. I believe the problem is that pyBIDS cannot find that image file. This likely a common error and the error message is super uninformative, so I need to fix that.

As for why pyBIDS cannot find the file, my guess is that the problem is the suffix has an " " in it. I think this confuses pyBIDS because the suffix is supposed to be everything after the last " " in the path. Try having the suffix be "aligned" and adding more filters if necessary, or renaming the files to a more unique suffix without the " _ ".

Let me know how it goes.

[arokem]

Did you have a chance to try this out, @cornelius-eichner ?

[cornelius-eichner]

Dear Ariel,

thank you so much for your reply and for your help! The renaming of files helped to solve the crash. So this issue can be closed.

Now that the initial crash is gone, I now encountered a second crash corresponding to the parallel processing of the custom scalars. However, I think it's better to post this to a separate issue.

Once again, thank you so much for your help!

Best, Cornelius

[arokem]

Thanks for reporting back! Please do let us know about your other crash on a separate issue.