TinasheMTapera
commented
4 years ago @isabelannwingert please paste the heuristic in as well
Closed isabelannwingert closed 4 years ago
TinasheMTapera
commented
4 years ago @isabelannwingert please paste the heuristic in as well
isabelannwingert
commented
4 years ago # NEW HEURISTIC PLANNING (NOTES)
### Pandas Dataframe import of two columns: UNIQUE Series-Description and {modality} label suffixes for BIDS filenames
### Create dictionary for modality labels
### These two objects are used to populate seq_info
# HEURISTIC_AUTOPOPULATE.py
### Isabel Ann Wingert & Tinashe M Tapera
##### isabel.wingert@pennmedicine.upenn.edu
##### Also available on the following GitHub Repositories
### 4th March 2020 v 1.0
# Importing Modalities and Packages for Use
import os # --- Working Directory Set-up
import pandas as pd # --- Pandas is a module that allows information from fw-heudiconv-tabulate to be passed
# create_key function for dictionary seq_info
def create_key(template, outtype=('nii.gz',), annotation_classes=None):
if template is None or not template:
raise ValueError('Template must be a valid format string')
return template, outtype, annotation_classes
from collections.abc import Mapping
def rec_key_replace(obj):
if isinstance(obj, Mapping):
return {key.replace('_', 'x').replace("-", "x"): rec_key_replace(val) for key, val in obj.items()}
return obj
os.chdir('/home/wingerti/wingert_workspace') ### Make sure Pandas DataFrame is in working folder
### Reading Unique Series Descriptions in with Pandas
df_series_desc = pd.read_excel('BIDSTestHUP3TLegacy_SeriesModalityInfo.xlsx', usecols="A")
series_desc = df_series_desc.values.tolist()
series_desc = [item for sublist in series_desc for item in sublist]
df_modality_labels = pd.read_excel('BIDSTestHUP3TLegacy_SeriesModalityInfo.xlsx', usecols="B")
modality_labels = df_modality_labels.values.tolist()
modality_labels = [item for sublist in modality_labels for item in sublist]
### Dictionary for Create Keys:
keys_series_desc = dict.fromkeys(series_desc, [])
key_modality_labels = dict.fromkeys(modality_labels, [])
### Dictionary for Modality Labels (by hand):
#df_modality_lookup =
#modality_lookup = {'T1w': 'T1w', 'T2w': 'T2w', 'FLAIR': 'FLAIR', 'bold': 'bold', 'dwi': 'dwi', 'localizer': 'localizer'}
ser_mod_pairs = dict(zip(series_desc, modality_labels))
ser_mod_pairs = rec_key_replace(ser_mod_pairs)
def infotodict(seqinfo):
global keys_series_desc
info = {}
for s in seqinfo:
protocol = s.protocol_name.lower()
series_desc = s.series_description.lower()
if series_desc in list(keys_series_desc.keys()):
keys_series_desc[series_desc].append(s.series_id)
print(keys_series_desc)
out_string = []
create_key_list = []
for key, values in keys_series_desc.items():
if key:
key = key.replace("_", "x",).replace("-", "x")
out_string_i = 'sub-{subject}_ses-{session}_acq-' + key
out_string.append(out_string_i)
for w in range(len(out_string)):
BIDS_filename = out_string[w] + '_' + modality_labels[w]
create_key_list.append(BIDS_filename)
for create_key_list_i in create_key_list:
new_key = create_key(create_key_list_i)
print(new_key)
return info, new_keyYou forgot to add the new_keys to info. Right now, your return statement is sending back a tuple of info and the key, but fw-heudiconv is only expecting one thing: a dictionary called info.
First, when you print(keys_series_desc) you get this:
{'3D_PASL': [], 'AAHead_Scout': [], 'AAHead_Scout_MPR_cor': [], 'AAHead_Scout_MPR_sag': [],That's good.
Then, when you print(new_key) you get this:
('sub-{subject}_ses-{session}_acq-3DxPASL_asl', ('nii.gz',), None)That looks good too. What's missing is to put those new_keys into info. I would recommend, in your last for loop:
for create_key_list_i in create_key_list:
new_key = create_key(create_key_list_i)
info[new_key] = [] # this creates the empty info dictNow you have an info object that you hacked. But the list of sequences still isn't full. So you have to go through your seqinfos again and add the seqinfo to the right place:
for s in seqinfo: # for each sequence, again
protocol = s.protocol_name.lower()
series_desc = s.series_description.lower()
for k, v in info.items(): # unpack the info dict that you created
if series_desc in k[0]: # if sequence name is in the key's string
info[k].append(s.series_id) # add it to infoThe end of your heuristic should look something like this:
def infotodict(seqinfo):
global keys_series_desc
info = {}
for s in seqinfo:
protocol = s.protocol_name.lower()
series_desc = s.series_description.lower()
if series_desc in list(keys_series_desc.keys()):
keys_series_desc[series_desc].append(s.series_id)
print(keys_series_desc)
out_string = []
create_key_list = []
for key, values in keys_series_desc.items():
if key:
key = key.replace("_", "x",).replace("-", "x")
out_string_i = 'sub-{subject}_ses-{session}_acq-' + key
out_string.append(out_string_i)
for w in range(len(out_string)):
BIDS_filename = out_string[w] + '_' + modality_labels[w]
create_key_list.append(BIDS_filename)
for create_key_list_i in create_key_list:
new_key = create_key(create_key_list_i)
info[new_key] = [] # this creates the empty info dict
for s in seqinfo: # for each sequence, again
protocol = s.protocol_name.lower()
series_desc = s.series_description.lower()
for k, v in info.items(): # unpack the info dict that you created
if series_desc in k[0]: # if sequence name is in the key's string
info[k].append(s.series_id) # add it to info
return info # only return infoAs always, if you're unsure, go into --dry-run mode and add as many print statements at every stage as you need
isabelannwingert
commented
4 years ago Hi @TinasheMTapera , thank you for the help, no bugs happen, I'm trying to figure out how to populate subject IDs and sessions in to the fields (everything is still being sorted into nonBids)
TinasheMTapera
commented
4 years ago @isabelannwingert in that case I'll close this issue just so we can keep track of numbers. You can re-open it if you're unsuccessful.
Hi @TinasheMTapera ,
I have solidified the heuristic that auto-populates keys per our goal.
It looks like there's a variable in the code
convert_to_bidsthat doesn't recognizeitemswithin the tuple object.Below is my output. Thanks again!