[ENH] PETXLSX Support

[bendhouseart]

Adding support for spreadsheets, creating PR for @marcelzwiers to see what I'm up to.

[bendhouseart]

I'm still uncertain as to how to match up spreadsheets with imaging data as they often aren't named even remotely close to each other, nor do the spreadsheets necessarily possess subject id or identifiers. It's really helter-skelter in PET land sometimes and it would be desirable that a user could match spreadsheets to PET imaging data via the bids editor.

That said, there's plenty of other testing and tooling for me to work on while I figure that out.

[marcelzwiers]

I'm still uncertain as to how to match up spreadsheets with imaging data as they often aren't named even remotely close to each other, nor do the spreadsheets necessarily possess subject id or identifiers. It's really helter-skelter in PET land sometimes and it would be desirable that a user could match spreadsheets to PET imaging data via the bids editor.

That's indeed tricky, because if they don't have a predictable name, how can we automate things? The user then needs to pick the excel for every subject/session, but bidsmaps map source datatypes to bids datatypes, so it is a shortlist of datatypes, not a longlist of sessions. If there is one excel sheet per datatype, then the user can import it, if the excel sheets varies per acquisition, then it needs to have a predictable name

[bendhouseart]

How well behaved will BIDSCoin be if we simply place a spreadsheet into the imaging folder? It might just be easier to have PET2BIDS do PET spreadsheet things if it detects a PET spreadsheet in or near the imaging folder. This won't be wholly transparent to the user, however any inputs from BIDSCoin (editor -> bidsmap) do already override spreadsheet input so idk....

[marcelzwiers]

How well behaved will BIDSCoin be if we simply place a spreadsheet into the imaging folder? It might just be easier to have PET2BIDS do PET spreadsheet things if it detects a PET spreadsheet in or near the imaging folder. This won't be wholly transparent to the user, however any inputs from BIDSCoin (editor -> bidsmap) do already override spreadsheet input so idk....

I'm not sure what you mean with imaging folder, but I suppose that's what I call the source folder. But to answer your question, BIDScoin will not notice the spreadsheet so it will behave as normal.

[bendhouseart]

BIDScoin will not notice the spreadsheet so it will behave as normal.

Ok, that might just be the easiest path forward.

[bendhouseart]

Alright, so I've integrated the changes you pushed to Donders-Institute/bidscoin pet-spreadsheets in, but I'm having trouble collecting the spreadsheet via _extatributes. It seems like an extra folder is getting inserted into the path, see:

PosixPath('/var/folders/5t/0cjjt4t10hz5_qk4tts9gkv40000gs/T/tmpzb0q6gbw/Users/galassiae/Projects/bidscoin/bidscointutorial/raw/sub-003/ses-01/30003-Phantom_PetAcquisition_AC Images/X-CAL_7.PT.Kalibrering_xca.30003.1.2022.04.26.15.04.22.218.xlsx')

Where the source data originates at:

`/Users/galassiae/Projects/bidscoin/bidscointutorial/raw/sub-003/ses-01/X-CAL_7.PT.Kalibrering_xca.30003.1.2022.04.26.15.04.22.218.xlsx'

raw.zip

bidsmap.yaml

```yaml # -------------------------------------------------------------------------------- # This is a template bidsmap with the key-value mappings for the different BIDS # datatypes (anat, func, dwi, etc). The datatype attributes are the keys that map # onto the BIDS labels. The bidsmap data-structure should be 5 levels deep: # # dict : dict : list : dict : dict # dataformat : datatype : run-item : bidsmapping : mapping-data # # NB: # 1) This bidsmap is intended to be used by the bidsmapper for data discovery. # Although generally not recommended, it can also be used directly by the bidscoiner # 2) This bidsmap is tailored to standard data acquisitions at the Donders Centre for # Cognitive Neuroimaging (DCCN). You may consider (Institute) users may create their # own bidsmap_[template].yaml or bidsmap_[sample].yaml file # # For more information, see: https://bidscoin.readthedocs.io # -------------------------------------------------------------------------------- Options: # -------------------------------------------------------------------------------- # General options and plugins # -------------------------------------------------------------------------------- bidscoin: version: 3.8.0 # BIDScoin version (should correspond with the version in ../bidscoin/version.txt) bidsignore: mrs/;extra_data/ # Semicolon-separated list of entries that are added to the .bidsignore file (for more info, see BIDS specifications), e.g. extra_data/;pet/;myfile.txt;yourfile.csv subprefix: sub- # The subject prefix of the source data sesprefix: ses- # The session prefix of the source data unknowntypes: [mrs, extra_data] # A list of datatypes that are converted to BIDS-like datatype folders ignoretypes: [exclude] # A list of datatypes that are excluded / not converted to BIDS unzip: # Wildcard pattern to select tarball/zip-files in the source folders that need to be unzipped (in a tempdir) to expose the data, e.g. '*.tar.gz' plugins: # List of plugins with plugin-specific key-value pairs (that can be used by the plugin) dcm2niix2bids: # See dcm2niix -h and https://www.nitrc.org/plugins/mwiki/index.php/dcm2nii:MainPage#General_Usage for more info command: dcm2niix # Command to run dcm2niix, e.g. "module add dcm2niix/1.0.20180622; dcm2niix" or "PATH=/opt/dcm2niix/bin:$PATH; dcm2niix" or /opt/dcm2niix/bin/dcm2niix or 'C:\"Program Files"\dcm2niix\dcm2niix.exe' (use quotes to deal with whitespaces in the path) args: -b y -z y -i n -l n # Argument string that is passed to dcm2niix. Tip: SPM users may want to use '-z n' (which produces unzipped NIfTI's, see dcm2niix -h for more information) anon: y # Set this anonymization flag to 'y' to round off age and discard acquisition date from the meta data meta: [.json, .tsv, .tsv.gz] # The file extensions of the equally named metadata sourcefiles that are copied over to the BIDS sidecar files # spec2nii2bids: # The settings for the spec2nii2bids plugin # command: spec2nii # Command to run spec2nii, e.g. "module add spec2nii; spec2nii" or "PATH=/opt/spec2nii/bin:$PATH; spec2nii" or /opt/spec2nii/bin/spec2nii or 'C:\"Program Files"\spec2nii\spec2nii.exe' (note the quotes to deal with the whitespace) # args: # Argument string that is passed to spec2nii (see spec2nii -h for more information) # anon: y # Set this anonymization flag to 'y' to round off age and discard acquisition date from the meta data # meta: [.json, .tsv, .tsv.gz] # The file extensions of the equally named metadata sourcefiles that are copied over to the BIDS sidecar files # multiraid: 2 # The mapVBVD argument for selecting the multiraid Twix file to load (default = 2, i.e. 2nd file) # pet2bids: # command: dcm2niix4pet # args: # anon: # meta: [.json, .tsv] pet2bids: command: dcm2niix4pet args: '' anon: y meta: - .json - .tsv - .xls - .xlsx DICOM: # -------------------------------------------------------------------------------- # DICOM key-value heuristics (DICOM fields that are mapped to the BIDS labels) # -------------------------------------------------------------------------------- subject: <> # This filesystem property extracts the subject label from the source directory. NB: Any property or attribute can be used as subject-label, e.g. session: <> # This filesystem property extracts the subject label from the source directory. NB: Any property or attribute can be used as session-label, e.g. anat: [] # ----------------------- All anatomical runs -------------------- func: [] # ----------------------- All functional runs -------------------- perf: [] # ----------------------- All perfusion runs --------------------- dwi: [] # ----------------------- All diffusion runs --------------------- fmap: [] # ----------------------- All fieldmap runs ---------------------- pet: [] # ----------------------- All PET runs --------------------------- eeg: [] # ----------------------- All EEG data --------------------------- ieeg: [] # ----------------------- All iEEG data -------------------------- extra_data: [] # ----------------------- All extra data ------------------------- exclude: [] # ----------------------- Data that will be left out ------------- PAR: # -------------------------------------------------------------------------------- # PAR key-value heuristics (Philips PAR fields that are mapped to the BIDS labels) # -------------------------------------------------------------------------------- subject: <> # This filesystem property extracts the subject label from the source directory. NB: Any property or attribute can be used as subject-label, e.g. session: <> # This filesystem property extracts the subject label from the source directory. NB: Any property or attribute can be used as session-label, e.g. anat: [] # ----------------------- All anatomical runs -------------------- func: [] # ----------------------- All functional runs -------------------- perf: [] # ----------------------- All perfusion runs --------------------- dwi: [] # ----------------------- All diffusion runs --------------------- fmap: [] # ----------------------- All fieldmap runs ---------------------- extra_data: [] # ----------------------- All extra data ------------------------- exclude: [] # ----------------------- Data that will be left out ------------- SPAR: # --------------------------------------------------------------------------------- # SPAR key-value heuristics (Philips MRS fields that are mapped to the BIDS labels) # --------------------------------------------------------------------------------- subject: <> # This filesystem property extracts the subject label from the source directory. NB: Any property or attribute can be used as subject-label, e.g. session: <> # This filesystem property extracts the subject label from the source directory. NB: Any property or attribute can be used as session-label, e.g. mrs: [] # ---------------------- All spectroscopy runs ------------------ extra_data: [] # ----------------------- All extra data ------------------------- exclude: [] # ----------------------- Data that will be left out ------------- Twix: # --------------------------------------------------------------------------------- # Twix key-value heuristics (Siemens MRS fields that are mapped to the BIDS labels) # --------------------------------------------------------------------------------- subject: <> # This filesystem property extracts the subject label from the source directory. NB: Any property or attribute can be used as subject-label, e.g. session: <> # This filesystem property extracts the subject label from the source directory. NB: Any property or attribute can be used as session-label, e.g. mrs: [] # ---------------------- All spectroscopy runs ------------------ extra_data: [] # ----------------------- All extra data ------------------------- exclude: [] # ----------------------- Data that will be left out ------------- Pfile: # ------------------------------------------------------------------------------ # P-file key-value heuristics (GE MRS fields that are mapped to the BIDS labels) # ------------------------------------------------------------------------------ subject: <> # This filesystem property extracts the subject label from the source directory. NB: Any property or attribute can be used as subject-label, e.g. session: <> # This filesystem property extracts the subject label from the source directory. NB: Any property or attribute can be used as session-label, e.g. mrs: [] # ---------------------- All spectroscopy runs ------------------ extra_data: [] # ----------------------- All extra data ------------------------- exclude: [] # ----------------------- Data that will be left out ------------- Nibabel: # -------------------------------------------------------------------------------- # Nibabel key-value heuristics (header fields that are mapped to the BIDS labels) # -------------------------------------------------------------------------------- subject: <> # This filesystem property extracts the subject label from the source directory. NB: Any property or attribute can be used as subject-label, e.g. session: <> # This filesystem property extracts the subject label from the source directory. NB: Any property or attribute can be used as session-label, e.g. anat: [] # ----------------------- All anatomical runs -------------------- func: [] # ----------------------- All functional runs -------------------- perf: [] # ----------------------- All perfusion runs --------------------- dwi: [] # ----------------------- All diffusion runs --------------------- fmap: [] # ----------------------- All fieldmap runs ---------------------- extra_data: [] # ----------------------- All extra data ------------------------- exclude: [] # ----------------------- Data that will be left out ------------- ECAT: # -------------------------------------------------------------------------------- # We use this to catalog ECAT files if they are preset # -------------------------------------------------------------------------------- subject: <> # This filesystem property extracts the subject label from the source directory. NB: Any property or attribute can be used as subject-label, e.g. session: <> pet: [] # ----------------------- All PET runs --------------------------- ```

[marcelzwiers]

A BIDScoin datasource() is an abstraction that reads source properties and, via plugins source attributes, and extended attributes. The get_attribute() function of pet2bids is already reading the excel attributes as normal attributes (next to the DICOM attributes), so the same data doesn't need to be read as extended attributes (you can just delete that code). So in this way, the excel attributes will be specific to pet2bids, and not available to all data formats. Users can enter the Excel key names (as dynamic values) in the bidseditor in the same way (transparently) anywhere in the bidsmap (e.g. in the bids table or in the meta table)

[marcelzwiers]

Sorry, the plugin doesn't seem to go forward anymore. You are are welcome to open a new one if you want to work on it again

[bendhouseart]

Hey no worries @marcelzwiers, I can't begrudge you for cleaning up your repository here on github and closing stale PR's.

Have been focused on everything else, apologies if it slowed you down.

Cheers and Happy Holidays!