search

AMP-SCZ / dpanonymize

Personally identifiable information remover
Apache License 2.0
0 stars 0 forks source link

Kcho/update path detection to use regrex #15

Closed kcho closed 3 years ago

kcho commented 3 years ago

I have noticed that lochness.XNAT module will download data to a subdirectory (eg- 2001_MR1) under mri directory in the phoenix structure. It could also be the case that other teams may add extra subdirectories under box and mediaflux as well. To account for this extra level of directories under the datatype directory, I've updated FileInPhoenixBIDS and FileInPhoenix classes to use regrex to be more flexible under these situations. Tested using pytest test/dpanonymize/test_dpanonymize.py.

Thanks.

tashrifbillah commented 3 years ago

Clarifiying discussion

It could also be the case that other teams may add extra subdirectories under box and mediaflux as well. To account for this extra level of directories under the datatype directory

In regard to your description, I am trying to understand the need for this new code:

https://github.com/AMP-SCZ/dpanonymize/blob/fa99674544b375a1aa3145be56ade86bcc6986d5/dpanonymize/__init__.py#L29-L45

The above code correctly detects processed, study, subject, and data groups for both beginning and ending extra paths:

file_path="/Some/Path/EXTRA/PROTECTED/BWH/raw/subject01/actigraphy/file"
In [98]: search.group('processed')
Out[98]: 'raw'

In [99]: search.group('study')
Out[99]: 'BWH'

In [100]: search.group('dtype')
Out[100]: 'actigraphy'

In [101]: search.group('subject')
Out[101]: 'subject01'
file_path="/Some/Path/to/PROTECTED/BWH/raw/subject01/actigraphy/EXTRA/file"
In [105]: search.group('subject')
Out[105]: 'subject01'

In [106]: search.group('dtype')
Out[106]: 'actigraphy'

In [107]: search.group('study')
Out[107]: 'BWH'

In [108]: search.group('processed')
Out[108]: 'raw'

Did you observe the same? If yes, the description should be updated as:

It could also be the case that other teams may add extra subdirectories under box/mediaflux and datatype directories. To account for this extra level of beginning and ending directories ...

kcho commented 3 years ago

It's to support extra folders under the datatype root directory, which is already the case for the data pulled by XNAT. eg) lochness.XNAT puts another folder under mri , eg) raw/subject01/mri/2001_MR.

Doc string has been updated to add a little more elaboration b31a9f3

kcho commented 3 years ago

Please consider merging my suggestion #16 and we are good to go. Thank you.

Merged. Thanks