search

GSTT-CSC / XNAT

Tracks general documentation, standard operating procedures (SOP) and helper scripts for XNAT.
15 stars 0 forks source link

Adds early version of iFIND Bash script #19

Closed heyhaleema closed 2 years ago

heyhaleema commented 2 years ago

This PR:

Due to recent IT system issues, I'm unable to access the latest version on the head node so I will update this as soon as possible.

* @dangerdika Since we sometimes have project-specific processes, I'm wondering if it might be helpful to separate the various tools (e.g. Python scripts, Bash scripts, etc.) into these folders rather than by function (e.g. anonymisation, query, etc.) - what do you think?

dangerdika commented 2 years ago

@heyhaleema I think by function makes it easier to re-use in the future, because after we hit like 50 projects we won't remember which tool is where anymore. Let's discuss at the meeting tomorrow and record the decision made here?

hshuaib90 commented 2 years ago

hey both, just wading in here. It would be good to get more eyes to review this design decision. Maybe post here the proposal.

heyhaleema commented 2 years ago

Hey @hshuaib90, @dangerdika and I discussed this in more depth, and we're in agreement that it'd be better to keep the folder structure as-is, i.e. separated by function, for future reusability.

Long-term, I'm thinking the folder structure* would look like:

XNAT/
├─ query/
├─ ingestion/
├─ anonymisation/
├─ export/
├─ project-processing/
├─ assets/
│  ├─ anon-flowchart.jpg
├─ src/
│  ├─ init.py
├─ .gitignore
├─ README.md

* Shout-out to this tree generator tool.

The distinction between the query and ingestion folders is that the former would be used more for cohort selection, e.g. from CogStack, SQL Server, etc. and tie-in with NDOO, and the latter would be the actual data ingestion.

I'm thinking export would just contain helper functions, e.g. file/directory compression, and project-processing as a miscellaneous folder for project-specific processing scripts, e.g. iFIND Bash script.

heyhaleema commented 2 years ago

I've now updated the script to the latest version* being used.

* The last step to zip the folders is currently commented out, though, whilst we (1) find out why there are different folder structures in the source data and (2) work on updating the script to appropriately handle these cases.