search

sina-mansour / UKB-connectomics

This repository will host scripts used to map structural and functional brain connectivity matrices for the UK biobank dataset.
https://www.biorxiv.org/content/10.1101/2023.03.10.532036v1
62 stars 7 forks source link

Scratch file system usage #2

Closed sina-mansour closed 2 years ago

sina-mansour commented 2 years ago

Following on the suggestion by @Lestropie (this commit):

RS: If you can guarantee adequate space, creating the scratch directory in /tmp/ would result in using a RAM file system, which will be way faster than a shared network filesystem. For re-executing data for which some files are pre-existing, you would need to explicitly manage the scratch directory location. My Python API provides command-line options for doing this.

sina-mansour commented 2 years ago

Hi Rob,

The UKB connectivity mapping pipeline script accepts input for a temporary file storage directory. The automation script uses this input to write all temporary files on the scratch space. This ensures that the code uses the fast RAM file system when possible.

Lestropie commented 2 years ago

Cool; as long as it's a performance issue that you're aware of and have accounted for.