search

sina-mansour / UKB-connectomics

This repository will host scripts used to map structural and functional brain connectivity matrices for the UK biobank dataset.
https://www.biorxiv.org/content/10.1101/2023.03.10.532036v1
62 stars 7 forks source link

Total files quota #35

Closed sina-mansour closed 2 years ago

sina-mansour commented 2 years ago

A limiting factor for running jobs on spartan is the total number of files saved. This is mainly due to inode limitations enforced by HPC admin. In order to bypass this limitation, the plan is to appropriately compress the computed files that we aim to save. Essentially, three sets of files related to: "functional connectomes", "structural connectomes", and "native brain atlases" should be saved into three different compressed archives. These three collections could then be uploaded as 3 separate bulk files to the UK biobank data storage.

sina-mansour commented 2 years ago

This new piece of code should do the job. I'll test and make sure.