UCL-RITS / rcps-buildscripts

Scripts to automate package builds on RC Platforms
MIT License
39 stars 27 forks source link

FreeSurfer: Install version 6.0 [IN03605615] #266

Closed balston closed 4 years ago

balston commented 5 years ago

https://surfer.nmr.mgh.harvard.edu/

User needs to use it with Matlab.

owainkenwayucl commented 5 years ago

Note this information re: patch: https://github.com/UCL-RITS/research-computing/issues/305

balston commented 5 years ago

More users wanting 6.0: IN03754183

balston commented 5 years ago

There is no mention in the current FreeSurfer release notes about the need for the Matlab fix that was needed with FreeSurfer 5.3.0.

balston commented 5 years ago

Updating the FreeSurfer 5.3 build script for 6.0.

balston commented 5 years ago

A test download of the 6.0 installer archive (4.6 Gb) has taken over two hours! I'm updating the build script to use a pre-download copy from:

/shared/ucl/apps/pkg-store/
balston commented 5 years ago

Build script uploaded and run on Myriad. Now sorting out a module file and running some tests ...

balston commented 5 years ago

First test

mri_convert sample-001.mgz sample-001.nii.gz

fails with:

mri_convert.bin sample-001.mgz sample-001.nii.gz
$Id: mri_convert.c,v 1.226 2016/02/26 16:15:24 mreuter Exp $
reading from sample-001.mgz...
--------------------------------------------------------------------------
GNU libc version: 2.17
ERROR: Systems running GNU glibc version greater than 2.15
  require a newly formatted license file (it's free). Please
  download a new one from the following page:
  http://surfer.nmr.mgh.harvard.edu/registration.html
--------------------------------------------------------------------------

Obtaining new license file ...

balston commented 5 years ago

Filled in on-line form and I'm now waiting for the license email.

balston commented 5 years ago

New license file saved in:

~ccspapp/software/freesurfer/license-6.0.txt

on Myriad and Legion.

balston commented 5 years ago

New license file copied to:

/shared/ucl/apps/freesurfer/6.0.0/freesurfer/.license

The build script has also been updated to use the new license file.Running first test again ...

balston commented 5 years ago

First test:

mri_convert sample-001.mgz sample-001.nii.gz

now works:

mri_convert.bin sample-001.mgz sample-001.nii.gz
$Id: mri_convert.c,v 1.226 2016/02/26 16:15:24 mreuter Exp $
reading from sample-001.mgz...
TR=7.25, TE=3.22, TI=600.00, flip angle=7.00
i_ras = (-0, -1, -0)
j_ras = (-0, 0, -1)
k_ras = (-1, 0, 0)
writing to sample-001.nii.gz...
balston commented 5 years ago

Next test is to try running recon-all in a job.

balston commented 5 years ago

recon-all test job submitted and has started running.

balston commented 5 years ago

OK test job ran out of time after 1 hour so I've resubmitted it requesting 5 hours instead.

balston commented 5 years ago

Just realised that to run multi-threaded recon-all needs to be passed the option:

-openmp $NSLOTS

so updated my job script and re-submitted test again.

balston commented 5 years ago

This time my test job is definitely using the number of cores (only 4 so it started running quickly) I asked for.

balston commented 5 years ago

Module file for FreeSurfer 6 created and uploaded to Myriad.

balston commented 5 years ago

Here's the example job script I'm using:

#!/bin/bash -l

# Batch script to run a serial FreeSurfer job on Myriad

# FreeSurfer Version 6.0.0
# This version of FreeSurfer supports OpenMP

# 1. Force bash as the executing shell.
#$ -S /bin/bash

# 2. Request 5 hours of wallclock time (format hours:minutes:seconds).
#    Change this to suit your requirements.
#$ -l h_rt=5:00:0

# 3. Request 5G of RAM per core. Change this to suit your requirements.
#$ -l mem=5G

# 4. Set the name of the job. You can change this if you wish.
#$ -N FreeSurfer_job_1m

# 5. Select 4 threads. 
#$ -pe smp 4

# 7. Set the working directory to somewhere in your scratch space.  This is
# a necessary step with the upgraded software stack as compute nodes cannot
# write to your $HOME.
#
# NOTE: this directory must exist.
#
# Replace "<your_UCL_id>" with your UCL user ID :)
#$ -wd /home/<your_UCL_id>/Scratch/FreeSurfer_examples

# 8. Load FreeSurfer module and point to your subjects directory.

module load xorg-utils/X11R7.7
module load freesurfer/6.0.0
module list
export SUBJECTS_DIR=~/Scratch/FreeSurfer_examples/subjects

# 9. Run FreeSurfer programs - replace with your command(s)

cd $SUBJECTS_DIR
echo "Working in directory; `pwd`"
echo ""
recon-all -openmp $NSLOTS -i sample-001.nii.gz -s bert -all
balston commented 5 years ago

Both users requesting FreeSurfer 6 have now been informed.

Installing on Legion as well.

balston commented 5 years ago

Installed on Legion and test job submitted.

balston commented 5 years ago

My test job on Legion is running and appears to be working correctly.

balston commented 5 years ago

Legion job carried on working until it ran out of wallclock time.

Installation on Legion completed.