Open tib-sim opened 10 months ago
I have not used it myself slurm, maybe SGE long time ago. I also have not used it through containers. Not sure if anyone else has done it.
You probably need to go step by step, make sure it works with singularity in a regular shell instead of slurm job. Your script must point to the container folders you are binding. If you manage to get it to work, sending slurm jobs will be a natural extension.
Out of curiosity, I am a big fan of containers for reproducibility purposes, but why are you trying to go through containers in this case? Might as well call the jobs through a regular Linda installation in the hpc.
Hi - Thanks for the prompt reply.
I understood my mistake the line srun singularity run --cleanenv --bind ${WD}:/home ${LINDA_SINGULARITY_IMG} Rscript SCRIPT_R
has to be edited as follows srun singularity run --cleanenv --bind ${WD}:/home ${LINDA_SINGULARITY_IMG} Rscript ${SCRIPT_R}
Actually I am using the container because R is not installed on the hpc and seems it takes some time to the IT team of my instution to install it. I did not want to wait too long before starting to play around with LINDA (looks like a very nice tool btw!). That is the main reason. Also with all the admin rights I am not sure how easy it would be to install R libraries so it is possible I'll stick with the container because it will make my life easier.
Also, I saw in an other issue raised the possibility that LINDA could become a BIDS-app, is there any update on this side?
Best, Simon
No, sorry, I have not planned to work on the bids app for now, but will see about it later.
Fyi, R packages do not require admin privileges, they will be installed in your home folder. You may not even need admin to install R, but I am not sure of that.
Please let the community know imhere if you make it work, so others can learn.
On Sun, Sep 3, 2023, 12:52 PM Simon Thibault @.***> wrote:
Hi - Thanks for the prompt reply.
I understood my mistake the line srun singularity run --cleanenv --bind ${WD}:/home ${LINDA_SINGULARITY_IMG} Rscript SCRIPT_R has to be edited as follows srun singularity run --cleanenv --bind ${WD}:/home ${LINDA_SINGULARITY_IMG} Rscript ${SCRIPT_R}```
Actually I am using the container because R is not installed on the hpc and seems it takes some time to the IT team of my instution to install it. I did not want to wait too long before starting to play around with LINDA (looks like a very nice tool btw!). That is the main reason. Also with all the admin rights I am not sure how easy it would be to install R libraries so it is possible I'll stick with the container because it will make my life easier.
Also, I saw in an other issue raised the possibility that LINDA could become a BIDS-app, is there any update on this side?
Best, Simon
— Reply to this email directly, view it on GitHub https://github.com/dorianps/LINDA/issues/36#issuecomment-1704351678, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACFJU7IK5UOBT6PEUTLY56LXYSYVJANCNFSM6AAAAAA4IVH43U . You are receiving this because you commented.Message ID: @.***>
One more question. Where did you find the singularity image? Converting from docker typically requires admin privileges.
On Sun, Sep 3, 2023, 12:52 PM Simon Thibault @.***> wrote:
Hi - Thanks for the prompt reply.
I understood my mistake the line srun singularity run --cleanenv --bind ${WD}:/home ${LINDA_SINGULARITY_IMG} Rscript SCRIPT_R has to be edited as follows srun singularity run --cleanenv --bind ${WD}:/home ${LINDA_SINGULARITY_IMG} Rscript ${SCRIPT_R}```
Actually I am using the container because R is not installed on the hpc and seems it takes some time to the IT team of my instution to install it. I did not want to wait too long before starting to play around with LINDA (looks like a very nice tool btw!). That is the main reason. Also with all the admin rights I am not sure how easy it would be to install R libraries so it is possible I'll stick with the container because it will make my life easier.
Also, I saw in an other issue raised the possibility that LINDA could become a BIDS-app, is there any update on this side?
Best, Simon
— Reply to this email directly, view it on GitHub https://github.com/dorianps/LINDA/issues/36#issuecomment-1704351678, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACFJU7IK5UOBT6PEUTLY56LXYSYVJANCNFSM6AAAAAA4IVH43U . You are receiving this because you commented.Message ID: @.***>
Ok thanks for the information, I am still not super familiar with the hpc. Actually in my former institution I was not allowed to use sudo
on the hpc. It looks like I have some admin rights on the hpc of my current institution so maybe it is possible to install R on my own.
Given I have been able to use batch processing to run 2 subjects in parallel on the hpc I think I will stick using the containers.
To get the image I ran the command singularity pull docker://dorianaps/linda:latest
. I don't think I need admin right to get it. At least I was able to get singularity images on the hpc of my former institution as well. Maybe they restricted the use of sudo
only. I am not so sure to be honest.
Hi - I wonder if someone already tried to used LINDA on singularity and use the BATCH processsing with slurm?
I tried the slurm code below with a R script encompassing the required lines of command to start LINDA.
So far, I have an error saying that the file does not exist while I have specified the right path.