search

AlexsLemonade / alsf-scpca

Management and analysis tools for ALSF Single-cell Pediatric Cancer Atlas data.
BSD 3-Clause "New" or "Revised" License
0 stars 1 forks source link

Update Nextflow container image #169

Open arkid15r opened 2 years ago

arkid15r commented 2 years ago

Address the concern regarding:

I see that you changed this back, which may be fine. I realized that I needed to change the location of the aws cli in nextflow settings, which I had neglected to do, and might be having some effect, though I don't think this is the main problem. I worry some about having two version of the aws cli, one of which is presumably older and may be missing features?

https://github.com/AlexsLemonade/scpca-nf/blob/ef585fd1ab13ef24920a39579e5fb61c2c7a263c/config/profile_awsbatch.config#L5

One note is that according to the nextflow docs, the aws cli is mounted into the docker image in kind of a strange way, which could also cause trouble: https://www.nextflow.io/docs/latest/aws.html#aws-cli

https://github.com/AlexsLemonade/alsf-scpca/pull/168/files#diff-36dc947c782c6593c13829bb3ffaaf2422f2df60bcab504861f29a0de056aca1R36

arkid15r commented 2 years ago

Looking at the install location, it appears that the binary gets installed in /usr/local/aws-cli with symlinks to /usr/local/bin. Because of the way nextflow mounts the aws directory, we might be able to get away with making a link from /usr/local/aws-cli/aws to /usr/local/aws-cli/bin/aws. This will result in nextflow mounting all of /usr/local/aws-cli which should not clobber anything. (If we were to specify the path to nextflow as /usr/local/bin/aws it would try to mount all of /usr/local in the docker images, which would probably be bad).