search

AlexsLemonade / scpca-nf

scpca-nf is the Nextflow workflow for processing Single-cell Pediatric Cancer Atlas Portal data
BSD 3-Clause "New" or "Revised" License
12 stars 2 forks source link

Use more specialized docker images for processes #747

Closed jashapiro closed 2 months ago

jashapiro commented 4 months ago

At the moment, we are using a single scpcaTools image for all processes that call code from that package. That image includes all recommended dependencies for scpcaTools, and python packages including scvi-tools. This makes for a pretty large image, and most steps don't use most of the packages that are included.

Following https://github.com/AlexsLemonade/scpcaTools/pull/270, we should have the ability to make more focused images, with either minimal dependencies, or adding additional dependencies to support single tasks. These might include tasks like Seurat demux, AnnData export, or Rmarkdown report generation. We can define which set of extras are are needed for each process, and then create and use smaller images at each step as required.

jashapiro commented 4 months ago

Note that the base image name with https://github.com/AlexsLemonade/scpcaTools/pull/270 changed to scpcatools with no hyphen. The hyphens now designate the smaller subimages for specific tasks, such as scpcatools-slim or scpcatools-scvi.

jashapiro commented 2 months ago

calling this closed by #762