Closed bentsherman closed 2 years ago
@spficklin I have moved all of the Palmetto-specific settings into a separate config file. Please review my remaining concern about the start_first_batch process.
@spficklin I think we've resolved everything. No more Palmetto-specific settings but a bunch of fixes and aesthetic improvements that I made while testing the pipeline on Palmetto. It's ready to merge.
Changes looked good and it ran fine on my end.... Merging!
This PR adds a number of tweaks that emerged as a result of me testing GEMmaker 2.0 on Palmetto. I will briefly describe them here:
Added apbspro
profile, similar to the oldpbs
profile, which provides a few basic settings specific to PBS Pro, including the Palmetto cluster which is used by many GEMmaker users.Changed thesingularity
profile to also mount the work/GEMmaker folder, which allows GEMmaker to work properly whenever the work directory resides outside of the launh directory.retrieve_sra_spots.py
which obtains the total number of spots in an SRA sample by parsing the metadata file. The spots is in turn saved as a trace directive indownload_runs
for the purpose of predicting resource usage. This script is very niche but it could also be generalized to extract any attribute from a metadata file.Added amax_jobs
param which controls the number of samples processed in parallel. Previously this behavior was controlled bymax_cpus
which already controls something else and was therefore confusing.multithreaded
label toprocess_medium
to take advantage of the new labels provided by nf-core.noclean
profile totest_noclean
for consistency with the other testing profiles.I apologize for the conglomeration of changes but it's a bunch of small changes so I figure it would be easier to hash them out in one thread. Let me know if I need to change anything to stay consistent with nf-core guidelines.