It's me again. Hope you are doing well. I've noticed two things that I've had to modify locally that should be feasible to tweak in your original code to avoid some headaches.
First, you install some of your python dependencies within the Snakemake pipeline with "pip." This, unfortunately, is not possible in many HPCs, as we don't have access to / can modify "/root." I locally modified one of your 'yaml' files with dependencies so that all are installed with conda. This fixed the problem immediately. Before doing so, the pipeline was giving me a mysterious message about 'pandas' not being installed, which was honestly quite annoying to track down. Hopefully this will prevent the same issue from occurring to others.
Second, I am sequencing my samples with a an out-of-campus company. As a consequence, my samples are being ran in a sequencer in which multiple lanes are being used at the same time. The way in which the 'fake' files are being used for the bcl2fastq unfortunately crashes under this setup. That is because the sequencer was ran with multiple lanes, but the company only provides me with one or more of those lanes. What I am having to do is to modify your "step1_bcl2fast.smk" file to add the lane information to it. This is something similar to what CellRanger does. I'd suggest you modify the code in such a way that the user has the option to provide the sequencing line to the pipeline in the setup file, so that it is incorporated in the 'fake' file.
Hi, Joobie.
It's me again. Hope you are doing well. I've noticed two things that I've had to modify locally that should be feasible to tweak in your original code to avoid some headaches.
First, you install some of your python dependencies within the Snakemake pipeline with "pip." This, unfortunately, is not possible in many HPCs, as we don't have access to / can modify "/root." I locally modified one of your 'yaml' files with dependencies so that all are installed with conda. This fixed the problem immediately. Before doing so, the pipeline was giving me a mysterious message about 'pandas' not being installed, which was honestly quite annoying to track down. Hopefully this will prevent the same issue from occurring to others.
Second, I am sequencing my samples with a an out-of-campus company. As a consequence, my samples are being ran in a sequencer in which multiple lanes are being used at the same time. The way in which the 'fake' files are being used for the bcl2fastq unfortunately crashes under this setup. That is because the sequencer was ran with multiple lanes, but the company only provides me with one or more of those lanes. What I am having to do is to modify your "step1_bcl2fast.smk" file to add the lane information to it. This is something similar to what CellRanger does. I'd suggest you modify the code in such a way that the user has the option to provide the sequencing line to the pipeline in the setup file, so that it is incorporated in the 'fake' file.
Hope this helps.
Best,
P