cnobles / iGUIDE

Bioinformatic pipeline for identifying dsDNA breaks by marker based incorporation, such as breaks induced by designer nucleases like Cas9.
https://iguide.readthedocs.io/en/latest/
GNU General Public License v3.0
20 stars 9 forks source link

Argument list too long error in core_stat_matrix command #45

Closed ressy closed 5 years ago

ressy commented 5 years ago

I have a large number of .stat files as input to the core_stat_matrix rule, and when it tries to call collect_stats.R the shell returns an "argument list too long" error because of all the input files. I've made it work for my case by sourcing the core_stat code as a function and using run: R("...") directly in the rule. That works, but then you need an extra dependency installed in the environment too (rpy2) for Snakemake's R function to work, so it's a bit different from the current code.

Here's what I'm doing, for reference: https://github.com/cnobles/iGUIDE/compare/master...ressy:snakemake-rpy

ressy commented 5 years ago

Whoops, tests fail because I added rpy2 to my requirements but not the build. I'll check that out.

ressy commented 5 years ago

@cnobles this is the directory() thing I was trying to remember earlier, may help with that approach instead: https://snakemake.readthedocs.io/en/stable/snakefiles/rules.html#directories-as-outputs

cnobles commented 5 years ago

Resolved in #51.