search

Kennedy-Lab-UW / Duplex-Seq-Pipeline

A standalone end-to-end data analysis pipeline for Duplex Sequencing
Other
21 stars 9 forks source link

Scheduler errors on Mac OS X Sierra #95

Open bkohrn opened 3 years ago

bkohrn commented 3 years ago

Situation: I did a new setup of the dev branch of the Duplex-Seq-Pipeline on a Mac OS X machine running Mac OS X Sierra v10.12.6 (not a clean install, unfortunately, but it's what I can work with from where I am). While attempting to run the test cases, I periodically get the following:

Traceback (most recent call last):
  File "/Users/loeblabm11/miniconda3/lib/python3.7/site-packages/snakemake/__init__.py", line 735, in snakemake
    keepincomplete=keep_incomplete,
  File "/Users/loeblabm11/miniconda3/lib/python3.7/site-packages/snakemake/workflow.py", line 972, in execute
    success = scheduler.schedule()
  File "/Users/loeblabm11/miniconda3/lib/python3.7/site-packages/snakemake/scheduler.py", line 406, in schedule
    else self.job_selector_ilp(needrun)
  File "/Users/loeblabm11/miniconda3/lib/python3.7/site-packages/snakemake/scheduler.py", line 625, in job_selector_ilp
    prob.solve()
  File "/Users/loeblabm11/miniconda3/lib/python3.7/site-packages/pulp/pulp.py", line 1737, in solve
    status = solver.actualSolve(self, **kwargs)
  File "/Users/loeblabm11/miniconda3/lib/python3.7/site-packages/pulp/apis/glpk_api.py", line 81, in actualSolve
    raise PulpSolverError("PuLP: Error while trying to execute "+self.path)
pulp.apis.core.PulpSolverError: PuLP: Error while trying to execute glpsol

I'm fairly sure this is a snakemake error, and I know I'm using v5.25.0 of snakemake. I'm currently inside the base conda environment on this system, which I don't really remember what I installed in it.

I apparently have the solver installed; it just isn't linked up correctly or something like that; not sure if there's anything I can do about it on my end since the installation is managed by mamba. I suppose it could be an issue with mamba, though.

This doesn't seem to happen in the most recent version of Snakemake, but the old solver issue (where the post-BLAST steps get scheduled before the BLAST steps) still seems to be happening there, so that doesn't help much (though I can get things to work by restarting the pipeline, since it doesn't stop entirely on the error and eventually gets to the BLAST step to provide the prerequisites for the post-BLAST steps).

For the present time, I would suggest avoiding Mac OS X if possible. I'm going to try and figure out what is happening and if there is anything I can do to fix it.

bkohrn commented 3 years ago

The other solver issue I mentioned is https://github.com/snakemake/snakemake/issues/771

bkohrn commented 3 years ago

Think we figured it out.

So what happened was that previously I was having issues with BLAST being rerun in rerun types where I didn't want it to be. To solve this, I applied the keyword ancient to the BLAST output file, which was described as "assume this file doesn't need regenerating", but which (at least when I put it in) would still generate the file if it didn't exist...except that in newer versions of Snakemake, the keyword ancient apparently isn't checking for file existence anymore as a prerequisite for running the rule it's used in.

I'll push a fix to this (and to updating to the latest Snakemake) in v3.0.0