search

BD2KGenomics / toil-lib

A common library for functions and tools used in toil-based pipelines
Apache License 2.0
4 stars 7 forks source link

Remove dockercall (resolves #68) #89

Closed fnothaft closed 7 years ago

fnothaft commented 7 years ago

This is just #78, but rebased to resolve the conflicts.

fnothaft commented 7 years ago

Jenkins, test this please.

jvivian commented 7 years ago

@fnothaft — Thanks a ton man!

fnothaft commented 7 years ago

Hmmm, even with the bump to 3.6.0 we're still having the deadlock issue.

fnothaft commented 7 years ago

JFC! It was just that the timeout parameter needed to be increased... I'll clean up the debugging commits tomorrow, and then I think this is good to go.

fnothaft commented 7 years ago

This is good to go from my side.

jvivian commented 7 years ago

fnothaft-meme

fnothaft commented 7 years ago

Thank you, @jvivian. I have taken the liberty of preparing remarks: if I may riff off of Newton, "If I have seen further, it is by using a more verbose level of logging."

I generally prefer pinning to a stable release.

fnothaft commented 7 years ago

Hi @cket! I just upgraded to the latest toil in PyPi (3.7.0a1.dev390) and am getting a runtime error with the nested services now:

RuntimeError: (RuntimeError("This job was passed a promise that wasn't yet resolved when it ran. The job 'SparkService' that fulfills this promise hasn't yet finished. This means that there aren't enough constraints to ensure the current job always runs after 'SparkService'. Consider adding a follow-on indirection between this job and its parent, or adding this job as a child/follow-on of 'SparkService'.",

I'm pretty sure that this is coming from the masterIP from the nested service job that is getting passed as a promise. I'm going to look and see if 1. the approach for using nested services changed or 2. if I can find the commit that broke this upstream in toil. I'll let you know. This should reproduce with the code that I have pushed, if you want to take a look on your end.

fnothaft commented 7 years ago

It seems like this issue came in somewhere between 3.7.0a1.dev389 and 3.7.0a1.dev390.

cket commented 7 years ago

@fnothaft that's not a good sign. If you don't find an easy solution feel free to open a ticket for this against Toil!

fnothaft commented 7 years ago

Actually, scratch that. It turns out it was an issue with my environment; not a bug in toil. Nevermind!

cket commented 7 years ago

Great, glad the issue is resolved!

fnothaft commented 7 years ago

This is good to merge from my side, @jvivian @cket