hal2assemblyHub.py creates empty directory lacking hub contents

[malcook]

I find that running

hal2assemblyHub.py      ./test/blanchette00/blanchette00.hal ./test/blanchette00/blanchette00.hub   

creates an empty hub directory if it is run in the same directory in which runProgressiveCactus.sh has been previously run

Workarounds:

• run it from a different directory
• OR deleting the jobTree directory

I expect this is not by design. Or, perhaps, the jobTree is supposed to be deleted upon successful job completion, but something is keeping this from happening? Perhaps PWD is NFS mounted storage? This is a reach.

[joelarmstrong]

This is a "feature" of the jobTree framework. It tries to resume a run if the jobTree already exists, but you're right, it doesn't properly delete it on completion, which causes weird problems like this. The restart functionality wasn't really thought through all the way.

Sadly jobTree was pretty much abandoned soon after all these tools were written, so none of this ever got fixed. Its successor is toil, which did think some of these things through, but we haven't gone through and converted the hal tools yet.

That said, this should be fairly easy to patch. I'll see if I can make a quick fix to delete the jobTree on completion (without breaking everything :)).

[malcook]

Hi Joel,

My workaround for now is just to rename it by adding a datestamp, but, I have no idea whether I should expect any future value in keeping it around, once the job is completed. Sounds like not. Thanks for the confirmation/attention. This is not urgent at all for me.

That said, if you have any advice or insight for me regarding my intent to use progressiveCactus to establish conservation between Killifish and a few other teleosts, I’d be quite obliged for that.

Thanks again,

~malcolm_cook@stowers.org

From: Joel Armstrong [mailto:notifications@github.com] Sent: Monday, October 09, 2017 10:56 AM To: glennhickey/progressiveCactus progressiveCactus@noreply.github.com Cc: Cook, Malcolm MEC@stowers.org; Author author@noreply.github.com Subject: Re: [glennhickey/progressiveCactus] hal2assemblyHub.py creates empty directory lacking hub contents (#90)

This is a "feature" of the jobTreehttps://github.com/benedictpaten/jobTree framework. It tries to resume a run if the jobTree already exists, but you're right, it doesn't properly delete it on completion, which causes weird problems like this. The restart functionality wasn't really thought through all the way.

Sadly jobTree was pretty much abandoned soon after all these tools were written, so none of this ever got fixed. Its successor is toilhttps://github.com/BD2KGenomics/toil, which did think some of these things through, but we haven't gone through and converted the hal tools yet.

That said, this should be fairly easy to patch. I'll see if I can make a quick fix to delete the jobTree on completion (without breaking everything :)).

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/glennhickey/progressiveCactus/issues/90#issuecomment-335200741, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AAdjugUGIDTJ-pmQbJ-rdKu5FRefZafeks5sqkIJgaJpZM4Pyl6X.