search

vgteam / toil-vg

Distributed and cloud computing framework for vg
Apache License 2.0
21 stars 14 forks source link

toil-vg call throws XGFormatError: Unimplemented XG format version: 0 #759

Open alancleary opened 5 years ago

alancleary commented 5 years ago

I have a graph that was built with Seqwish and augmented with reads mapped with GraphAligner. I used vg version 1.16.0 to build an xg for the augmented graph

vg index -x graph.aug.xg graph.aug.vg

I then used toil-vg with Singularity (which used vg version 1.11.0) to call variants

toil-vg call \
        --config config_wg.yaml \
        --realTimeLogging --logInfo --realTimeStderr \
        --workDir /workdir \
        /jobstore \
        graph.aug.xg \
        sample \
        /outstore \
        --gams reads.gam \
        --chroms 1

This resulted in an XGFormatError

Unimplemented XG format version: 0

Is this because I'm using a different version of vg than toil-vg, or could it be something else? Just looking for guidance on debugging really. Unfortunately, I can't share my data, but let me know if there's anything else I can tell you that would help. Much thanks!

adamnovak commented 5 years ago

I suspect that your problem is that between those vg versions we started packing the XG files in a new framing format. The older vg versions don't know how to unpack the framing format to get the actual version of the contained XG files, and the files no longer start with the "XG" magic number, so it gets reported as version 0.

You can probably unwrap the XG file into something that 1.11 might be able to read with vg view --extract-tag XG whatever.xg > whatever.unwrapped.xg. Unless there have been other changes to the XG format between those versions, that will work. If there have been other changes, it will at least be able to complain about the right XG version number.

You ought to be able to run toil-vg with newer versions of vg; it is part of the vg CI process. Is there a reason you are sticking with 1.11?

On 8/27/19, Alan Cleary notifications@github.com wrote:

I have a graph that was built with Seqwish and augmented with reads mapped with GraphAligner. I used vg version 1.16.0 to build an xg for the augmented graph

vg index -x graph.aug.xg graph.aug.vg

I then used toil-vg with Singularity (which used vg version 1.11.0) to call variants

toil-vg call \
        --config config_wg.yaml \
        --realTimeLogging --logInfo --realTimeStderr \
        --workDir /workdir \
        /jobstore \
        graph.aug.xg \
        sample \
        /outstore \
        --gams reads.gam \
        --chroms 1

This resulted in an XGFormatError

Unimplemented XG format version: 0

Is this because I'm using a different version of vg than toil-vg, or could it be something else? Just looking for guidance on debugging really. Unfortunately, I can't share my data, but let me know if there's anything else I can tell you that would help. Much thanks!

-- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/vgteam/toil-vg/issues/759

alancleary commented 5 years ago

Thanks for the quick reply, @adamnovak.

I installed toil-vg via pip. Looking at the package on PyPi, it apparently hasn't been updated since Nov 3, 2018. I was using the whole genome config file that toil-vg can generate which I now see is hard-coded to vg version 1.11.0 in this older version.

I'll just pull the master branch (which I see is hard-coded to use vg version 1.16.0) and build it locally.

Cheers!

adamnovak commented 5 years ago

@glennhickey We probably should do a new release. You have some changes to make for the new vg call; should we make a PyPI release after that?

glennhickey commented 5 years ago

I think it's worth doing a release now with the latest stable docker image or release in the config.

On Wed, Aug 28, 2019 at 4:35 PM Adam Novak notifications@github.com wrote:

@glennhickey https://github.com/glennhickey We probably should do a new release. You have some changes to make for the new vg call; should we make a PyPI release after that?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/vgteam/toil-vg/issues/759?email_source=notifications&email_token=AAG373WOIY7MZ2LP6PLXZCDQG3OSFA5CNFSM4IQOFUO2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5MMOQQ#issuecomment-525911874, or mute the thread https://github.com/notifications/unsubscribe-auth/AAG373VXPAM3JKTEFWTA2XDQG3OSFANCNFSM4IQOFUOQ .