Error when sorting the graph using -Y option

[LiaOb21]

Dear developers, Thank you so much for this tool!

I created a pangenome of Arabidopsis following the minigraph-cactus pangenome pipeline. Then, I installed odgi using conda to visualise the pangenome graph. The output of the minigraph-cactus pipeline is a .gfa.gz file that was unizipped to create the .og file using the following command:

odgi build -g Arabidopsis_pg/Arabidopsis_pg.gfa -o odgi/Arabidopsis_pg.og -t 20 -P -O

I first tried to visualise the output as follows:

odgi viz -i odgi/Arabidopsis_pg.og -o odgi/Arabidopsis_pg.png

And this is the output:

Then, I tried to sort the graph, in order to see if a better visualisation is possible, but this is the error that I got using the -Y option for sorting:

odgi sort -i odgi/Arabidopsis_pg.og -t 20 -P -Y -o odgi/Arabidopsis_pg_sorted.og odgi: /opt/conda/conda-bld/odgi_1657286605938/work/build/sdsl-lite-prefix/src/sdsl-lite-build/include/sdsl/int_vector.hpp:1360: sdsl::int_vector<<anonymous> >::reference sdsl::int_vector<<anonymous> >::operator[](const size_type&) [with unsigned char t_width = 1; sdsl::int_vector<<anonymous> >::reference = sdsl::int_vector_reference<sdsl::int_vector<1> >; sdsl::int_vector<<anonymous> >::size_type = long unsigned int]: Assertionidx < this->size()' failed. Aborted (core dumped) `

When I try to randomly sort the graph (-r option), it works, but the output looks very strange: odgi sort -i odgi/Arabidopsis_pg.og -t 20 -O -P -r -o odgi/Arabidopsis_pg_random_sorted.og

Do you have any idea about what can be the issue? Am I missing something in the steps to follow?

Thank you so much in advance for any help!

Cheers,

Lia

[subwaystation]

Dear @LiaOb21,

Thanks for trying out ODGI and applying it to a minigraph + cactus GFA. I am actually not sure where the error is coming from. Could be an actual bug or a conda issue. I would need a full traceback from a tool like gdb https://www.sourceware.org/gdb/. It would be something like:

gdb odgi
run sort -i odgi/Arabidopsis_pg.og -t 20 -P -Y -o odgi/Arabidopsis_pg_sorted.og

Or would it be possible to somehow share the data?

Best, Simon

[LiaOb21]

Hi Simon,

Thank you so much for your quick reply!

Here the traceback:

(gdb) run sort -i odgi/Arabidopsis_pg.og -t 20 -P -Y -o odgi/Arabidopsis_pg_sorted.og
Starting program: path/to/miniconda3/bin/odgi sort -i odgi/Arabidopsis_pg.og -t 20 -P -Y -o odgi/Arabidopsis_pg_sorted.og
Missing separate debuginfos, use: yum debuginfo-install glibc-2.28-151.el8.x86_64
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
[New Thread 0x1554671ff700 (LWP 899803)]
[New Thread 0x15543e9ff700 (LWP 899804)]
[New Thread 0x15543e7fe700 (LWP 899805)]
[New Thread 0x15543e5fd700 (LWP 899806)]
[New Thread 0x15543e3fc700 (LWP 899807)]
[New Thread 0x15543e1fb700 (LWP 899808)]
[New Thread 0x15543dffa700 (LWP 899809)]
[New Thread 0x15543ddf9700 (LWP 899810)]
[New Thread 0x15543dbf8700 (LWP 899811)]
[New Thread 0x15543d9f7700 (LWP 899812)]
[New Thread 0x15543d7f6700 (LWP 899813)]
[New Thread 0x15543d5f5700 (LWP 899814)]
[New Thread 0x15543d3f4700 (LWP 899815)]
[New Thread 0x15543d1f3700 (LWP 899816)]
[New Thread 0x15543cff2700 (LWP 899817)]
[New Thread 0x15543cdf1700 (LWP 899818)]
[New Thread 0x15543cbf0700 (LWP 899819)]
[New Thread 0x15543c9ef700 (LWP 899820)]
[New Thread 0x15543c7ee700 (LWP 899821)]
[New Thread 0x15543c5ed700 (LWP 899822)]
[New Thread 0x15543c3ec700 (LWP 899823)]
[New Thread 0x15543c1eb700 (LWP 899824)]
[New Thread 0x15543bfea700 (LWP 899825)]
[New Thread 0x15543bde9700 (LWP 899826)]
[New Thread 0x15543bbe8700 (LWP 899827)]
[New Thread 0x15543b9e7700 (LWP 899828)]
[New Thread 0x15543b7e6700 (LWP 899829)]
[New Thread 0x15543b5e5700 (LWP 899830)]
[New Thread 0x15543b3e4700 (LWP 899831)]
[New Thread 0x15543b1e3700 (LWP 899832)]
[New Thread 0x15543afe2700 (LWP 899833)]
[New Thread 0x15543ade1700 (LWP 899834)]
[New Thread 0x15543abe0700 (LWP 899835)]
[New Thread 0x15543a9df700 (LWP 899836)]
[New Thread 0x15543a7de700 (LWP 899837)]
[New Thread 0x15543a5dd700 (LWP 899838)]
[New Thread 0x15543a3dc700 (LWP 899839)]
[New Thread 0x15543a1db700 (LWP 899840)]
[New Thread 0x155439fda700 (LWP 899841)]
[New Thread 0x155439dd9700 (LWP 899842)]
[New Thread 0x155439bd8700 (LWP 899843)]
[New Thread 0x1554399d7700 (LWP 899844)]
[New Thread 0x1554397d6700 (LWP 899845)]
[New Thread 0x1554395d5700 (LWP 899846)]
[New Thread 0x1554393d4700 (LWP 899847)]
[New Thread 0x1554391d3700 (LWP 899848)]
[New Thread 0x155438fd2700 (LWP 899849)]
[New Thread 0x155438dd1700 (LWP 899850)]
[New Thread 0x155438bd0700 (LWP 899851)]
[New Thread 0x1554389cf700 (LWP 899852)]
[New Thread 0x1554387ce700 (LWP 899853)]
[New Thread 0x1554385cd700 (LWP 899854)]
[New Thread 0x1554383cc700 (LWP 899855)]
[New Thread 0x1554381cb700 (LWP 899856)]
[New Thread 0x155437fca700 (LWP 899857)]
[New Thread 0x155437dc9700 (LWP 899858)]
[New Thread 0x155437bc8700 (LWP 899859)]
[New Thread 0x1554379c7700 (LWP 899860)]
[New Thread 0x1554377c6700 (LWP 899861)]
[New Thread 0x1554375c5700 (LWP 899862)]
[New Thread 0x1554373c4700 (LWP 899863)]
[New Thread 0x1554371c3700 (LWP 899864)]
[New Thread 0x155436fc2700 (LWP 899865)]
[New Thread 0x155436dc1700 (LWP 899866)]
[New Thread 0x155436bc0700 (LWP 899867)]
[New Thread 0x1554369bf700 (LWP 899868)]
[New Thread 0x1554367be700 (LWP 899869)]
[New Thread 0x1554365bd700 (LWP 899870)]
[New Thread 0x1554363bc700 (LWP 899871)]
[New Thread 0x1554361bb700 (LWP 899872)]
[New Thread 0x155435fba700 (LWP 899873)]
[New Thread 0x155435db9700 (LWP 899874)]
[Thread 0x1554671ff700 (LWP 899803) exited]
[Thread 0x155435db9700 (LWP 899874) exited]
[Thread 0x1554363bc700 (LWP 899871) exited]
[Thread 0x1554367be700 (LWP 899869) exited]
[Thread 0x1554379c7700 (LWP 899860) exited]
[Thread 0x155437bc8700 (LWP 899859) exited]
[Thread 0x1554381cb700 (LWP 899856) exited]
[Thread 0x1554383cc700 (LWP 899855) exited]
[Thread 0x155438dd1700 (LWP 899850) exited]
[Thread 0x155438fd2700 (LWP 899849) exited]
[Thread 0x1554393d4700 (LWP 899847) exited]
[Thread 0x1554399d7700 (LWP 899844) exited]
[Thread 0x155439dd9700 (LWP 899842) exited]
[Thread 0x15543a7de700 (LWP 899837) exited]
[Thread 0x15543afe2700 (LWP 899833) exited]
[Thread 0x15543b1e3700 (LWP 899832) exited]
[Thread 0x15543b3e4700 (LWP 899831) exited]
[Thread 0x15543c1eb700 (LWP 899824) exited]
[Thread 0x15543c3ec700 (LWP 899823) exited]
[Thread 0x1554375c5700 (LWP 899862) exited]
[Thread 0x15543b7e6700 (LWP 899829) exited]
[Thread 0x15543a9df700 (LWP 899836) exited]
[Thread 0x155436bc0700 (LWP 899867) exited]
[Thread 0x1554373c4700 (LWP 899863) exited]
[Thread 0x15543bbe8700 (LWP 899827) exited]
[Thread 0x1554397d6700 (LWP 899845) exited]
[Thread 0x155436fc2700 (LWP 899865) exited]
[Thread 0x155439bd8700 (LWP 899843) exited]
[Thread 0x15543abe0700 (LWP 899835) exited]
[Thread 0x15543a3dc700 (LWP 899839) exited]
[Thread 0x1554377c6700 (LWP 899861) exited]
[Thread 0x155437fca700 (LWP 899857) exited]
[Thread 0x15543b9e7700 (LWP 899828) exited]
[Thread 0x155437dc9700 (LWP 899858) exited]
[Thread 0x1554387ce700 (LWP 899853) exited]
[Thread 0x155438bd0700 (LWP 899851) exited]
[Thread 0x1554369bf700 (LWP 899868) exited]
[Thread 0x15543a5dd700 (LWP 899838) exited]
[Thread 0x15543bde9700 (LWP 899826) exited]
[Thread 0x1554389cf700 (LWP 899852) exited]
[Thread 0x15543bfea700 (LWP 899825) exited]
[Thread 0x1554385cd700 (LWP 899854) exited]
[Thread 0x15543ade1700 (LWP 899834) exited]
[Thread 0x1554361bb700 (LWP 899872) exited]
[Thread 0x15543a1db700 (LWP 899840) exited]
[Thread 0x1554391d3700 (LWP 899848) exited]
[Thread 0x1554371c3700 (LWP 899864) exited]
[Thread 0x15543b5e5700 (LWP 899830) exited]
[Thread 0x1554365bd700 (LWP 899870) exited]
[Thread 0x155436dc1700 (LWP 899866) exited]
[Thread 0x155439fda700 (LWP 899841) exited]
[Thread 0x155435fba700 (LWP 899873) exited]
[Thread 0x1554395d5700 (LWP 899846) exited]
odgi: /opt/conda/conda-bld/odgi_1657286605938/work/build/sdsl-lite-prefix/src/sdsl-lite-build/include/sdsl/int_vector.hpp:1360: sdsl::int_vector<<anonymous> >::reference sdsl::int_vector<<anonymous> >::operator[](const size_type&) [with unsigned char t_width = 1; sdsl::int_vector<<anonymous> >::reference = sdsl::int_vector_reference<sdsl::int_vector<1> >; sdsl::int_vector<<anonymous> >::size_type = long unsigned int]: Assertion `idx < this->size()' failed.

Thread 1 "odgi" received signal SIGABRT, Aborted.
0x000015555472537f in raise () from /lib64/libc.so.6

Let me know if this is enough, otherwise I can share the data with you. Thank you again!

Cheers,

Lia

[subwaystation]

Forgot to tell you to type bt in gdb after the program has finished. Then we will have the full traceback. Could you please also try your command with the current ODGI docker image? Instructions are at https://github.com/pangenome/odgi#docker=.

Thanks!

[subwaystation]

It is expected that you can't understand the graph, when it is randomly sorted. But using a randomly sorted graph as in input for our sorting algorithms, we can show they actually work.

[LiaOb21]

Hi Simon,

Here the traceback:

(gdb) run sort -i odgi/Arabidopsis_pg.og -t 20 -P -Y -o odgi/Arabidopsis_pg_sorted.og
Starting program: path/to/miniconda3/bin/odgi sort -i odgi/Arabidopsis_pg.og -t 20 -P -Y -o odgi/Arabidopsis_pg_sorted.og
Missing separate debuginfos, use: yum debuginfo-install glibc-2.28-151.el8.x86_64
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
[New Thread 0x1554671ff700 (LWP 922774)]
[New Thread 0x15543e9ff700 (LWP 922776)]
[New Thread 0x15543e7fe700 (LWP 922777)]
[New Thread 0x15543e5fd700 (LWP 922778)]
[New Thread 0x15543e3fc700 (LWP 922779)]
[New Thread 0x15543e1fb700 (LWP 922780)]
[New Thread 0x15543dffa700 (LWP 922781)]
[New Thread 0x15543ddf9700 (LWP 922782)]
[New Thread 0x15543dbf8700 (LWP 922783)]
[New Thread 0x15543d9f7700 (LWP 922784)]
[New Thread 0x15543d7f6700 (LWP 922785)]
[New Thread 0x15543d5f5700 (LWP 922786)]
[New Thread 0x15543d3f4700 (LWP 922787)]
[New Thread 0x15543d1f3700 (LWP 922788)]
[New Thread 0x15543cff2700 (LWP 922789)]
[New Thread 0x15543cdf1700 (LWP 922790)]
[New Thread 0x15543cbf0700 (LWP 922791)]
[New Thread 0x15543c9ef700 (LWP 922792)]
[New Thread 0x15543c7ee700 (LWP 922793)]
[New Thread 0x15543c5ed700 (LWP 922794)]
[New Thread 0x15543c3ec700 (LWP 922795)]
[New Thread 0x15543c1eb700 (LWP 922796)]
[New Thread 0x15543bfea700 (LWP 922797)]
[New Thread 0x15543bde9700 (LWP 922798)]
[New Thread 0x15543bbe8700 (LWP 922799)]
[New Thread 0x15543b9e7700 (LWP 922800)]
[New Thread 0x15543b7e6700 (LWP 922801)]
[New Thread 0x15543b5e5700 (LWP 922802)]
[New Thread 0x15543b3e4700 (LWP 922803)]
[New Thread 0x15543b1e3700 (LWP 922804)]
[New Thread 0x15543afe2700 (LWP 922805)]
[New Thread 0x15543ade1700 (LWP 922806)]
[New Thread 0x15543abe0700 (LWP 922807)]
[New Thread 0x15543a9df700 (LWP 922808)]
[New Thread 0x15543a7de700 (LWP 922809)]
[New Thread 0x15543a5dd700 (LWP 922810)]
[New Thread 0x15543a3dc700 (LWP 922811)]
[New Thread 0x15543a1db700 (LWP 922812)]
[New Thread 0x155439fda700 (LWP 922813)]
[New Thread 0x155439dd9700 (LWP 922814)]
[New Thread 0x155439bd8700 (LWP 922815)]
[New Thread 0x1554399d7700 (LWP 922816)]
[New Thread 0x1554397d6700 (LWP 922817)]
[New Thread 0x1554395d5700 (LWP 922818)]
[New Thread 0x1554393d4700 (LWP 922819)]
[New Thread 0x1554391d3700 (LWP 922820)]
[New Thread 0x155438fd2700 (LWP 922821)]
[New Thread 0x155438dd1700 (LWP 922822)]
[New Thread 0x155438bd0700 (LWP 922823)]
[New Thread 0x1554389cf700 (LWP 922824)]
[New Thread 0x1554387ce700 (LWP 922825)]
[New Thread 0x1554385cd700 (LWP 922826)]
[New Thread 0x1554383cc700 (LWP 922827)]
[New Thread 0x1554381cb700 (LWP 922828)]
[New Thread 0x155437fca700 (LWP 922829)]
[New Thread 0x155437dc9700 (LWP 922830)]
[New Thread 0x155437bc8700 (LWP 922831)]
[New Thread 0x1554379c7700 (LWP 922832)]
[New Thread 0x1554377c6700 (LWP 922833)]
[New Thread 0x1554375c5700 (LWP 922834)]
[New Thread 0x1554373c4700 (LWP 922835)]
[New Thread 0x1554371c3700 (LWP 922836)]
[New Thread 0x155436fc2700 (LWP 922837)]
[New Thread 0x155436dc1700 (LWP 922838)]
[New Thread 0x155436bc0700 (LWP 922839)]
[New Thread 0x1554369bf700 (LWP 922840)]
[New Thread 0x1554367be700 (LWP 922841)]
[New Thread 0x1554365bd700 (LWP 922842)]
[New Thread 0x1554363bc700 (LWP 922843)]
[New Thread 0x1554361bb700 (LWP 922844)]
[New Thread 0x155435fba700 (LWP 922845)]
[New Thread 0x155435db9700 (LWP 922846)]
[Thread 0x1554671ff700 (LWP 922774) exited]
[Thread 0x1554385cd700 (LWP 922826) exited]
[Thread 0x155436fc2700 (LWP 922837) exited]
[Thread 0x155437bc8700 (LWP 922831) exited]
[Thread 0x155437dc9700 (LWP 922830) exited]
[Thread 0x1554387ce700 (LWP 922825) exited]
[Thread 0x1554389cf700 (LWP 922824) exited]
[Thread 0x155438bd0700 (LWP 922823) exited]
[Thread 0x155438fd2700 (LWP 922821) exited]
[Thread 0x1554391d3700 (LWP 922820) exited]
[Thread 0x1554393d4700 (LWP 922819) exited]
[Thread 0x1554395d5700 (LWP 922818) exited]
[Thread 0x155439bd8700 (LWP 922815) exited]
[Thread 0x155439dd9700 (LWP 922814) exited]
[Thread 0x155439fda700 (LWP 922813) exited]
[Thread 0x15543abe0700 (LWP 922807) exited]
[Thread 0x15543b1e3700 (LWP 922804) exited]
[Thread 0x15543b5e5700 (LWP 922802) exited]
[Thread 0x15543bbe8700 (LWP 922799) exited]
[Thread 0x15543bde9700 (LWP 922798) exited]
[Thread 0x15543a7de700 (LWP 922809) exited]
[Thread 0x155435db9700 (LWP 922846) exited]
[Thread 0x15543a9df700 (LWP 922808) exited]
[Thread 0x15543a3dc700 (LWP 922811) exited]
[Thread 0x15543ade1700 (LWP 922806) exited]
[Thread 0x15543a5dd700 (LWP 922810) exited]
[Thread 0x15543a1db700 (LWP 922812) exited]
[Thread 0x155438dd1700 (LWP 922822) exited]
[Thread 0x15543b9e7700 (LWP 922800) exited]
[Thread 0x1554381cb700 (LWP 922828) exited]
[Thread 0x1554383cc700 (LWP 922827) exited]
[Thread 0x1554361bb700 (LWP 922844) exited]
[Thread 0x1554399d7700 (LWP 922816) exited]
[Thread 0x1554367be700 (LWP 922841) exited]
[Thread 0x155436bc0700 (LWP 922839) exited]
[Thread 0x15543b3e4700 (LWP 922803) exited]
[Thread 0x1554397d6700 (LWP 922817) exited]
[Thread 0x1554365bd700 (LWP 922842) exited]
[Thread 0x15543c3ec700 (LWP 922795) exited]
[Thread 0x1554379c7700 (LWP 922832) exited]
[Thread 0x1554373c4700 (LWP 922835) exited]
[Thread 0x15543bfea700 (LWP 922797) exited]
[Thread 0x1554369bf700 (LWP 922840) exited]
[Thread 0x15543c1eb700 (LWP 922796) exited]
[Thread 0x155435fba700 (LWP 922845) exited]
[Thread 0x155437fca700 (LWP 922829) exited]
[Thread 0x1554363bc700 (LWP 922843) exited]
[Thread 0x155436dc1700 (LWP 922838) exited]
[Thread 0x1554371c3700 (LWP 922836) exited]
[Thread 0x1554375c5700 (LWP 922834) exited]
[Thread 0x15543afe2700 (LWP 922805) exited]
[Thread 0x15543b7e6700 (LWP 922801) exited]
[Thread 0x1554377c6700 (LWP 922833) exited]
odgi: /opt/conda/conda-bld/odgi_1657286605938/work/build/sdsl-lite-prefix/src/sdsl-lite-build/include/sdsl/int_vector.hpp:1360: sdsl::int_vector<<anonymous> >::reference sdsl::int_vector<<anonymous> >::operator[](const size_type&) [with unsigned char t_width = 1; sdsl::int_vector<<anonymous> >::reference = sdsl::int_vector_reference<sdsl::int_vector<1> >; sdsl::int_vector<<anonymous> >::size_type = long unsigned int]: Assertion `idx < this->size()' failed.

Thread 1 "odgi" received signal SIGABRT, Aborted.
0x000015555472537f in raise () from /lib64/libc.so.6
(gdb) bt
#0  0x000015555472537f in raise () from /lib64/libc.so.6
#1  0x000015555470fdb5 in abort () from /lib64/libc.so.6
#2  0x000015555470fc89 in __assert_fail_base.cold.0 () from /lib64/libc.so.6
#3  0x000015555471da76 in __assert_fail () from /lib64/libc.so.6
#4  0x0000555555826ecb in xp::XP::from_handle_graph_impl(odgi::graph_t&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, unsigned long const&) ()
#5  0x00005555558271da in xp::XP::from_handle_graph(odgi::graph_t&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, unsigned long const&) ()
#6  0x000055555582731d in xp::XP::from_handle_graph(odgi::graph_t&, unsigned long const&) ()
#7  0x00005555557081fd in odgi::main_sort(int, char**) ()
#8  0x00005555556e3408 in odgi::subcommand::Subcommand::operator()(int, char**) const ()
#9  0x0000155554711493 in __libc_start_main () from /lib64/libc.so.6
#10 0x00005555555c0369 in _start ()

I will now try docker. Thank you!

Cheers,

Lia

[subwaystation]

Thanks! Looks like during the path index construction something goes out of place. Could be, because of the libc.so.6. Or could be a bug. It would be of great help, if you could share the data somehow. You can also drop an mail to simon.heumos@qbic.uni-tuebingen.de.

Curious, if the docker worked.

[ekg]

This would typically indicate some error in the path space or indexing of it.

Can you share the graph?

Did you convert it into P-line format with gfa-wp?

On Thu, Jul 14, 2022, 11:29 Simon Heumos @.***> wrote:

Thanks! Looks like during the path index construction something goes out of place. Could be, because of the libc.so.6. Or could be a bug. It would be of great help, if you could share the data somehow. You can also drop an mail to @.***

Curious, if the docker worked.

— Reply to this email directly, view it on GitHub https://github.com/pangenome/odgi/issues/430#issuecomment-1184216042, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABDQEICFTXQV2WA25LAMEDVT7MZBANCNFSM53O5ST7Q . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[LiaOb21]

Hi @ekg,

I just dropped an email with my .gfa.gz file to Simon. I didn't convert it into P-lines, probably this is what I am missing. I'm going to try now! Thank you so much!

[LiaOb21]

Hi @ekg and @subwaystation,

I am sorry, but I don't have really clear how to use gfa-wp. I also tried to use vg convert, how it was suggested in another issue:

vg convert ../Arabidopsis_pg/Arabidopsis_pg.gfa -o -t 20

But it is crashing and it doesn't produce any output (I installed vg tools using conda). I would like to try gfa-wp, could you please give me more informations?

Thank you so much!

Lia

[ekg]

You give gfa-wp the minigraph cactus output and it should convert it into GFAv1 format following PanSN naming and P-line for paths. This is compatible with odgi and other tools that use path lines (it's still compatible with VG as well).

Then you should be able to load it into odgi build. If the paths are in W-lines they won't load, but the format conversion is trivial.

On Thu, Jul 14, 2022, 14:06 LiaOb21 @.***> wrote:

Hi @ekg https://github.com/ekg and @subwaystation https://github.com/subwaystation,

I am sorry, but I don't have really clear how to use gfa-wp. I also tried to use vg convert, how it was suggested in another issue:

vg convert ../Arabidopsis_pg/Arabidopsis_pg.gfa -o -t 20

But it is crashing and it doesn't produce any output (I installed vg tools using conda). I would like to try gfa-wp, could you please give me more informations?

Thank you so much!

Lia

— Reply to this email directly, view it on GitHub https://github.com/pangenome/odgi/issues/430#issuecomment-1184367292, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABDQENMIH2OTT5C25WHHL3VT77D5ANCNFSM53O5ST7Q . You are receiving this because you were mentioned.Message ID: @.***>

[LiaOb21]

Hi,

I was finally able to use gfa-wp. It is probably easy for who is already familiar with this kind of tools, but It took a while to me to go through the documentation, install rust, and build the package, sorry.

Anyway, after running gfa-wp, and the following command on the converted .gfa file:

odgi build -g Arabidopsis_pg.gfa -o Arabidopsis_pg.og -t 20 -P -O

If I try again to sort the graph, I obtain the same error:

odgi sort -i Arabidopsis_pg.og --threads 20 -P -Y -o Arabidopsis_pg_sorted.og 

odgi: /opt/conda/conda-bld/odgi_1657286605938/work/build/sdsl-lite-prefix/src/sdsl-lite-build/include/sdsl/int_vector.hpp:1360: sdsl::int_vector<<anonymous> >::reference sdsl::int_vector<<anonymous> >::operator[](const size_type&) [with unsigned char t_width = 1; sdsl::int_vector<<anonymous> >::reference = sdsl::int_vector_reference<sdsl::int_vector<1> >; sdsl::int_vector<<anonymous> >::size_type = long unsigned int]: Assertion `idx < this->size()' failed.
Aborted (core dumped)

Traceback:

(gdb) run sort -i Arabidopsis_pg.og --threads 20 -P -Y -o Arabidopsis_pg_sorted.og
Starting program: path/to/miniconda3/bin/odgi sort -i Arabidopsis_pg.og --threads 20 -P -Y -o Arabidopsis_pg_sorted.og
Missing separate debuginfos, use: yum debuginfo-install glibc-2.28-151.el8.x86_64
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
[New Thread 0x1554671ff700 (LWP 932899)]
[New Thread 0x15543e9ff700 (LWP 932901)]
[New Thread 0x15543e7fe700 (LWP 932902)]
[New Thread 0x15543e5fd700 (LWP 932903)]
[New Thread 0x15543e3fc700 (LWP 932904)]
[New Thread 0x15543e1fb700 (LWP 932905)]
[New Thread 0x15543dffa700 (LWP 932906)]
[New Thread 0x15543ddf9700 (LWP 932907)]
[New Thread 0x15543dbf8700 (LWP 932908)]
[New Thread 0x15543d9f7700 (LWP 932909)]
[New Thread 0x15543d7f6700 (LWP 932910)]
[New Thread 0x15543d5f5700 (LWP 932911)]
[New Thread 0x15543d3f4700 (LWP 932912)]
[New Thread 0x15543d1f3700 (LWP 932913)]
[New Thread 0x15543cff2700 (LWP 932914)]
[New Thread 0x15543cdf1700 (LWP 932915)]
[New Thread 0x15543cbf0700 (LWP 932916)]
[New Thread 0x15543c9ef700 (LWP 932917)]
[New Thread 0x15543c7ee700 (LWP 932918)]
[New Thread 0x15543c5ed700 (LWP 932919)]
[New Thread 0x15543c3ec700 (LWP 932920)]
[New Thread 0x15543c1eb700 (LWP 932921)]
[New Thread 0x15543bfea700 (LWP 932922)]
[New Thread 0x15543bde9700 (LWP 932923)]
[New Thread 0x15543bbe8700 (LWP 932924)]
[New Thread 0x15543b9e7700 (LWP 932925)]
[New Thread 0x15543b7e6700 (LWP 932926)]
[New Thread 0x15543b5e5700 (LWP 932927)]
[New Thread 0x15543b3e4700 (LWP 932928)]
[New Thread 0x15543b1e3700 (LWP 932929)]
[New Thread 0x15543afe2700 (LWP 932930)]
[New Thread 0x15543ade1700 (LWP 932931)]
[New Thread 0x15543abe0700 (LWP 932932)]
[New Thread 0x15543a9df700 (LWP 932933)]
[New Thread 0x15543a7de700 (LWP 932934)]
[New Thread 0x15543a5dd700 (LWP 932935)]
[New Thread 0x15543a3dc700 (LWP 932936)]
[New Thread 0x15543a1db700 (LWP 932937)]
[New Thread 0x155439fda700 (LWP 932938)]
[New Thread 0x155439dd9700 (LWP 932939)]
[New Thread 0x155439bd8700 (LWP 932940)]
[New Thread 0x1554399d7700 (LWP 932941)]
[New Thread 0x1554397d6700 (LWP 932942)]
[New Thread 0x1554395d5700 (LWP 932943)]
[New Thread 0x1554393d4700 (LWP 932944)]
[New Thread 0x1554391d3700 (LWP 932945)]
[New Thread 0x155438fd2700 (LWP 932946)]
[New Thread 0x155438dd1700 (LWP 932947)]
[New Thread 0x155438bd0700 (LWP 932948)]
[New Thread 0x1554389cf700 (LWP 932949)]
[New Thread 0x1554387ce700 (LWP 932950)]
[New Thread 0x1554385cd700 (LWP 932951)]
[New Thread 0x1554383cc700 (LWP 932952)]
[New Thread 0x1554381cb700 (LWP 932953)]
[New Thread 0x155437fca700 (LWP 932954)]
[New Thread 0x155437dc9700 (LWP 932955)]
[New Thread 0x155437bc8700 (LWP 932956)]
[New Thread 0x1554379c7700 (LWP 932957)]
[New Thread 0x1554377c6700 (LWP 932958)]
[New Thread 0x1554375c5700 (LWP 932959)]
[New Thread 0x1554373c4700 (LWP 932960)]
[New Thread 0x1554371c3700 (LWP 932961)]
[New Thread 0x155436fc2700 (LWP 932962)]
[New Thread 0x155436dc1700 (LWP 932963)]
[New Thread 0x155436bc0700 (LWP 932964)]
[New Thread 0x1554369bf700 (LWP 932965)]
[New Thread 0x1554367be700 (LWP 932966)]
[New Thread 0x1554365bd700 (LWP 932967)]
[New Thread 0x1554363bc700 (LWP 932968)]
[New Thread 0x1554361bb700 (LWP 932969)]
[New Thread 0x155435fba700 (LWP 932970)]
[New Thread 0x155435db9700 (LWP 932971)]
[Thread 0x1554671ff700 (LWP 932899) exited]
[Thread 0x1554379c7700 (LWP 932957) exited]
[Thread 0x155437dc9700 (LWP 932955) exited]
[Thread 0x155437fca700 (LWP 932954) exited]
[Thread 0x1554373c4700 (LWP 932960) exited]
[Thread 0x1554385cd700 (LWP 932951) exited]
[Thread 0x1554387ce700 (LWP 932950) exited]
[Thread 0x155438fd2700 (LWP 932946) exited]
[Thread 0x1554391d3700 (LWP 932945) exited]
[Thread 0x1554393d4700 (LWP 932944) exited]
[Thread 0x1554397d6700 (LWP 932942) exited]
[Thread 0x1554399d7700 (LWP 932941) exited]
[Thread 0x155439bd8700 (LWP 932940) exited]
[Thread 0x155439dd9700 (LWP 932939) exited]
[Thread 0x155435db9700 (LWP 932971) exited]
[Thread 0x155439fda700 (LWP 932938) exited]
[Thread 0x15543a1db700 (LWP 932937) exited]
[Thread 0x15543a7de700 (LWP 932934) exited]
[Thread 0x15543abe0700 (LWP 932932) exited]
[Thread 0x15543b3e4700 (LWP 932928) exited]
[Thread 0x15543b9e7700 (LWP 932925) exited]
[Thread 0x1554389cf700 (LWP 932949) exited]
[Thread 0x1554383cc700 (LWP 932952) exited]
[Thread 0x155438dd1700 (LWP 932947) exited]
[Thread 0x1554395d5700 (LWP 932943) exited]
[Thread 0x1554365bd700 (LWP 932967) exited]
[Thread 0x15543b5e5700 (LWP 932927) exited]
[Thread 0x155435fba700 (LWP 932970) exited]
[Thread 0x1554369bf700 (LWP 932965) exited]
[Thread 0x155438bd0700 (LWP 932948) exited]
[Thread 0x15543ade1700 (LWP 932931) exited]
[Thread 0x1554367be700 (LWP 932966) exited]
[Thread 0x15543bbe8700 (LWP 932924) exited]
[Thread 0x15543a9df700 (LWP 932933) exited]
[Thread 0x15543c1eb700 (LWP 932921) exited]
[Thread 0x15543b7e6700 (LWP 932926) exited]
[Thread 0x15543afe2700 (LWP 932930) exited]
[Thread 0x155437bc8700 (LWP 932956) exited]
[Thread 0x155436dc1700 (LWP 932963) exited]
[Thread 0x1554363bc700 (LWP 932968) exited]
[Thread 0x1554377c6700 (LWP 932958) exited]
[Thread 0x1554371c3700 (LWP 932961) exited]
[Thread 0x155436fc2700 (LWP 932962) exited]
[Thread 0x15543b1e3700 (LWP 932929) exited]
[Thread 0x1554361bb700 (LWP 932969) exited]
[Thread 0x15543a5dd700 (LWP 932935) exited]
[Thread 0x1554381cb700 (LWP 932953) exited]
[Thread 0x15543a3dc700 (LWP 932936) exited]
[Thread 0x15543bde9700 (LWP 932923) exited]
[Thread 0x155436bc0700 (LWP 932964) exited]
[Thread 0x1554375c5700 (LWP 932959) exited]
[Thread 0x15543bfea700 (LWP 932922) exited]
[Thread 0x15543c3ec700 (LWP 932920) exited]
odgi: /opt/conda/conda-bld/odgi_1657286605938/work/build/sdsl-lite-prefix/src/sdsl-lite-build/include/sdsl/int_vector.hpp:1360: sdsl::int_vector<<anonymous> >::reference sdsl::int_vector<<anonymous> >::operator[](const size_type&) [with unsigned char t_width = 1; sdsl::int_vector<<anonymous> >::reference = sdsl::int_vector_reference<sdsl::int_vector<1> >; sdsl::int_vector<<anonymous> >::size_type = long unsigned int]: Assertion `idx < this->size()' failed.

Thread 1 "odgi" received signal SIGABRT, Aborted.
0x000015555472537f in raise () from /lib64/libc.so.6

(gdb) bt
#0  0x000015555472537f in raise () from /lib64/libc.so.6
#1  0x000015555470fdb5 in abort () from /lib64/libc.so.6
#2  0x000015555470fc89 in __assert_fail_base.cold.0 () from /lib64/libc.so.6
#3  0x000015555471da76 in __assert_fail () from /lib64/libc.so.6
#4  0x0000555555826ecb in xp::XP::from_handle_graph_impl(odgi::graph_t&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, unsigned long const&) ()
#5  0x00005555558271da in xp::XP::from_handle_graph(odgi::graph_t&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, unsigned long const&) ()
#6  0x000055555582731d in xp::XP::from_handle_graph(odgi::graph_t&, unsigned long const&) ()
#7  0x00005555557081fd in odgi::main_sort(int, char**) ()
#8  0x00005555556e3408 in odgi::subcommand::Subcommand::operator()(int, char**) const ()
#9  0x0000155554711493 in __libc_start_main () from /lib64/libc.so.6
#10 0x00005555555c0369 in _start ()

So, the conversion of the file didn't resolve the issue.

Cheers,

Lia

[AndreaGuarracino]

To be sure that is a system problem and not a graph problem, can you try with the graph available in odgi repository? This one https://raw.githubusercontent.com/pangenome/odgi/master/test/chr6.C4.gfa

dgi sort -i chr6.C4.gfa -Y -o chr6.C4.sorted.og -t 16 -P

[LiaOb21]

It probably worked, I think:

odgi sort -i chr6.C4.gfa -Y -o chr6.C4.sorted.og -t 16 -P
[odgi::sort] warning: the given file "chr6.C4.gfa" is not in ODGI format. To save time in the future, please use odgi build -i=[FILE], --idx=[FILE] -o=[FILE], --out=[FILE] to generate a graph in ODGI format. Such a graph can be supplied to all ODGI subcommands. Building graph in ODGI format from given GFA.
[odgi::gfa_to_handle] building nodes:  -nan% @ 0.00e+00/s elapsed: 00:00:00:00 remain: -2147483648:-3:-14:-8
[odgi::gfa_to_handle] building edges:  -nan% @ 0.00e+00/s elapsed: 00:00:00:00 remain: -2147483648:-3:-14:-8
error [xp]: Graph to index is not optimized. Please run 'odgi sort' using -O, --optimize.

odgi sort -i chr6.C4.gfa -Y -o chr6.C4.sorted.og -t 16 -P -O
[odgi::sort] warning: the given file "chr6.C4.gfa" is not in ODGI format. To save time in the future, please use odgi build -i=[FILE], --idx=[FILE] -o=[FILE], --out=[FILE] to generate a graph in ODGI format. Such a graph can be supplied to all ODGI subcommands. Building graph in ODGI format from given GFA.
[odgi::gfa_to_handle] building nodes:  -nan% @ 0.00e+00/s elapsed: 00:00:00:00 remain: -2147483648:-3:-14:-8
[odgi::gfa_to_handle] building edges:  -nan% @ 0.00e+00/s elapsed: 00:00:00:00 remain: -2147483648:-3:-14:-8
path_sgd_zipf_space_max: 100
path_sgd_zipf_max_number_of_distributions: 101
[odgi::path_linear_sgd] 1D path-guided SGD:  -nan% @ 0.00e+00/s elapsed: 00:00:00:00 remain: -2147483648:-3:-14:-8

ls -lh
total 2.1M
   2.1M Jul 14 15:56 chr6.C4.gfa
   60 Jul 14 15:58 chr6.C4.sorted.og

[AndreaGuarracino]

Something is off, -O should not be necessary, and unfortunately, it is not working.

Which odgi-conda version have you installed (odgi version, hoping it works)? Can you update odgi to the latest version possible and try again?

[LiaOb21]

Thank you @AndreaGuarracino!

odgi version
v0.7.3 "Fissaggio"

It should be the latest one, right?

[subwaystation]

Yes, it is the latest release @LiaOb21.

I just tried your commands with the latest ODGI from todays master and the v0.7.2 - Radler. Both times it worked flawlessly.

odgi build -g Arabidopsis_pg.gfa -o Arabidopsis_pg.og -P -t 8
[odgi::gfa_to_handle] building nodes:  0.00% @ 3.60e+05/s elapsed: 00:00:00:00 
[odgi::gfa_to_handle] building nodes:  3.15% @ 1.50e+06/s elapsed: 00:00:00:00
...
[odgi::gfa_to_handle] building paths: 100.00% @ 1.46e+01/s elapsed: 00:00:00:02 remain: 00:00:00:00

odgi sort -i Arabidopsis_pg.og -Y -o Arabidopsis_pg.og.Y -P -t 15
path_sgd_zipf_space_max: 100
path_sgd_zipf_max_number_of_distributions: 101
[odgi::path_linear_sgd] calculating linear SGD schedule (4.10571e-13 1 100 0 0.01)
[odgi::path_linear_sgd] calculating zetas for 102 zipf distributions
[odgi::path_linear_sgd] 1D path-guided SGD:  0.00% @ 1.35e+04/s elapsed: 00:00:0
...
[odgi::path_linear_sgd] 1D path-guided SGD: 100.00% @ 1.31e+07/s elapsed: 00:00:01:33 remain: 00:00:00:00

Did you use a python 3.10 environment with conda? If so please switch to 3.9. Since you can't use docker, can you at least use singularity?

[subwaystation]

While some regions look better aligned now, more wild links appear!

[subwaystation]

odgi sort -i Arabidopsis_pg.og -p Ys -o Arabidopsis_pg.og.Ys -P -t 16

Including a :bug: topological sort in the pipeline, it would look quite neat, however, the topological sort moves some nodes to the right (which it shouldn't!), so lot's of long range links are introduced.

[subwaystation]

I suspect you also can't build from source @LiaOb21 ?

[LiaOb21]

Hi @subwaystation,

Thank you so much! Here my python version, which should be ok:

python --version
Python 3.9.12

I can try to build from source if you think this could help to solve the problem. Using singularity it is not easy in our system apparently!

[LiaOb21]

I found that conda is not installing the right C++ version automatically. This is the version that I had:

gcc --version
gcc (GCC) 8.4.1 20200928 (Red Hat 8.4.1-1)

g++ --version
g++ (GCC) 8.4.1 20200928 (Red Hat 8.4.1-1)

So, I installed in the odgi conda environment a newer version of C++.

conda install -c conda-forge gcc
conda install -c conda-forge gxx

gcc --version
gcc (conda-forge gcc 12.1.0-16) 12.1.0

g++ --version
g++ (conda-forge gcc 12.1.0-16) 12.1.0

Anyway, the problem is persisting. Is there any other dependency I should check before of trying to build the package from source?

[subwaystation]

Hi @LiaOb21,

the idea I had in mind was to build ODGI from scratch not using any conda dependencies. This will then tell us if the conda package is the problem or rather your setup.

Would that be somehow possible? Instructions are at https://odgi.readthedocs.io/en/latest/rst/installation.html#building-from-source.

Best, Simon

[subwaystation]

Any updates @LiaOb21?

[LiaOb21]

Hi Simon,

Sorry for having been silent for so long. I moved my work to another server, and I installed odgi using Docker there. However, when trying to sort the graph with -Y I had exactly the same error I had with odgi installed through conda. I will try now to install it from source in this new machine, hopefully here it will work. I'll keep you updated.

Cheers,

Lia

[LiaOb21]

Hi,

I think I was finally able to correctly install odgi via the guix-genomics git repository. So far I tried just to sort the graph available in odgi repository, as suggested by @AndreaGuarracino. Here the figures I obtained:

odgi viz -i chr6.C4.og -o chr6.C4.png

odgi viz -i chr6.C4.sorted.og -o chr6.C4.sorted.png

I will then try with my pangenome. Can you confirm it is working properly now? Thank you in advance.

Cheers,

Lia

[LiaOb21]

Ok, I can confirm you that odgi is now working with my pangenome too :) Probably there is something wrong with conda and docker.

Thanks a lot for your help!!

[subwaystation]

Awesome! Thanks for your trials!

[shangshanzhizhe]

Hello all,

I encountered the identical problem -- sort didn't work with -Y option. Below is the details:

I was using Singularity package pulled from dockerhub, label 1688452591.

The commands were:

singularity exec --cleanenv -H $PWD -e ~/scratch/apps/singularity_packages/odgi.simg odgi build -g 30044_12.scaffold_1.gfa -o 30044_12.scaffold_1.og -P

which worked well.

Sort command was: singularity exec --cleanenv -H $PWD -e ~/scratch/apps/singularity_packages/odgi.simg odgi sort -i 30044_12.scaffold_1.og -o 30044_12.scaffold_1.sorted.og --threads 2 -P -Y

Gave the error message: odgi: /odgi/build/sdsl-lite-prefix/src/sdsl-lite-build/include/sdsl/int_vector.hpp:1360: sdsl::int_vector<<anonymous> >::reference sdsl::int_vector<<anonymous> >::operator[](const size_type&) [with unsigned char t_width = 1; sdsl::int_vector<<anonymous> >::reference = sdsl::int_vector_reference<sdsl::int_vector<1> >; sdsl::int_vector<<anonymous> >::size_type = long unsigned int]: Assertionidx < this->size()' failed.`

And It turned out that this only occurs when -Y is used.

Would you be able to provide some suggestions?

Many thanks, Shangzhe

[subwaystation]

Maybe you need to set the temporary directory with -C specifically? I also ran into some troubles running odgi on our cluster using Singularity. This was fixing my issues. Were you able to try this out not using Singularity?

[shangshanzhizhe]

Thank you very much for the prompt reply!

Sadly the error message stills identical with -C added. I tried both sbatch and salloc (to simulate a local machine) but both failed.

Any ideas?

Thanks, Shangzhe

[subwaystation]

If you could share the graph I can try to run this locally. If it works it could be a Singularity I/O issue. If I can reproduce the bug, I will take a closer look ;)

[shangshanzhizhe]

Thanks for the assistance!! Here is the link: link The original file I mentioned was a couple Gb so I shared a shorter scaffold here.

[subwaystation]

Hi @shangshanzhizhe, your input graph contains W lines which are not supported by ODGI. We only support the GFA format where paths are encoded with P lines. With vg convert you can go from W to P, but wenever tried if ODGI can work with such a file. Feedback welcome!

[glennhickey]

Please also note that minigraph-cactus can now produce ODGI output directly with --chrom-og for chromosome graphs and/or --odgi for the full genome graph.

[shangshanzhizhe]

Hello @subwaystation,

You were correct. It turned out to be the problem of the GFA version. I haven't tried the odgi sort deeply but a basic test with -Y worked when GFA v1.0 was used.

And also thank you @glennhickey for the suggestion. I'm using a older version of cactus-pangenome but I will explore it.

Many thanks, Shangzhe

[subwaystation]

Happy to hear this is solved @shangshanzhizhe

Cool feature @glennhickey !