search

vgteam / sequenceTubeMap

displays multiple genomic sequences in the form of a tube map
MIT License
177 stars 24 forks source link

Processing large pangenome graph requests before timeout #124

Open brettChapman opened 3 years ago

brettChapman commented 3 years ago

Hi

I thought I better create a new issue.

I'm getting time out errors on all my requests from displaying all paths in my graph (to display paths in the paths drop-down menu) to running vg chunk. Is there a way to pre-cache any data, such as paths, or tweak the timeout settings in sequenceTubeMap to give vg chunk time enough to process the request? All related error messages are in issue https://github.com/vgteam/sequenceTubeMap/issues/123

Thanks.

adamnovak commented 3 years ago

I noted this in #123, but I think getting this working well depends on vg support for on-disk random access in graphs, probably via the memory mapping push.

We do have some support for on-disk random access in vg in specially organized, compressed, and indexed .vg graphs (see: vg index --index-sorted-vg), but vg chunk doesn't know how to read those currently, and they don't help vg paths at all. On the vg side we decided not to pursue this method of indexing, in favor of memory mapping.

The timeouts are all happening client-side; it might be good to work out how to adjust the timeouts in browsers and document that as a workaround.

brettChapman commented 3 years ago

Thanks @adamnovak for the detailed explanation.

I look forward to seeing how sequenceTubeMap develops over time. For now I'll be loading in many smaller indexed graphs (.xg) as mounted files.