search

ucscXena / ucsc-xena-client

Functional genomics browser
Apache License 2.0
57 stars 42 forks source link

data page loading issue for datasets with very large number of identifiers #348

Open jingchunzhu opened 5 years ago

jingchunzhu commented 5 years ago

this page does not load:

https://xenabrowser.net/datapages/?dataset=WGBS%2Ftcga.WGBS.xena&host=https%3A%2F%2Fatacseq.xenahubs.net&removeHub=https%3A%2F%2Fxena.treehouse.gi.ucsc.edu%3A443

acthp commented 5 years ago

possible approaches to this:

1) A quick work-around is to decouple the field count from the rest of the data loading. The field count will eventually time out & we'll need to display some sort of result ("couldn't get field count").

2) Limit the field count to something we expect to return in a reasonable time, and display the overflow, like "> 1 million fields"

3) Review the query plan in h2, and see if there are any db or aws params that would reduce the query time, e.g. larger cache or main memory

4) Track field count at load time, as we do for row count. This requires db schema migration.