clingen-data-model / clinvar-streams

1 stars 0 forks source link

Fix OOM error on large queries #40

Closed theferrit32 closed 3 years ago

theferrit32 commented 3 years ago

For the initial clinvar release in clinvar-combiner, loading the list of variations with clojure.java.jdbc/query results in an out of memory error. Container is limited in k8s to 1 GiB, JVM has default limits though, which seems to be 256 MiB.

The result sets should be processed lazily though so that this is not a major concern. Especially for variation and clinical assertions this is important.