fhirbase / fhirbase-plv8

[DEPRECATED] Fhirbase 2.0 is an FHIR relational storage
https://health-samurai.io/fhirbase
Other
105 stars 39 forks source link

introduce totalMethod param to get fast but approx count #138

Closed karsiwek-kainos closed 8 years ago

karsiwek-kainos commented 8 years ago

usage: no param - calculates "total" basing on standard COUNT(*) function _totalMethod=std - same as above _totalMethod=improved - calculated approximaged total using count_estimate function - which is using EXPLAIN data _totalMethod=no - do not calculate total at all (fastest but no total data, and no links to next, prev, first and last

results for 1M patients: fhirbase=# explain analyze select fhir_search('{"resourceType":"Patient", "queryString":""}');

QUERY PLAN

Result (cost=0.00..0.26 rows=1 width=0) (actual time=480.715..480.715 rows=1 loops=1) Planning time: 0.012 ms Execution time: 480.725 ms (3 rows)

fhirbase=# explain analyze select fhir_search('{"resourceType":"Patient", "queryString":"_totalMethod=no"}');

QUERY PLAN

Result (cost=0.00..0.26 rows=1 width=0) (actual time=3.415..3.415 rows=1 loops=1) Planning time: 0.025 ms Execution time: 3.434 ms

fhirbase=# explain analyze select fhir_search('{"resourceType":"Patient", "queryString":"_totalMethod=improved"}');

QUERY PLAN

Result (cost=0.00..0.26 rows=1 width=0) (actual time=10.147..10.148 rows=1 loops=1) Planning time: 0.012 ms Execution time: 10.157 ms (3 rows)

although select fhir_search('{"resourceType":"Patient", "queryString":"_totalMethod=std"}'); returns: {"resourceType":"Bundle","type":"searchset","total":1000000,"link":[{" ...

and select fhir_search('{"resourceType":"Patient", "queryString":"_totalMethod=improved"}'); returns: {"resourceType":"Bundle","type":"searchset","total":1000074,"link":[{ ....

which gives quite a satysfying results, but error is might be bigger for smaller volumes of data.

niquola commented 8 years ago

Cool. Please provide tests & documentation in readme for this feature and i will merge it. And may be simplify namings: _total={no,estimated,exact} another option to move this flag in params:

{"resourceType":"Patient",  "total": "estimated", "queryString":"name=...."}