jdemaeyer / brightsky

JSON API for DWD's open weather data.
https://brightsky.dev/
MIT License
290 stars 18 forks source link

Investigate dumping PostGIS, adding location index #52

Closed jdemaeyer closed 4 years ago

jdemaeyer commented 4 years ago

We can probably get by with the much more lightweight earthdistance: https://blog.rebased.pl/2020/04/07/why-you-probably-dont-need-postgis.html

jdemaeyer commented 4 years ago

Benchmarks with PostGIS (no index):

50 one-day queries,  by lat/lon, sequential:   6.70 s
50 one-day queries,  by lat/lon, parallel  :   3.15 s
50 one-week queries, by lat/lon, sequential:  10.28 s
50 one-week queries, by lat/lon, parallel  :   5.71 s
50 one-day queries,  by station, sequential:   1.12 s
50 one-day queries,  by station, parallel  :   0.52 s
50 one-week queries, by station, sequential:   2.77 s
50 one-week queries, by station, parallel  :   1.25 s
50 one-day queries,  by source,  sequential:   0.52 s
50 one-day queries,  by source,  parallel  :   0.24 s
50 one-week queries, by source,  sequential:   1.93 s
50 one-week queries, by source,  parallel  :   0.89 s

With earthdistance (no index):

50 one-day queries,  by lat/lon, sequential:   6.84 s               
50 one-day queries,  by lat/lon, parallel  :   3.23 s
50 one-week queries, by lat/lon, sequential:  10.50 s
50 one-week queries, by lat/lon, parallel  :   6.08 s
50 one-day queries,  by station, sequential:   0.89 s
50 one-day queries,  by station, parallel  :   0.46 s
50 one-week queries, by station, sequential:   2.68 s
50 one-week queries, by station, parallel  :   1.25 s
50 one-day queries,  by source,  sequential:   0.62 s
50 one-day queries,  by source,  parallel  :   0.26 s
50 one-week queries, by source,  sequential:   2.21 s
50 one-week queries, by source,  parallel  :   1.04 s

With earthdistance and index:

50 one-day queries,  by lat/lon, sequential:   2.22 s
50 one-day queries,  by lat/lon, parallel  :   1.23 s
50 one-week queries, by lat/lon, sequential:   5.79 s
50 one-week queries, by lat/lon, parallel  :   3.78 s
50 one-day queries,  by station, sequential:   0.87 s
50 one-day queries,  by station, parallel  :   0.43 s
50 one-week queries, by station, sequential:   2.56 s
50 one-week queries, by station, parallel  :   1.24 s
50 one-day queries,  by source,  sequential:   0.55 s
50 one-day queries,  by source,  parallel  :   0.26 s
50 one-week queries, by source,  sequential:   2.11 s
50 one-week queries, by source,  parallel  :   1.01 s