Stationlite performance (application level)

[damb]

Features and Changes:

• Improve eida-stationlite performance at application level. Implement a more efficient post processing. Stats follow.

Bugfixes:

• Remove duplicate stream epochs when parsing query arguments.

Note, that this PR does not come along with DB related performance improvements. DB related performance improvements will be provided within a separate PR.

[damb]

Stats: Measuring eida-stationlite's performance before and after the improvements were implemented.

Tested were 2 exemplary queries. No caching is involved (which might be a reasonable future improvement, though). Tests were performed using the docker setup from docker/prod.

• N = 100
• Columns: Percentiles | Response start time | Response total time (times in seconds)

---

Query: /eidaws/routing/1/query?service=dataselect

Before (3c91919d597e74912e0a25b1259d93221f729646):

50%: [49.3235745 49.35793  ]
85%: [50.35789195 50.38779645]
90%: [50.6607964 50.694202 ]
95%: [51.7777449  51.80475235]
99%: [53.6291339  53.65851338]

Now (b83ad43897d16a1eba4de98ab059f644bca78097):

50%: [46.143741 46.147206]
85%: [46.9783062  46.98176795]
90%: [47.3548628 47.3583668]
95%: [47.734457 47.737973]
99%: [49.02552683 49.02930682]

This test corresponds to a single query at the Postgres DB backend which resolves to the entire EIDA inventory for service=dataselect. Routes currently are merged at query-time. After applying the code changes response times reduce about 8% (for the 95% percentile).

---

Query: /eidaws/routing/1/query?net=_ALPARRAY&service=dataselect

Before (3c91919d597e74912e0a25b1259d93221f729646):

50%: [46.1099925 46.1187615]
85%: [47.9555212  47.96371295]
90%: [48.8415157 48.8504823]
95%: [51.24864225 51.25896515]
99%: [60.34334897 60.35259966]

Now (b83ad43897d16a1eba4de98ab059f644bca78097):

50%: [30.6283145 30.629127 ]
85%: [32.7689919 32.769609 ]
90%: [33.0936488 33.0941317]
95%: [33.92988885 33.93052435]
99%: [38.05804655 38.05859462]

Querying the virtual network _ALPARRAY (service=dataselect) results in ~ 750 sub-queries to the Postgres DB backend (assuming the routing of the virtual network was configured in a way that each station was routed with a single entry). This also involves merging the results of the sub-queries, afterwards. Response time decreases by ~ 34 % (for the 95% percentile).