search

geopython / GeoHealthCheck

Service Status and QoS Checker for OGC Web Services
https://geohealthcheck.org
MIT License
84 stars 71 forks source link

Partial fix for #326: cache results derived from runs in resource model #394

Closed fsteggink closed 3 years ago

fsteggink commented 3 years ago

The goal of this PR is to cache information calculated / derived from runs in the resource model. This improves the generation speed of pages containing resource lists (home page, page per resource type, or 'Show all') with approximately factor 5.

Note that self.runs is still used in multiple places. For this PR I chose not to replace SQLAlchemy functionality (like .count, .order_by or .filter_by) by their Python equivalents. However, I think that when this is done, the time to generate a page containing resource lists will improve even more.

justb4 commented 3 years ago

Think this is a good first step. So this is internal caching right, no new DB column and for the lifetime of the Probe instance object? We do a similar thing for metadata (Capabilities), but with caching at class-level: https://github.com/geopython/GeoHealthCheck/blob/master/GeoHealthCheck/probe.py#L129, but possibly this PR fix here is simple enough and a first step towards further optimizations. I thought of a ProbeStats table there as a global cache, but that has quite some implications in multi-threaded/processing setups.