powa-team / powa-web

PoWA user interface
http://powa.readthedocs.io/
74 stars 31 forks source link

`qual_constants` ignores `powa_qualstats_constvalues_history_current` #175

Closed tbe closed 5 months ago

tbe commented 1 year ago

While setting up a PoC for short lived benchmarking containers for internal usage, we couldn't see any constants for quals.

Reason for this is, that the qual_constants query only considers powa_qualstats_constvalues_history, but this table only receives data if the coalesce treshold is reached.

As this parameter is described as Defines the amount of records to group together in the table., i assume this is a bug in powa-web.

rjuju commented 1 year ago

Hi,

The "coalesce" parameter is indeed the number of rows to group together, to benefit from TOAST compression (and also have some speedup when displaying big interval for some queries).

About your issue, qual_constants is supposed to retrieved data from both _history and _history_current version of the tables. For the history_current version, this happens at https://github.com/powa-team/powa-web/blob/master/powa/sql/__init__.py#L323C18-L323C62:

[...]
UNION ALL
            SELECT *
            FROM powa_qualstats_aggregate_constvalues_current(:server, :from, :to)
[...]

with powa_qualstats_aggregate_constvalues_current() being defined in powa-archivist, https://github.com/powa-team/powa-archivist/blob/master/powa--4.1.2.sql#L3798-L3915 as:

[...]
FROM powa_qualstats_constvalues_history_current q
[...]

The usual reason for not getting enough data from pg_qualstats datasource is because by default pg_qualstats only consider 1% of the queries (1/max_connections to be exact), which is especially impacting if you do a short lived benchmarking. Did you change this parameter before doing your benchmarking?

tbe commented 1 year ago

My bad, i didn't see: https://github.com/powa-team/powa-web/blob/4.1.3/powa/sql/__init__.py#L323C18-L323C62

Current Settings are:

track_io_timing = on

pg_stat_statements.max = 10000
pg_stat_statements.track = all

pg_qualstats.max = 100000
pg_qualstats.sample_rate = 1

pg_wait_sampling.profile_pid = false

powa.frequency = 1min

So, this should be ok for our use case. I also can find entries for the searched timerange in the *_current view. I will have a deeper look later today.

rjuju commented 1 year ago

Ok! Let me know if you find any problem, or if any improvements / modifications are needed.