timescale / promscale

[DEPRECATED] Promscale is a unified metric and trace observability backend for Prometheus, Jaeger and OpenTelemetry built on PostgreSQL and TimescaleDB.
https://www.timescale.com/promscale
Apache License 2.0
1.33k stars 169 forks source link

Adds support for new downsampling UX #1808

Closed Harkishen-Singh closed 1 year ago

Harkishen-Singh commented 1 year ago

Signed-off-by: Harkishen-Singh harkishensingh@hotmail.com

This commit updates the UX for creating downsampled data for Prometheus metrics.

Now, downsampling can be done by

metrics:
  downsampling:
  - interval: 5m
    retention: 30d
  - interval: 1h
    retention: 365d

The above dataset-config setting will create 2 downsampling, ds_5m (interval 5m & retention 30d) and ds_1h (interval 1h & retention 365d).

If an entry is removed, like

metrics:
  downsampling:
  - interval: 1h
    retention: 365d

Then we disable the ds_5m downsampling from refreshing. Deletion can only happen via SQL, CALL _prom_catalog.delete_downsampling(<schema_name>). This keeps the UX clean and simple.

If the removed downsampling is applied again later on, we simply enable the refreshing.

This also means that this commit adds ability to enable or disable downsampling as well.

Description

Merge requirements

Please take into account the following non-code changes that you may need to make with your PR:

Harkishen-Singh commented 1 year ago

Note for reviewers: The lint tests fails at the moment. I haven't fixed them since I aim to first be sure that we are fine with the new way day.Duration is implemented. Once we are fine, I will fix the linting tests. Otherwise, if we plan to change day.Duration down the line, it will consume good time updating the tests.

Harkishen-Singh commented 1 year ago

Note: Commit Adds support to query default downsampling views in absence of __column__ is not directly related to this PR, but it was more comfortable and faster for me to include in this PR, than wait for approval and merge and then open a following PR.

Harkishen-Singh commented 1 year ago

This (_prom_catalog.create_or_update_downsampling) might be helpful while reviewing.

niksajakovljevic commented 1 year ago

Note: Commit Adds support to query default downsampling views in absence of column is not directly related to this PR, but it was more comfortable and faster for me to include in this PR, than wait for approval and merge and then open a following PR.

Btw it does seem that commit is gone so can't really follow through that part