Create VictoriaMetrics connector

[valyala]

It would be great to create Presto connector for VictoriaMetrics TSDB.

The connector should export a table with the following fields:

accountID uint32 not null  -- used only for multi-tenant access in cluster version
projectID uint32 not null  -- used only for multi-tenant access in cluster version
name string not null -- time series name in Prometheus format: {__name__="foo", label1="bar", ...}
timestamp int64 not null -- UTC timestamp in milliseconds
value float64 not null -- time series value for the given timestamp.

Then PromQL time series selector could be expressed in SQL. For example, the following PromQL selector metric{label1=~"foo.+bar"} could be expressed with the following SQL filter: regexp_like(name, '__name__="metric") and regexp_like(name, 'label1="foo.+bar"'). Such SQL query is easy to transform to the corresponding PromQL selector, which then can be sent to /api/v1/export API together with start and end args obtained from SQL filters on timestamp.

This is just an example - probably better table structure and better syntax can be used.

cc'ing @dlazerka

This is a copy of this issue.

Update: name could have MAP(string, string) type instead of string type.

[electrum]

This sounds useful. @valyala are you interested in working on this yourself? We can answer any questions and provide guidance on Slack: https://prestosql.io/slack.html

[valyala]

Thanks for the pointer to your Slack.

@dima-vm from our team is going to work on this soon.

[findepi]

@dlazerka don't hesitate to ask questions on #dev channel on the slack.

[electrum]

@dima-vm I see that you submitted a PR for a connector to the PrestoDB project. Are you interested in submitting one to this project as well?

[dima-vm]

Absolutely! That PR is pretty raw right now and not ready for review yet. By the way, do you know how to work with predicate pushdown? It seems like only empty unenforcedConstraint is supported when using predicate pushdown (only TRUE can be returned, otherwise presto-main code throws exception expecting OriginalExpression, not RowExpression: https://github.com/prestodb/presto/pull/13777/files#diff-8ed2886a05a92cd213ebc2244a4793c6R163)

[electrum]

That seems like it might be a bug in PrestoDB. In PrestoSQL, pushdown is handled via ConnectorMetadata.applyFilter() which returns ConstraintApplicationResult containing the remaining filter. You can see examples of using this in TpchMetadata and JmxMetadata. Most connector just return the original constraint because they can't make any guarantees (thus the pushdown is simply a hint and the engine will still do filtering).

[findepi]

@dima-vm please also join #dev on our slack (https://prestosql.io/slack.html)

[pangyifish]

wondering if we have any plan for supporting this?

[dima-vm]

Hi @pangyifish , sorry didn't have time to work on that. Let me know if you still interested in it, probably better to discuss in VM community slack: https://slack.victoriametrics.com/

[maximveksler]

Hi @pangyifish , sorry didn't have time to work on that. Let me know if you still interested in it, probably better to discuss in VM community slack: https://slack.victoriametrics.com/

Hey, was this eventually converged?