Closed nityanandagohain closed 2 weeks ago
Few changes
number
type on tag_attrbutes will require frontend changes, keeping it as float64 and then in qs we are mapping it to number.
Also tried checking out the issue with minmax index for numbers, only bloom filter is working and not the minmax with numbers in maps.
CREATE TABLE test_table
(
id UInt64,
valueInt Map(String, Int64),
valueFloat Map(String, Float64),
onlyVal Float64,
index valueint_key_index mapKeys(valueInt) TYPE bloom_filter GRANULARITY 4,
index valuefloat_key_index mapKeys(valueFloat) TYPE bloom_filter GRANULARITY 4,
INDEX valueint_idx mapValues(valueInt) TYPE minmax GRANULARITY 1,
INDEX valueint_idx_b mapValues(valueInt) TYPE bloom_filter GRANULARITY 1,
INDEX valueFloat_idx mapValues(valueFloat) TYPE minmax GRANULARITY 1,
INDEX valueFloat_idx_b mapValues(valueFloat) TYPE bloom_filter GRANULARITY 1,
INDEX onlyV onlyVal TYPE minmax GRANULARITY 1,
INDEX onlyV_b onlyVal TYPE bloom_filter GRANULARITY 1,
) ENGINE=MergeTree ORDER BY id;
INSERT INTO test_table SELECT number as id, map(toString(number), number), map(toString(number), number), number FROM numbers(1000000);
--- not working on maps explain indexes=1 select * from test_table where valueFloat['1000'] > 999 limit 10;
explain indexes=1 select * from test_table where valueFloat['1000'] = 1000.0 limit 10;
explain indexes=1 select * from test_table where valueInt['1000'] > 999 limit 10;
explain indexes=1 select * from test_table where valueInt['1000'] = 1000 limit 10;
-- working on raw column explain indexes=1 select * from test_table where onlyVal = 1000.0 limit 10;
explain indexes=1 select * from test_table where onlyVal > 1000.0 limit 10; Also when > , < is used index on the key is also not applied, on when = is used index on the key is applied.
Decision w.r.t index
Create issues on clickhouse.
cc @srikanthccv
in
and create issue for frontend changes to remove from logs detail.
In doesn't work with string values.This left in new schema
=
instead of in
https://github.com/SigNoz/signoz/issues/5597_
attributes_
attributes or the normal ones[ ] Default materialized column if any required? (not a blocker or much linked with this issue)
cc @ankitnayan @raj-k-singh please add if you think anything else. (apart from JSON search which will be addressed separately)
number
instead of int64 and float64Update tags to use number instead of int64 and float64, breaking changeAny different orderby (resource) should be added as a where clause in resource filter. (OR)(since coverd by groupBy)Number support in tag attribute API's[ ]
Support for number typeDecisions to be taken
AND
while filtering for it in resources, we can useOR
here, if we don't use then no filtering will be done. ref:- https://github.com/SigNoz/signoz/pull/5420[x] As of now the otel collector writes to two table, the logs_v2 and logs. logs is the old table with order by timestamp, id. Should we keep it the same or create a materialized view for it ?
We have moved on from logs and will be just using logs_v2
[x] Do we even require materialized columns ?
Yes only for group by