Closed tantra35 closed 4 months ago
Mentioned commit only dealt with printing the expressions, not any matching.
If I try your example, it just matches for me. How did your c-relay get configured? What regex library is it using? (relay -v will show which)
Yes you right i alredy realized it
In test stand we use follow configuration
cluster send-localhost
forward
localhost:2333
;
match "^local\.nomad-jobs\.[^;]+;.+"
send to
send-localhost
stop
;
match *
send to
blackhole
stop
;
carbon-c-relay
-v output
carbon-c-relay v3.7.4 (636551-dirty)
enabled support for: gzip ssl
regular expressions library: libc
on localhost we launch just simple socket server that write to stdout all that is get from socket:
#!/bin/bash
CARBONPORT=2333
echo "Start listening on port tcp ${CARBONPORT} ..."
while read line
do
echo "$line"
done < <(nc -k -l ${CARBONPORT})
echo "Good bye"
then we begin put metrics to carbon-c-relay
like this:
local.nomad-jobs.aav01.default.an-abtests-web-back.bytask.cpu.Percent.p99;tg=an-abtests-web;task=django_celery_beat 12 1675080983
and doesn't see any output, so metrics goes to "blackhole"
Also i must mention that in version 3.5 this quoted expression works
For expiriments we try to replace(experiments with additional slashes):
"^local\.nomad-jobs\.[^;]+;.+" -> "^local\\\.nomad-jobs\\\.[^;]+;.+"
and this is doesn't help, metrics doesn't selected as expected, and goes to blackhole on test stand
regular expressions library: libc
I'm almost sure this is the problem.
You mention this works on 3.5, can you post the regex library used by that version?
Hm, regexp library is the same, one difference is a ubuntu release(in docker container) for 3.5 we use ubuntu 16.04(xenial), and for 3.7 ubuntu 20.04(focal)
# carbon-c-relay -v
carbon-c-relay v3.5 (973e08-dirty)
enabled support for: gzip ssl
regular expressions library: libc
Also just replace libc re library to PCRE:
carbon-c-relay v3.7.4 (636551-dirty)
enabled support for: gzip ssl
regular expressions library: PCRE2
without success
carbon-c-relay v3.7.4 (3ce549)
enabled support for: gzip lz4 ssl
regular expressions library: oniguruma
that gives me:
match "^local.nomad-jobs.[^;]+;.+"
send to blackhole
stop
;
local.nomad-jobs.aav01.default.an-abtests-web-back.bytask.cpu.Percent.p99;tg=an-abtests-web;task=django_celery_beat
match
^local.nomad-jobs.[^;]+;.+ (regex) -> local.nomad-jobs.aav01.default.an-abtests-web-back.bytask.cpu.Percent.p99;tg=an-abtests-web;task=django_celery_beat
blackholed
stop
e.g. it matches your expression. I don't understand why this would change for you. There are differences between regex libraries for sure (some need escaping, others don't) but if you can run 3.5 next to 3.7.4 and get a different match, it's very likely the c-relay code that makes the difference.
In think https://github.com/grobian/carbon-c-relay/commit/4ecb19330f063ca8d851066834a82f9a5caf0787 is more of a problem to you. It would mean that "^local.nomad-jobs..+
would simply match the same thing, for ;.+
is never exposed to the matching.
(bug/fault in the test routine)
Hm, but how now detect metrics with tags? For example in our case we want that tagged metrics goes to the different metrics storage
I think that remove tags part from metric name is wrong decision
Yes, for that case, you'd disable tag support, by adding ;
to the set of allowed characters.
It gets more complicated if you want to have consistent-hashing on the metric name (without tags) at the same time.
I think this is a configuration issue
seems after https://github.com/grobian/carbon-c-relay/commit/f3ee3f1bc18d29f1321518a1aa8f3292603536ea broken follow expression(it is quoted to escape
;
)now it doesn't select metrics like this:
so metrics went to the wrong destination