search

grobian / carbon-c-relay

Enhanced C implementation of Carbon relay, aggregator and rewriter
Apache License 2.0
380 stars 107 forks source link

Validation doesn't work on Ubuntu Trusty (glibc 2.19) #238

Closed deniszh closed 7 years ago

deniszh commented 7 years ago

Trying to use some generic metric validation, like validate ^[-+]?[0-9]*.?[0-9]+([eE][-+]?[0-9]+)?\ [0-9.]+$ else drop on Ubuntu Trusty. (It was working perfect on Ubuntu Precise btw) First trying that:

cluster zone1
    carbon_ch replication 1
        graph001:2103
        graph007:2103
        graph004:2103
    ;
cluster zone2
    carbon_ch replication 1
        graph002:2103
        graph008:2103
        graph005:2103
    ;
cluster zone3
    carbon_ch replication 1
        graph003:2103
        graph009:2103
        graph006:2103
    ;

match *
    validate ^[-+]?[0-9]*.?[0-9]+([eE][-+]?[0-9]+)?\ [0-9.]+$ else drop
    send to zone1
    ;
match *
    validate ^[-+]?[0-9]*.?[0-9]+([eE][-+]?[0-9]+)?\ [0-9.]+$ else drop
    send to zone2
    ;
match *
    validate ^[-+]?[0-9]*.?[0-9]+([eE][-+]?[0-9]+)?\ [0-9.]+$ else drop
    send to zone3
    stop
    ;

Getting this error:

[2016-11-29 11:54:24] starting carbon-c-relay v2.2 (2e76c1), pid=30277
configuration:
    relay hostname = graph001
    listen port = 2003
    workers = 4
    send batch size = 2500
    server queue size = 25000
    server max stalls = 4
    statistics submission interval = 60s
    listen backlog = 32
    server connection IO timeout = 600ms
    routes configuration = /root/relay.conf

[2016-11-29 11:54:24] invalid expression '^[-+]?[0-9]*.?[0-9]+([eE][-+]?[0-9]+)?\' for match: Trailing backslash
[2016-11-29 11:54:24] failed to read configuration '/root/relay.conf'

OK, trying to replace '\ ' with '\s'. Looks like working:

[2016-11-29 11:55:20] starting carbon-c-relay v2.2 (2e76c1), pid=30330
configuration:
    relay hostname = graph001
    listen port = 2003
    workers = 4
    send batch size = 2500
    server queue size = 25000
    server max stalls = 4
    statistics submission interval = 60s
    listen backlog = 32
    server connection IO timeout = 600ms
    routes configuration = /root/relay.conf

parsed configuration follows:
cluster zone1
    carbon_ch replication 1
        graph001:2103
        graph007:2103
        graph004:2103
    ;
cluster zone2
    carbon_ch replication 1
        graph002:2103
        graph008:2103
        graph005:2103
    ;
cluster zone3
    carbon_ch replication 1
        graph003:2103
        graph009:2103
        graph006:2103
    ;

match
        *
        validate
        ^[-+]?[0-9]*.?[0-9]+([eE][-+]?[0-9]+)?\s[0-9.]+$
        else
        drop
    send to zone1
    ;
match
        *
        validate
        ^[-+]?[0-9]*.?[0-9]+([eE][-+]?[0-9]+)?\s[0-9.]+$
        else
        drop
    send to zone2
    ;
match
        *
        validate
        ^[-+]?[0-9]*.?[0-9]+([eE][-+]?[0-9]+)?\s[0-9.]+$
        else
        drop
    send to zone3
    stop
    ;

But it's not:

a a a
match
    * -> a
    carbon_ch(zone1)
        graph004:2103
match
    * -> a
    carbon_ch(zone2)
        graph005:2103
match
    * -> a
    carbon_ch(zone3)
        graph006:2103
    stop
1 1 1
match
    * -> 1
    carbon_ch(zone1)
        graph001:2103
match
    * -> 1
    carbon_ch(zone2)
        graph005:2103
match
    * -> 1
    carbon_ch(zone3)
        graph009:2103
    stop
grobian commented 7 years ago

Can you check what libc's are on the respective distros? For the regex engine I'm relying on the libc implementation.

I'm planning on autoconfing the project so we can pull in more heavy dependencies (like pcre) where desirable to fix these kinds of problems.

deniszh commented 7 years ago

Precise has glibc 2.15, Trusty - 2.19

grobian commented 7 years ago

ok, I have 2.22 to try with

grobian commented 7 years ago 
a a a
match
    * -> a
    validate
        ^[-+]?[0-9]*.?[0-9]+([eE][-+]?[0-9]+)?\ [0-9.]+$ -> a a
        fail -> drop
match
    * -> a
    validate
        ^[-+]?[0-9]*.?[0-9]+([eE][-+]?[0-9]+)?\ [0-9.]+$ -> a a
        fail -> drop
match
    * -> a
    validate
        ^[-+]?[0-9]*.?[0-9]+([eE][-+]?[0-9]+)?\ [0-9.]+$ -> a a
        fail -> drop
    stop

so I guess the problem for you is that "Trailing backslash" problem?

grobian commented 7 years ago

Your example shows it isn't validating at all, this is confusing. It should print the validate pattern, e.g.:

1 1 1 
match
    * -> 1
    validate
        ^[-+]?[0-9]*.?[0-9]+([eE][-+]?[0-9]+)?\ [0-9.]+$ -> 1 1
        match
    carbon_ch(zone1)
        127.0.0.1:2101
deniszh commented 7 years ago

Yes, if I replacing '\ ' with '\s' it doesn't validate at all for some reason. And it didn't accept '\ ' because of 'Trailing slash'.

deniszh commented 7 years ago

I tried to replace '\ ' with ' ' - doesn't work too, because treating space as separator of regexes:

[2016-11-30 14:02:42] starting carbon-c-relay v2.2 (2e76c1), pid=28521
configuration:
    relay hostname = graph001
    listen port = 2003
    workers = 4
    send batch size = 2500
    server queue size = 25000
    server max stalls = 4
    statistics submission interval = 60s
    listen backlog = 32
    server connection IO timeout = 600ms
    routes configuration = /root/relay.conf

parsed configuration follows:
cluster zone1
    carbon_ch replication 1
        graph001:2103
        graph007:2103
        graph004:2103
    ;
cluster zone2
    carbon_ch replication 1
        graph002:2103
        graph008:2103
        graph005:2103
    ;
cluster zone3
    carbon_ch replication 1
        graph003:2103
        graph009:2103
        graph006:2103
    ;

match
        *
        validate
        ^[-+]?[0-9]*.?[0-9]+([eE][-+]?[0-9]+)?
        [0-9.]+$
        else
        drop
    send to zone1
    ;
match
        *
        validate
        ^[-+]?[0-9]*.?[0-9]+([eE][-+]?[0-9]+)?
        [0-9.]+$
        else
        drop
    send to zone2
    ;
match
        *
        validate
        ^[-+]?[0-9]*.?[0-9]+([eE][-+]?[0-9]+)?
        [0-9.]+$
        else
        drop
    send to zone3
    stop
    ;

a a a
match
    * -> a
    carbon_ch(zone1)
        graph004:2103
match
    * -> a
    carbon_ch(zone2)
        graph005:2103
match
    * -> a
    carbon_ch(zone3)
        graph006:2103
    stop
grobian commented 7 years ago

glibc-2.22:

1 1 1
match
    * -> 1
    validate
        ^[-+]?[0-9]*.?[0-9]+([eE][-+]?[0-9]+)?\ [0-9.]+$ -> 1 1
        match
    carbon_ch(zone1)
        127.0.0.1:2101
match
    * -> 1
    validate
        ^[-+]?[0-9]*.?[0-9]+([eE][-+]?[0-9]+)?\s[0-9.]+$ -> 1 1
        match
    carbon_ch(zone2)
        127.0.0.1:2102
match
    * -> 1
    validate
        ^[-+]?[0-9]*.?[0-9]+([eE][-+]?[0-9]+)?[:space:][0-9.]+$ -> 1 1
        fail -> drop
    stop

but neither \s nor [:space:] work on Solaris, where I needed the '\ ' hack

grobian commented 7 years ago

hang on

[2016-11-30 14:02:42] starting carbon-c-relay v2.2 (2e76c1), pid=28521

commit 2e76c1 is the release commit, are you sure you're doing what you think you're doing? v2.2 doesn't recognise the syntax, and hence sees it as just other things to match on.

grobian commented 7 years ago

v2.2 doesn't support/understand validate

deniszh commented 7 years ago

Ah, I forgot that I rolled it back, sorry. Will check one more time.

deniszh commented 7 years ago

Same thing is on 2.3 too: When using '\ ' I'm getting trailing slash error:

[2016-12-01 14:46:30] starting carbon-c-relay v2.3 (633ff6), pid=1582
configuration:
    relay hostname = graph001
    listen port = 2003
    workers = 4
    send batch size = 2500
    server queue size = 25000
    server max stalls = 4
    statistics submission interval = 60s
    listen backlog = 32
    server connection IO timeout = 600ms
    routes configuration = /root/relay.conf

[2016-12-01 14:46:30] invalid expression '^[-+]?[0-9]*.?[0-9]+([eE][-+]?[0-9]+)?\' for match: Trailing backslash
[2016-12-01 14:46:30] failed to read configuration '/root/relay.conf'

When using '\s' I'm getting no match:

[2016-12-01 14:49:09] starting carbon-c-relay v2.3 (633ff6), pid=2184
configuration:
    relay hostname = graph001
    listen port = 2003
    workers = 4
    send batch size = 2500
    server queue size = 25000
    server max stalls = 4
    statistics submission interval = 60s
    listen backlog = 32
    server connection IO timeout = 600ms
    routes configuration = /root/relay.conf

parsed configuration follows:
cluster zone1
    carbon_ch replication 1
        graph001:2103
        graph007:2103
        graph004:2103
    ;
cluster zone2
    carbon_ch replication 1
        graph002:2103
        graph008:2103
        graph005:2103
    ;
cluster zone3
    carbon_ch replication 1
        graph003:2103
        graph009:2103
        graph006:2103
    ;

match
        *
        validate
        ^[-+]?[0-9]*.?[0-9]+([eE][-+]?[0-9]+)?\s[0-9.]+$
        else
        drop
    send to zone1
    ;
match
        *
        validate
        ^[-+]?[0-9]*.?[0-9]+([eE][-+]?[0-9]+)?\s[0-9.]+$
        else
        drop
    send to zone2
    ;
match
        *
        validate
        ^[-+]?[0-9]*.?[0-9]+([eE][-+]?[0-9]+)?\s[0-9.]+$
        else
        drop
    send to zone3
    stop
    ;

a a a
match
    * -> a
    carbon_ch(zone1)
        graph004:2103
match
    * -> a
    carbon_ch(zone2)
        graph005:2103
match
    * -> a
    carbon_ch(zone3)
        graph006:2103
    stop
deniszh commented 7 years ago

Argh, it's still from 2.3 release commit. Sorry, I'm really need to fix my build script. Sorry again.