Improvement/sasi index for paths

[ifesdjeen]

I'm going to do some extensive testing tomorrow, and it's a bit late so i might have made some stupid mistake.

So just putting it out to start a conversation.

[ifesdjeen]

I've pushed and rebased the latest version. It "generally" works. At least it doesn't break previous indexes :)

My only observation is that indexes lag significantly under heavy write load. In order to avoid it, we need either sets of bloom filters. Do you think it's ok to use blomstre for that or you prefer sets or some other implementation?

[ifesdjeen]

I've checked several other implemetnations, including the guava one. Unfortunately, they all are non thread safe. At least I haven't found any other concurrent implementation. So I sticked to my impl for now. Alternative path (in future) might be to give access to the metric registry, where we save the metric names anyways.

[ifesdjeen]

There was a very good comment by @arodland that bloom filter logic was inverted, therefore incorrect. I've switched the implementation to nbhm used in the engine to be available for the index, too.

[ifesdjeen]

Minimal configuration:

engine:
  rules:
    default: [ "1s:1h" ]

api:
  host: 0.0.0.0
  port: 8080

input:
  - type: carbon
    port: 2003

index:
  type: agent

drift:
  type: agent

store:
  cluster: 'localhost'
  keyspace: 'metric'

logging:
  level: info
  console: true
  files:
    - "/var/log/cyanite/cyanite.log"
  overrides:
    io.cyanite: "debug"

reporter:
  metrics:
    reporters:
      graphite:
        interval: 1
        opts:
          host: 127.0.0.1
          port: 2003
          prefix: internal

drift:
  type: volatile

index:
  type: cassandra
  cluster: 'localhost'
  keyspace: 'metric'

[pyr]

Overall ok with the code, and looking forward to a c* only solution. Are we still able to query using traditional graphite syntax with this? I.e: non prefix wildcard searches, and curly searches?

[ifesdjeen]

Currently, in prefixes and leaves, we're replacing glob patterns (trimmed down to the very first one), with prefix LIKE query. After that, we fall back to glob. So it should™ work.

[pyr]

sounds good.

[ifesdjeen]

<3