search

criteo / biggraphite

Simple Scalable Time Series Database
Apache License 2.0
129 stars 36 forks source link

BigGraphite with hybrid access (Cassandra + ElasticSearch) #535

Closed zerosoul13 closed 5 years ago

zerosoul13 commented 5 years ago

Hi there,

We've been enjoying the good life of BigGraphite for a while now and want to test using Cassandra and ElasticSearch to have an idea of how much performance we could gain.

I'm having trouble getting BigGraphite to syncdb as it would with Cassandra for data (or metadata). I don't see any activity while running syncdb on ES container and I'm wondering what I'm missing while Cassandra container does show "normal" activity upon sync.

Is there a step by step guide on how to setup BigGraphite to use Cassandra and ES for metadata?

Setup

For testing purposes, I'm using 2 docker images:

1 Cassandra container as stand-alone cluster (image: cassandra:latest) 1 ES container as stand-alone cluster (image: elasticsearch:6.8.3)

Command executed

/opt/graphite/bin/bgutil --data_driver=cassandra  --cassandra_contact_points=<cassandra ips> --cassandra_username=<edited> --cassandra_password=<edited> --metadata_driver=elasticsearch --elasticsearch_hosts=<ES host> -- syncdb --storage-schemas /opt/graphite/conf/storage-schemas.conf
iksaif commented 5 years ago

Hello, syncdb is basically a noop for elasticsearch. Your command line looks ok and you should be able to use ES as-is. We haven't had many users but the last time I tried it worked pretty well. If you find anything that doesn't I'll be happy to review PRs.

iksaif commented 5 years ago

FYI, I started a PR to have docker-compose to make these tests easiers, which I never finished: https://github.com/criteo/biggraphite/pull/477 maybe that could be useful to you

zerosoul13 commented 5 years ago

Updated our image to the latest (0.14.14) and integration with elasticsearch worked out of the box. Thanks!