search

quickwit-oss / quickwit

Cloud-native search engine for observability. An open-source alternative to Datadog, Elasticsearch, Loki, and Tempo.
https://quickwit.io
Other
8.25k stars 337 forks source link

[WishList] Support more sources #1000

Open fulmicoton opened 2 years ago

fulmicoton commented 2 years ago

Comment here if you need/want support for

Ideally, make it clear whether you already have such a message queue in production or not.

splix commented 2 years ago

Hi. I'm not quite sure it's a right place to vote, but I'd like to see a better support for Google Cloud services, because we're building on it. So PubSub and I guess Storage also. Thank you.

fulmicoton commented 2 years ago

In the main branch, (but not in Quickwit 0.2) we already support custom endpoint for storage. All S3 compatible storage are -in theory- supported. I have not tested that with Google Cloud storage specifically however.

For Google PubSub, it is more complicated story. I opened an issue here: https://github.com/quickwit-oss/quickwit/issues/1107

If you can share more about your use case, I'd love to hear about it. (If you can share it, but prefer to do it privately, my email is paul at quickwit.io)

BrutalSimplicity commented 2 years ago

This is an awesome project. I've been wanting a serverless search engine solution for years. Really happy to see the support this has gained.

I'm an AWS geek right now - GCP next! Given that, I would love to get this up and running against an AWS Kinesis stream. So there's my vote! Thanks for all the hard work!

galactus009 commented 2 years ago

I would like to see support for NATS

fulmicoton commented 2 years ago

I did not know about nats. https://github.com/nats-io/nats.rs https://docs.nats.io/nats-concepts/overview

fulmicoton commented 2 years ago

@BrutalSimplicity Thanks for your message! Kinesis is on our task list, so it should happen very fast. You can check the progress one this issue https://github.com/quickwit-oss/quickwit/issues/1157

fmassot commented 2 years ago

+1 for NATS as Nuclia is using it.

juan-riveros commented 2 years ago

Redis Streams and Redis Pub/Sub!

I love having a small, selfhosted stack, so minio/ceph/garage+redis+quickwit seems like a dream for most of my projects.

calvin2021y commented 2 years ago

scylladb/cassandra

miton18 commented 1 year ago

+1 on Pulsar, already running in production on our side

jorda95225 commented 1 year ago

+1 on Pulsar, already running in production on our side

fulmicoton commented 1 year ago

@jorda95225 @miton18 Stay tuned, it will be released with quickwit 0.5, available in the next few weeks.

blablacio commented 1 year ago

+1 for Scylla/Cassandra, which is already used in our production systems for our chat service.

drusellers commented 1 year ago

+1 for RabbitMQ (Queue or Stream)

olivierboudet commented 4 months ago

Hi, I would like to have rabbitmq support also which is already running in production.