search

grafana / loki

Like Prometheus, but for logs.
https://grafana.com/loki
GNU Affero General Public License v3.0
23.34k stars 3.38k forks source link

HDFS as object store support #7941

Open akashkale13 opened 1 year ago

akashkale13 commented 1 year ago

Ingestor currently supports multiple object stores to write index as well as chunks like S3, InMemory etc. However I one usecase I plan to use boltdb shipper along with HDFS as HDFS being an open source and scalable component. However there is no implementation for it as of now. Are there any plans around adding this support?

dannykopping commented 1 year ago

My understanding is that HDFS does not work well with small files.

Is that still a problem?

jeschkies commented 1 year ago

A quick search revealed this. If there's an interest from the community it should be fine. However, I'm not sure if our block storage client should have HDFS support.