search

grafana / loki

Like Prometheus, but for logs.
https://grafana.com/loki
GNU Affero General Public License v3.0
23.83k stars 3.44k forks source link

Example for "All Cassandra" Config, More through documentation of config #9889

Closed DaveWK closed 8 months ago

DaveWK commented 1 year ago

Is your feature request related to a problem? Please describe. The example snippets in the documentation are difficult to follow for certain configurations, and the page which enumberates the loki config sections does not provide enough detail for every section, so there's a lot of jumping around looking for specific section's potential/allowed values.

Describe the solution you'd like I'd like the snippets to be less like pseudocode and more practical. By having a lot of sections that are a subsection of a complete config, it makes it difficult to understand what a full config is supposed to look like. For instance the "schema_config" description is very lacking in useful details: https://grafana.com/docs/loki/latest/configuration/#schema_config

which means that I'm forced to root around for examples that aren't exactly my use case and Frankenstein together a config.

My particular use case is that I want to make an "all cassandra" config (no aws, no gcs, no s3, no filesystem), but I seem to be missing some piece related to how the "chunk" storage works and how to properly set up those config stanzas. I'd love to see a more complete config example for my use case, but would also be happy if the documentation were just more clear as to what the parameters were for the "chunk" and "schema_config" sections were.

DaveWK commented 1 year ago

I found my issue. I did not explicitly enable the table manager: https://grafana.com/docs/loki/latest/upgrading/#the-single-binary-no-longer-runs-a-table-manager

This probably should be documented more prominently, as keeping the data storage tier as a separate service from the stateless pieces makes it much easier to avoid data loss and corruption in the event loki is restarted or has an issue.

chaudum commented 1 year ago

Hey @DaveWK Regarding your "all Cassandra" setup: We discuss deprecating old/legacy storage backends (Cassandra, Bigtable, DynamoDB, gRPC, BoltDB) in favour of having a single, opinionated way of running Loki (object store backed chunks and TSDB), which is capable of scaling and supports all the recently added features. So we probably don't revisit the docs for that part any more.

However, for the part of a more through documentation, I agree that there is demand. Having less configuration options should also help with putting out a solid, full configuration example.

JStickler commented 8 months ago

Closing as Cassandra was deprecated in 2.9. We do not plan any further development for Cassandra as a storage backend.