search

kubernetes-retired / spartakus

[EOL] Anonymous Usage Collector
Apache License 2.0
74 stars 30 forks source link

Documenting the usage of database backend options #31

Closed mhausenblas closed 5 years ago

mhausenblas commented 6 years ago

Currently there exists no documentation for database backend options, esp. around the HTTP option. Create user-friendly docs incl. example in a docs/ dir.

Original issue for clarity reasons:

~Currently, the aggregation and reporting back-end is hardwired to BigQuery. It would be very beneficial, for example for on-prem deployments or individuals who don't want to or can not use BQ to make Spartakus support different back-ends in a pluggable manner. For example, one might want to use Minio+PrestoDB or simply a RDBMS such as Postgres.~

mhausenblas commented 6 years ago

@thockin WDYT? Also, I seem not be able to self-assign this one?

squat commented 6 years ago

@mhausenblas currently the way to do this is to use the provided http database option. Both volunteers and collectors can report directly to an http endpoint, which means the record is simply POSTed as JSON to the provided URL. Someone wanting to report to a custom database could have volunteers report to an http service that accepts records and sends them off to the desired backing database. This is all the collector does in reality.

This repo is very slim and I think that unless any plugin system is likewise simple and slim it would be better to encourage use of the http backend as the plugin system. For example, users could deploy a Postgres Spartakus “plugin” as a sidecar to the volunteer that listens on some port, e.g. 8443 and configure the volunteer to post to localhost:8443.

mhausenblas commented 6 years ago

Thanks a lot for the speedy reply @squat much appreciated. I must have overlooked that HTTP database option, can you please point me to the docs for it?

squat commented 6 years ago

@mhausenblas there are no user-friendly docs covering database options at the moment :/ this would be a good addition to the project. To enumerate the options, the best choice is to spartakus volunteer —print-databases

mhausenblas commented 6 years ago

Thanks for taking the time to sync on this topic @squat and I've changed the title and description. Can you please review and either LGTM it or add as you see fit?

mhausenblas commented 6 years ago

A few things that should be documented in addition to above:

thockin commented 6 years ago

docs are good

I am OK to add more backend modules, but we should refactor away from a monolith package into an interface and a bunch of plugin implementations. but yeah, it's basically an HTTP-to-whatever adaptor.

fejta-bot commented 5 years ago

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle stale

fejta-bot commented 5 years ago

Stale issues rot after 30d of inactivity. Mark the issue as fresh with /remove-lifecycle rotten. Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle rotten

fejta-bot commented 5 years ago

Rotten issues close after 30d of inactivity. Reopen the issue with /reopen. Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /close

k8s-ci-robot commented 5 years ago

@fejta-bot: Closing this issue.

In response to [this](https://github.com/kubernetes-incubator/spartakus/issues/31#issuecomment-504624313): >Rotten issues close after 30d of inactivity. >Reopen the issue with `/reopen`. >Mark the issue as fresh with `/remove-lifecycle rotten`. > >Send feedback to sig-testing, kubernetes/test-infra and/or [fejta](https://github.com/fejta). >/close Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.