Container Group, A-Series and exposing configuration as JSON

[Sturgelose]

Hello there!

First, thanks a lot for the dashboards, they works amazingly! However, I have some comments overhere:

1. Container Group tab: how to use it? In some dashboards there is a "Container Group" tab in the top, usually empty. It tries to fetch this value from Prometheus, but it doesn't even exist.. Also, I have Rancher exporting enabled, but still there is nothing inside there. How can I populate that selector?

2. A-Series: are they implemented? There are some series that show random stuff such as A-Series, and editing it, it shows comments on fake data. Is it still in implementation, or it's just missconfiguration?

3. Exposing the configuration as JSON instead of DB

Could we expose the configuration as a JSON? doing so as a DB, this means that every time the container gets updated, the users get reseted and all the custom configuration disappears. Maybe it would be better to have it as a JSON, and load/update it to Grafana in a more upgradable way.

Anyways, thanks a lot again for your work. Also, I'd not have any problem to do some pull requests to the repository to do part of this fixes :)

Have a nice week!

[Rucknar]

Hey @Sturgelose,

I'll run through your feedback,always open to pull requests though if you wish to make improvements.

1. This is something i spotted on Friday as well. I've imported that dashboard from the Grafana catalog. It seems however to need some tweaking. Out of interest, are the hosts your monitoring running RancherOS?

2. Looks like a bug to me, i'll re-create and push up a fix.

3. This was my original plan, at the time of creation. However i hit issues because the datasources can't be populated the same way. Grafana does have the ability to load dashboards from a directory of JSON files, but not datasources. The caveat being that this was the case when i last tried.

I've seen various methods of interacting with the API once online but this felt more convoluted and would require much more code outside the standard grafana Docker file.

If there is a way though, perhaps some new config parameters then i'd be happy for it to switch over!

[Sturgelose]

Hey @Rucknar,

Thanks a lot for the fast answer!

1. No, we using AWS-linux which is simply Ubuntu. We wanted now something easy to use and we will try later with CoreOS or RancherOS if we have time. When checking in Prometheus, the rancher_container_group key doesn't exist, so I'm not quite sure from where it should be populated..

2. Thanks a lot!

3. I'll take a look, or maybe we can open an issue to Grafana. Otherwise, maybe a startup script detecting if the datasources config is loaded could be an option.... But I'll try to do some research :)

EDIT: found this: https://github.com/grafana/grafana/issues/5674 I may take a look and do a pull request there, if I have some time to understand grafana, which being Javascript, I think it will be easy for me :D

Again, thanks a lot!

[Sturgelose]

Mini update:

I tried to figure out the container_group label. Seems, you have to add this label to each of the containers to make it work, or at least this is what it seems even if it doesn't really work out..., and still not sure if it's rancher, docker, cadvisor or who is loading this label into prometheus.

Proposal: use container_label_io_rancher_project_name or similar to filter. This allows to filter by Stack (or even by service if wanted), and labels are added automagically by Rancher (so less configuration).

In the templating section, change $containergroup to have the value of label_values(rancher_stack_state{state="active"}, name), and then in the graphs where we need to filter, add a filter of container_label_io_rancher_project_name=~"$containergroup".

For example, sum(rate(container_network_receive_bytes_total{name=~".+",container_label_io_rancher_project_name=~"$containergroup"}[$interval])) by (name)

What do you think?

[Rucknar]

Sounds perfect! Fancy a pull request?

[Sturgelose]

I'll do it in a while :) It needs lots of handmade changes in the config, and when I have that, then I'll push the reference JSONs :) This way at least there can be some tracking of changes JSON-wise, because the DB is RAW :D

[Rucknar]

@Sturgelose Had a look through the dashboard in more detail. It's simply got way too many errors in how it's put together. It's one i imported from the grafana.net community. I'll work on re-doing the dashboard and i'll use the labelling as you proposed.

[andrewsav-bt]

Adding a container_group does not seem to do anything. Has anyone figured out how is this supposed to work?