Compatibility with influxdb 2.0

[faandg]

Hi,

Just wondering, any plans to switch to Flux queries to support influxdb 2.0 or are we waiting for influxdb 2.0 to also support influxQL (somewhere later this year)?

Thanks for creating such an amazing dashboard btw.

[VictorRobellini]

I haven't messed with flux but I've read good and bad things about it. I'm not a developer and outside of this project, I don't work with Influx, which limits my time learning it. If I had the time to invest to learn more about Influx I would focus on structural issues first. Better indexing, TSI, and downsampling and retention. Currently, the database will just grow until you run out of space. Fixing those issues would be a priority over flux. I would love to take advantage of the join capabilities and other bells and whistles, but first I would want to make sure the data itself is in a good place. My current solution is to just drop the db when it gets to be too large, which sucks.

[VictorRobellini]

Aside from the above commentary, it looks like Influxdb 2.0 supports InfluxQL. I can't say if you could just drop it in and it would work, but it's very probable that it would just work.

[faandg]

I can certainly respect that choice.

I ran into this bit of information This was a while ago though so it could be it now supports InfluxQL and the errors I saw when I tested it were related to something else. I'll take another look soon.

[faandg]

As per the screenshot, you are correct and influxdb 2.0 should now support InfluxQL.

However, the problem seems to be that the Grafana datasource configuration does not allow me to select InfluxQL and connect to an InfluxDB v2.0 using v2 configuration settings. It assumes that, because I want to use InfluxQL, that I also want to connect to an InfluxDB 1.x:

Choosing InfluxQL changes the "InfluxDB details" section to:

So selecting InfluxQL prevents you from using InfluxDB 2's token, organization and bucket configuration.

In contrast, selecting Flux gives you token, organization and bucket configuration:

I will open an issue for this on the Grafana side because I believe it should be made possible.

[VictorRobellini]

Thanks for the update. Interesting find. Can you post a link to your issue? I'm interested in seeing how it plays out.

[faandg]

Here you go: https://github.com/grafana/grafana/issues/32183 I imagine it might take some time though, seeing the amount of open issues at the moment.

[faandg]

They basically said "hey this is possible, here's a blog" and closed the ticket. Took me a while to figure everything out though. Could write another blog about this :)

In the InfluxDB v2 UI: 1) Create an admin user (I use influxadm), organization (I use 'mist') and bucket (I use 'pfsense') 2) Data-> Tokens, generate a READ token for grafana 3) Data-> Tokens, generate a READ/WRITE token for telegraf (WRITE alone might be sufficient but haven't tested it)

In pfsense: 4) The telegraf package UI does not support outputting to v2 yet, so the output needs to be added via the 'Additional configuration for Telegraf' Services -> Telegraf, remove everything related to output and deselect influxDB (ctrl + click)

Take the Telegraf config from this repo and add the output part:

[[outputs.influxdb_v2]]
  urls = ["http://INFLUXDB_HOST:8086"]
  token = "TOKEN"
  organization = "mist"
  bucket = "pfsense"

! Replace TOKEN with the token from step 3 and INFLUXDB_HOST with your InfluxDB v2 host.

5) InfluxDB v2 uses organization + bucket and v1 uses database + retention policy (dbrp). Grafana expects we configure a v1 object when using InfluxQL as query language. To make this work we have to create an object in InfluxDB which is basically a mapping between v2 and v1 so it also exposes a v1 endpoint. This cannot be done in the InfluxDB UI afaik, you need to create this mapping using the API or the CLI. I used the CLI. a) create config to authenticate via CLI (use the organization and admin user's token you created in step 1)

influx config create --config-name influxadm_config \
  --host-url http://INFLUXDB_HOST:8086 \
  --org mist \
  --token TOKEN \
  --active

b) create the mapping (you can get the bucket-id and the org-id from the InfluxDB UI as well):

influx v1 dbrp create \
  --db pfsense \
  --rp autogen \
  --bucket-id 00oxo0oXx000x0Xo \
  --org-id 00oxo0oXx000x0Xo \
  --default

Side note: You need to specify an rp (retention policy). On my first attempt I created one called pfsense-rp, but after loading the dashboard, the errors showed me that some panels were expecting an rp called 'autogen', which is the default rp in InfluxDB v1. Hence on my second attempt I named it autogen and the query errors were gone. I have no clue what the actual retention is on this thing or if it differs from the bucket's settings.

6) Creating the Grafana datasource

• set a name
• set your query language to InfluxQL
• set the URL
• set a custom HTTP Header 'Authorization' with value 'Token <actual token you created in step 2>' (I messed up this part and it took me a while to figure out)
• set the database name to the name of your bucket
• you don't need user and password since we're using a token
• HTTP GET or POST, both work for this dashboard
• test and save

And holy ****, it works.

PS: I am unable to test the pfBlocker panel because I don't have pfBlocker active at the moment.

[vavsaftoiu]

Hi there,

Thanks a lot for sorting this out. Incredible work and really nice that you shared the solution.

[lgwapnitsky]

I've been working on rewriting this in Flux, and have forked the repo. See issue #44 for more info

[rpclarke]

@faandg - Thank you so much! I spent hours yesterday morning trying to figure this out and came across this post. This worked for me.