Followed Install Guide - No Dashboard Info

[JSylvia007]

Howdy there!!

I followed your guide, and I have all the necessary data being collected in Graylog (I think), but Grafana doesn't show anything in the dashboard.

Any idea what I'm doing wrong? Or even where to look?

[samara8609]

@opc40772 I think it may be broken due to upgrade to newer version of pfsense, i get the same thing, it almost looks like the template isn't taking.

[opc40772]

@JSylvia007 Excuse me the delay. Did you follow the whole guide?

[opc40772]

@samara8609 Our pfsense is in its latest version. Regards

[JSylvia007]

@opc40772 I did. I finally found out what the issue was... First, there are additional items that need to be installed from the command line... Grafana Panels. I needed these:

grafana-cli plugins install grafana-piechart-panel
grafana-cli plugins install grafana-worldmap-panel
grafana-cli plugins install savantly-heatmap-panel

Secondly, I needed to change the Pipeline order to be last inside Graylog.

I still don't have geographic location (as you can see below), BUT, I do see some information now...

How does Geo information get added?

[opc40772]

By default graylog does not bring a database for geo localization so we must download it from MaxMind City databases. wget -t0 -c http://geolite.maxmind.com/download/geoip/database/GeoLite2-City.tar.gz Next we extract the contents of the file tar -xvf GeoLite2-City.tar.gz and we copy the GeoLite2-City.mmdb file to /etc/graylog/server. Note that the folder created when decomposing GeoLite2-City.tar.gz may differ from the tutorial on the date. cp GeoLite2-City20180306/ GeoLite2-City.mmdb /etc/graylog/ server Now in the graylog website we must activate the Geo-Location Processor and go to system / Configuration. We can see that it is disabled, we press the Update button. We activate it by checking in Enable Geo-Location porcessor what else we leave by default. As long as messages containing ips are coming, they will obtain additional data such as coordinates, country code, city name.  I hope this helps you. Best regards

[JSylvia007]

@opc40772 Yea... I did all that and just verified it... it's not being used for some reason...

[opc40772]

@JSylvia007 Did you fill the field "Path of the database of MaxMind"?

[JSylvia007]

@opc40772 Yes, and I verified that it's correct.

[opc40772]

Go to the pfsense greylog stream and verify that the fields are being generated

They are generated automatically.

Regards.

[opc40772]

@JSylvia007 They should already be shown in the Grafana

[JSylvia007]

@opc40772 I figured it out!! GeoIP needs to come after Pipeline in Graylong Config... It's all working!

You're the greatest. This is some amazing visibility.

[opc40772]

@JSylvia007 Message Processors Configuration Order

[opc40772]

@JSylvia007 Thanks, is a pleasure.

[opc40772]

@JSylvia007 If you wish and you encourage to implement an IDS in this case Suricata in your pfsense and have it also in grafana here I leave the link. https://github.com/opc40772/suricata-graylog

[samara8609]

@opc40772 The last problem i seem to have is that when i want recent results i have to set the time in both graylog and grafana to minus 5 hours, its like its trying to compensate for UTC, was there a setting you had to set to deal with that?

[opc40772]

My pfsense pipeline is

rule "timestamp_pfsense_for_grafana" when has_field("timestamp") then // the following date format assumes there's no time zone in the string let source_timestamp = parse_date(substring(to_string(now("America/Habana")),0,23), "yyyy-MM-dd'T'HH:mm:ss.SSS"); let dest_timestamp = format_date(source_timestamp,"yyyy-MM-dd HH:mm:ss"); set_field("real_timestamp", dest_timestamp); end

You must set your correct timezone. In my case was America/Havana

Regards

[samara8609]

@opc40772 I got it figured out, i'm running in docker and all my containers were running on UTC timezone, once i got it fixed to my TZ they all show correctly.

[opc40772]

@samara8609 Ok. Regards.

[e-d-i-t]

I had many problems getting everything to work so I decided to install everything from scratch on a single ubuntu server. Though I am merely an MS admin, padding my shoulder as after 5 days I got it working. No building revproxy, got one externally. Now I can harden stuff with the firewall within ubuntu and set some apps to localhost and such... One thing which still kept me with empty screens was the timestamp_pfsense_for_grafana... Don't use it. Set server and every app you install to your local timezone and use "timestamp" as field for creating Grafana Data Source...