Firewall Map dont work + Sensei (Zenarmor) working?

[fietsDEV]

Hello,

first of all i want to say thank you for that great dashboard with a great instruction!

Now nearly everything works like a charm. Just the Firewall-Section dont show any data at all. Is it possible to connect this section with Sensei (Zenarmor) from my OPNsense? If, how?

Best regards

[fietsDEV]

Okay. I get Data now into the Firewall-Section. I forgot to install the content pack on graylog. But the Map dont get any circles.

if i search for src_ip_geo_country in graylog i dont find any entries.

[freddykromer]

I was working on the same problem for 5 hours, trying to understand the whole thing. In the end it was just the processing order in graylog. So graylog first has to recieve the messages and then put it through the pipeline, where the geoip-values are added. To fix it, go to System / Configurations - Configurations and click on update under Message Processors Configuration. Make sure that the Message Filter Chain is above the Pipeline Processor.

[ghost]

Hi I am in the same situation, ie pie charts of firewall blocks, but no map, event after following freddykromer's advice.

Please could some one run curl -X GET "localhost:9200/_cat/indices/*?v=true&s=index&pretty" and see if the indices are like mine? health status index uuid pri rep docs.count docs.deleted store.size pri.store.size green open gl-events_0 wITCrxNKTTSYLmw4s8WRtw 4 0 0 0 832b 832b green open gl-system-events_0 KFIyowpbQ_iF_T4B2J8JQQ 4 0 0 0 832b 832b green open graylog_0 FeCV5P9aSYiD6JzTDabUHw 4 0 579 0 330.1kb 330.1kb green open opnsense_filterlog_0 Mtp_EgDSR66ro7XtfXxfVA 4 0 76085 0 31mb 31mb

Thanks

[tonygauderman]

Here is what I see on the elasticsearch container.

[root@fe217dfe1daa elasticsearch]# curl -X GET "localhost:9200/_cat/indices/*?v=true&s=index&pretty"
health status index                uuid                   pri rep docs.count docs.deleted store.size pri.store.size
green  open   gl-events_0          odfDozrTRva0bNWYezUomw   4   0          0            0       832b           832b
green  open   gl-system-events_0   KEDc2JVrRimae3EASaWoEw   4   0          0            0       832b           832b
green  open   graylog_0            SImmUjwaTC-lKCEg98XznQ   4   0    1413127            0    600.7mb        600.7mb
green  open   opnsense_filterlog_0 3eyVphwoSpK3z80Tex-S8g   4   0          0            0       832b           832b

[ghost]

Thanks for the reply. At first sight it appears the graylog index is fuller than the opnsense index on yours, taking into consideration the probable difference between the two system re time and how many block rules are set up. This is the reverse situation to mine, so if yours is working ok?, then something is wrong. Its sort of suggests the issue in in the graylog area.

Thanks again

[tonygauderman]

Yea, I saw that too, and am wondering why my graylog is so full. I am sending IDS logs too but I don't think that explains all of it. I actually think something is not quite right with my setup either even though I am seeing the map in the dashboard.

[tonygauderman]

My stream wasn't set to use the opnsense index, so that data from my system isn't very useful to you. Seems to indicated that the data you are seeing there doesn't explain your situation.

[ghost]

I loaded a copy of the Grafana dashboard, and changed the terms to dst_ip_country_code. I now have data on the map.

[bsmithio]

I loaded a copy of the Grafana dashboard, and changed the terms to dst_ip_country_code. I now have data on the map.

I'm not sure where dst_ip_country_code is coming from. Possibly ZenArmor or Suricata. src_ip_geo_country is what is used for GeoIP on Graylog for the firewall logs. Here's how to search for it.

[ghost]

Hi

As far as I can tell its from the firewall, as port 9999 is blocked and the ip address is one the device uses to phone home. Zenarmour as far as I can tell is not sending anything to the elastic search used in the docker, which is on a separate system to opnsense. I followed the very well written instructions, and every thing worked perfectly, except the presentation of data on the map. Here is a sample from the stream

Timestamp 2022-04-28 17:13:10.000 Received by Syslog UDP on  ab39ba75 / 4343cbcb89a8 Stored in index opnsense_filterlog_0 Routed into streams • OPNsense / filterlog action block datalen 97 direction in dst_ip 51.132.0.248 dst_ip_city_name London dst_ip_country_code GB dst_ip_geolocation 51.5095,-0.0955 dst_port 9999 facility local0 facility_num 16 filterlog_ipv4_udp 689,,,9aedcb56418331258aa26bbd06774b6b,igb0,match,block,in,4,0x0,,64,4178,0,none,17,udp,117,192.168.1.107,51.132.0.248,15600,9999,97 flags none id 4178 interface igb0 ip_version 4 length 117 level 6 message opnsense.foo.ba filterlog[10297]: 689,,,9aedcb56418331258aa26bbd06774b6b,igb0,match,block,in,4,0x0,,64,4178,0,none,17,udp,117,192.168.1.107,51.132.0.248,15600,9999,97 offset 0 protocol_id 17 protocol_name udp reason match rule_number 689 source opnsense.foo.ba src_ip 192.168.1.xxx src_port 15600 timestamp 2022-04-28 17:13:10.000 +00:00 tos 0x0 tracker 9aedcb56418331258aa26bbd06774b6b ttl 64

[ghost]

Yea, I saw that too, and am wondering why my graylog is so full. I am sending IDS logs too but I don't think that explains all of it. I actually think something is not quite right with my setup either even though I am seeing the map in the dashboard.

Hi I believe Suricata is going to influxdb, not elastic search, as it is fed via telegraf. Annoyingly there is no influxdb/geoip support (well there is a plugin for telegraf but I never got it to work), so I decided to try the docker provided here. Afaik graylog does not have an influxdb output, and telegraf only has a graylog input for graylog metrics. I am looking to see if I can route suricata to graylog and elasticsearch to get geoip support for alerts and flows

PS just noticed opnsense telegraf (should be renamed as telegraf is an influxdb specific application) that you can feed data to graylog. I noticed you commented that you send IDS logs to graylog?, if so please can you tell me how you got it working, or the document you used.

Thanks

Aimee

[bsmithio]

A few things you could try:

1. Check if GeoIP database file is in /usr/share/graylog/data/data/ on your Graylog container.

sudo docker exec -it graylog /bin/bash

Then make sure GeoLite2-Country.mmdb is in /usr/share/graylog/data/data

2. Make sure Message Processors are in the right order like so:

3. You can test if GeoIP works by navigating to System -> Lookup Tables -> Title GeoIP -> Test Lookup then enter an IP address.

Here's a list of IP addresses by country that you can use to test. https://lite.ip2location.com/ip-address-ranges-by-country

You should get output similar to this:

[robertr1229]

In my docker container Graylog:5.0.5 the directory /usr/share/graylog/data/data did not exist I had to

docker exec -it /bin/bash mkdir /usr/share/graylog/data/data mv GeoLite2-Country_*/GeoLite2-Country.mmdb /usr/share/graylog/data/data/

[mitchweaver]

Having same issue. Nothing containing geo being sent to the graylog stream.
Everything else is coming in and looking fine.

Using the lookup table per @bsmithio's suggestion I do get a correct lookup result.

Map still empty:

[bsmithio]

Having same issue. Nothing containing geo being sent to the graylog stream. Everything else is coming in and looking fine.

Using the lookup table per @bsmithio's suggestion I do get a correct lookup result.

Map still empty:

Hi, try setting your message processors in System / Configurations -> Configurations -> Message Processors Configuration to this:

This should resolve that issue.

[mitchweaver]

@bsmithio wow!! It did!! Many thanks

and I see you fixed it in the configuration guide too 😄 8b98ed326f17d67c178c064c4dcb2f34bf1c2683

[michelpfirter]

Hi,

first of all, thank you @bsmithio for your work.

I was struggling with this issue also, having no firewall data shown at all in Grafana but having data in Graylog. For me it was to check the rfc5424 box in OPNsense Logging / targets what turned the logs in Graylog to have named fields (action, application_name and so on).

While writing this post I just noticed that the screenshot on your blog post is missing the check on rfc checkbox while in github it is checked correctly. Unfortunately I am not that experienced on the matter and was following the blog post instructions 😄

Furthermore the root users (admin) timezone setting in docker-compose.yml is not respected and was still UTC. To fix that, I had to edit the graylog.conf (/usr/share/graylog/data/config) and set to my timezone there. Since I mapped the containers data directory to a directory on the host, I did not need to do that for the graylog.conf file solely, which would be an alternative. However this is a minor thing, but I wanted to bring it to your attention and contribute ... where I am able to.

docker-compose.yml

...
  graylog:
    container_name: graylog
    image: graylog/graylog:5.0.2
    volumes:
      - ./graylog_data:/usr/share/graylog/data
#      - graylog_data:/usr/share/graylog/data
...

graylog.conf

...
# The time zone setting of the root user. See http://www.joda.org/joda-time/timezones.html for a list of valid time zones.
# Default is UTC
#root_timezone = UTC
root_timezone = <My/JodaTimezone>
...

Thank you