philhagen / sof-elk

Configuration files for the SOF-ELK VM
GNU General Public License v3.0
1.47k stars 276 forks source link

GeoIP Documentation inconsistent #237

Closed mthbrown closed 2 years ago

mthbrown commented 2 years ago

Hi,

I copied over an http log to /logstash/httpd and although it was picked up by Filebeat and sent to Logstash, there is no GeoIP enrichment. Here is an example of the output: https://pastebin.com/RVcYm0Qr

As you can see, source_geo:asnstr is set to ASN: Not Available and there is no GeoIP data.

I thought this might have to do with requiring a Maxmind license so I created an account and created / updated /etc/GeoIP.conf and successfully ran: sudo /usr/bin/geoipupdate. I then sent Logstash a config reload command (SIGHUP signal) and copied a new file into the ingestion point but it gave the same result it previously did. Any ideas? Thanks.

philhagen commented 2 years ago

Hello - please use the included geoip_bootstrap.sh script - this will ensure the files are created in the right places, etc.

I need to better express that in the documentation, though - so I'll keep this issue open and try to get that knocked out soonest.

mthbrown commented 2 years ago

Thanks. Tested it out and it worked after running the script. Out of curiosity, is there a reason it doesn't use the default ones that come bundled with Logstash out of the box? Vanilla Logstash allows this without requiring you to create an account.

philhagen commented 2 years ago

The license changed and the last version of the database that uses the old license (aka is redistributable) is old and outdated. This is a problem for all GeoIP-enabled platforms, unfortunately. Rather than enable a function by default that is known to provide suspect results, we opted to provide no enrichment data until the feature is explicitly enabled by the user.

mthbrown commented 2 years ago

Thank you for the clarification + all the work you've put into SOF-ELK.

philhagen commented 2 years ago

re-opening to track need for documentation update/clarification