geoip2 filter causes a lot of warning when cityName not provided.

[WENPIN1]

[2023/01/05 11:55:42] [ warn] [filter:geoip2:geoip2.0] cannot get value: The lookup path does not match the data (key that doesn't exist, array index bigger than the array, expected array or map where none exists) {"time":"2023-01-05T03:55:41.894644Z","ip":"8.8.8.8","country":"United States","isocode":"US","cityName":null,"longitude":-97.822,"latitude":37.751}

[WENPIN1]

Bug Report

Describe the bug When the GeoIP2 filter is enabled, a lot of warning message appeared in the stardard error as folllows; [2023/01/05 11:55:42] [ warn] [filter:geoip2:geoip2.0] cannot get value: The lookup path does not match the data (key that doesn't exist, array index bigger than the array, expected array or map where none exists) {"time":"2023-01-05T03:55:41.894644Z","ip":"8.8.8.8","country":"United States","isocode":"US","cityName":null,"longitude":-97.822,"latitude":37.751}

To Reproduce

filebeat 2.0.8 with Database GeoLite2-City_20221230/GeoLite2-City.mmdb downloaded from maxmind Steps to reproduce the problem: Configure Fluent-bit as follows:

[INPUT] name Dummy Dummy {"ip":"8.8.8.8"} Tag dummy.warn

[INPUT] name Dummy Dummy {"ip":"128.101.101.101"} Tag dummy.ok

[FILTER] Name geoip2 Match *

Database GeoLite2-City_20221230/GeoLite2-City.mmdb

Database GeoLite2-City.mmdb
Lookup_key ip
Record country ip %{country.names.en}
Record isocode ip %{country.iso_code}
Record cityName ip %{city.names.en}
Record longitude ip %{location.longitude}
Record latitude ip %{location.latitude}

[OUTPUT] Name stdout Match * Format json_lines Json_date_key time json_date_format iso8601

Expected behavior

All not found warning messages should have a property to control its showing.

Screenshots

[github-actions[bot]]

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 5 days. Maintainers can add the exempt-stale label.

[github-actions[bot]]

This issue was closed because it has been stalled for 5 days with no activity.

[0xErnie]

We just stubled over this error flooding our logs. We mitigated it by setting Log_Level error on the filter like this:

[FILTER]
    Name geoip2
    Match *
    Database /var/lib/GeoIP/GeoLite2-City.mmdb
    Lookup_key Source_IP
    Record country Source_IP %{country.names.en}
    Record city Source_IP %{city.names.en}
    Record isocode Source_IP %{country.iso_code}
    # Ignore all warnings, otherwise log will be flooded when no city is found.
    Log_Level error

The error is produced in the imported library https://github.com/fluent/fluent-bit/blob/5626757c8a580accda3a072d920c63998071b3b7/plugins/filter_geoip2/libmaxminddb/src/maxminddb.c#L2145-L2147 which could be updated. Source is available at https://github.com/maxmind/libmaxminddb

Maybe it would help to add a property to catch this error 'MMDB_LOOKUP_PATH_DOES_NOT_MATCH_DATA_ERROR'. Documentation to usage of the library and it's error codes can be found here: https://github.com/maxmind/libmaxminddb/blob/main/doc/libmaxminddb.md#status-codes