open-telemetry / opentelemetry-collector-contrib

Contrib repository for the OpenTelemetry Collector
https://opentelemetry.io
Apache License 2.0
2.73k stars 2.16k forks source link

Fetch a GeoIP Database for MaxMind GeoIP Processor #33510

Open rogercoll opened 3 weeks ago

rogercoll commented 3 weeks ago

Component(s)

processor/geoip

Is your feature request related to a problem? Please describe.

Initial issue: https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/32663

Currently, a local file path must be provided to the MaxMind GeoIP processor in order to retrieve the associated IP metadata. This approach requires users to manually download and manage the GeoIP database files, ensuring they are kept up-to-date and properly licensed. This manual process can be cumbersome and error-prone. Automating the retrieval of the GeoIP database would significantly streamline the setup and maintenance process.

Describe the solution you'd like

The idea would be to initially download the latest available database from MaxMind's Direct download link: https://dev.maxmind.com/geoip/updating-databases#directly-downloading-databases. The following configuration options will be needed:

Describe alternatives you've considered

TODO: Verify if it would be feasible to use the auth extension instead of the custom account_id and license_key configuration options.

Additional context

Automatic database fetch

If neither a local path nor a license key is specified, the processor should be able to fetch a default GeoIP database automatically, similar to Elasticsearch's geoip processor. This feature would eliminate the need to specify a custom configuration.

Elasticsearch automatically downloads updates for these databases from the Elastic GeoIP endpoint: https://geoip.elastic.co/v1/database. The processor could use the same endpoint to automatically download the latest available database.

Licensing

There are some concerns regarding the implementation of this feature if we use Elastic's https://geoip.elastic.co/v1/database as default download endpoint. Could there be any licensing implications by using the same URL? MaxMind databases are shared under the CC BY-SA 4.0 license.

github-actions[bot] commented 3 weeks ago

Pinging code owners for processor/geoip: @andrzej-stencel @michalpristas @rogercoll. See Adding Labels via Comments if you do not have permissions to add labels yourself.