GeoIP download will fail if airgapped environment only allows access to geoip.elastic.co

[philippkahr]

Elasticsearch Version

8.1.1

Installed Plugins

No response

Java Version

bundled

OS Version

macOS 12.3

Problem Description

When starting Elasticsearch it tries to download the GeoIP Package from geoip.elastic.co as documented here. However this is just a "redirect" to a google storage.

Thus in an airgapped environment one must allow access to geoip.elastic.co and the storage.googleapis.com which opens up a ton of possible security issues.

The geoip package should be served through the geoip.elastic.co URL.

curl -s 'https://geoip.elastic.co/v1/database?elastic_geoip_service_tos=agree' | jq
[
  {
    "age": 127526,
    "md5_hash": "e6bef7d0e11181b37f0288353f07179f",
    "name": "GeoLite2-ASN.tgz",
    "provider": "maxmind",
    "updated": 1648684830,
    "url": "https://storage.googleapis.com/elastic-paisano-production/maxmind/GeoLite2-ASN.tgz?X-Goog-Algorithm=GOOG4-RSA-SHA256&X-Goog-Credential=elastic-paisano-production%40elastic-apps-163815.iam.gserviceaccount.com%2F20220331%2Fhenk%2Fstorage%2Fgoog4_request&X-Goog-Date=20220331T000031Z&X-Goog-Expires=86400&X-Goog-SignedHeaders=host&X-Goog-Signature=3dcf41eae54a1be2cbe546e82489b91177e9f70b8338d98add6d5773ced2d8e10b11adcad03fca3acbc0842a60cc2dddc35955ef6d22388e92bcfa9cec10b002b2b597fff3ffd340748631dae4aedc81d379e59d1c27875fa65a4db0e432c9780b1b373253f31d2d6f7a8f586fa0c912ea3b397d6327ba4a50ba1c17a1b5a170ab5bfe79549ed88657780bfd0c810d57f616c21786f9f34382e94d560728b7d9f720b41fc98d960f83b64462c62d31eb5d4206be52c9c2524104f3ac8d12672efb88d1a3f1e93e4dcb892905fbf13820eb12c1a3bddaad29d992fed9d47630599e7d063dc3220494937fc10e85422a8295291ba83792e0b5effa072bf6e2ab0c"
  },
  {
    "age": 125938,
    "md5_hash": "444ac9617d53977802cb05754069d9a2",
    "name": "GeoLite2-City.tgz",
    "provider": "maxmind",
    "updated": 1648684830,
    "url": "https://storage.googleapis.com/elastic-paisano-production/maxmind/GeoLite2-City.tgz?X-Goog-Algorithm=GOOG4-RSA-SHA256&X-Goog-Credential=elastic-paisano-production%40elastic-apps-163815.iam.gserviceaccount.com%2F20220331%2Fhenk%2Fstorage%2Fgoog4_request&X-Goog-Date=20220331T000031Z&X-Goog-Expires=86400&X-Goog-SignedHeaders=host&X-Goog-Signature=075371fe882f58e0e3bfd27039d36b938f6aa97a05e76d0dcebe7866598d88bad6e11872d2a186a7a5ea6d06cb3af0dfebcbe42c79a5f6d719c9b0a7306486c67b4d808342692bb6ac2267b73c317708a479146be84faaec116365361aff8e5301a81a01aa9eb35a7e786dad1d0503420ee40d3dfecbda1612bfc279265edf0600b172544b48d63d512d30cd72815e783fe6c62ba4829e43d3892c57ce498a140d38a28feec1927c8b199288542242f6c0e6d8b772899af5cceed9996fabfcdbccc5f29702b13878d70878bd4d72c88df87249c27925e53ae83e2d831339e5275219498985d11ae7b3a9fd6145e5dba1adb1c5549d664154495f5a3dd1910736"
  },
  {
    "age": 126158,
    "md5_hash": "c7407c37660fdb158fb4d448f782d478",
    "name": "GeoLite2-Country.tgz",
    "provider": "maxmind",
    "updated": 1648684829,
    "url": "https://storage.googleapis.com/elastic-paisano-production/maxmind/GeoLite2-Country.tgz?X-Goog-Algorithm=GOOG4-RSA-SHA256&X-Goog-Credential=elastic-paisano-production%40elastic-apps-163815.iam.gserviceaccount.com%2F20220331%2Fhenk%2Fstorage%2Fgoog4_request&X-Goog-Date=20220331T000029Z&X-Goog-Expires=86400&X-Goog-SignedHeaders=host&X-Goog-Signature=293fa3197465c60221304d3659096ce844e566ea4e4273cf3f438f213fd1bd2ef645d30d2ffc08a9e863bc90657c8061b3b07d879976f98a02dd2805cd1a44749e8add4cbe368efd4d49774a1f00488f66aa97955c9f272baaf0b7f403c5ec7f2c18c2451764a6f35985b0e6cd35895ebf186b514842ed0424f7fcd621c818ecbe1a5e15d5bf4213286529161456e2b3bc6369f5dba3f84ab5e3641114f831d0b71e1879916564a9786904f911cae58d6ebe4a1e1533d2c43c46fc2d15a92db5ebe1cd80dc7792d31499a29086f1f008171a730e46a1ee85a980996526c3f53239ab6ab1a4e7943e55302ffa67ec0492f761eec6fcdcef9f0bfe3ef9dbd6a172"
  }
]

Steps to Reproduce

^^

Logs (if relevant)

No response

[philippkahr]

Discovered by @smnschneider

[elasticmachine]

Pinging @elastic/es-data-management (Team:Data Management)