search

elastic / elasticsearch

Free and Open Source, Distributed, RESTful Search Engine
https://www.elastic.co/products/elasticsearch
Other
1.45k stars 24.88k forks source link

[DOC] Wording in GeoIP instructions for database download can lead to confusion #86964

Open lucabelluccini opened 2 years ago

lucabelluccini commented 2 years ago

Elasticsearch Version

7.14+

Installed Plugins

No response

Java Version

bundled

OS Version

N/A

Problem Description

The documentation page https://www.elastic.co/guide/en/elasticsearch/reference/current/geoip-processor.html#manage-geoip-database-updates contains Use a proxy endpoint, but this can be intended/misread as a real HTTP Proxy.

Instead, the setting ingest.geoip.downloader.endpoint can only be used in concomitance with the solution Use a custom endpoint or putting in place a reverse proxy.

According to the issue https://github.com/elastic/elasticsearch/issues/77069, HTTP Proxy support is not yet available.

I think we would need to have settings similar to the ones in Watcher (https://www.elastic.co/guide/en/elasticsearch/reference/current/notification-settings.html#general-notification-settings) (e.g. xpack.http.proxy.*).

Steps to Reproduce

N/A

Logs (if relevant)

No response

elasticmachine commented 2 years ago

Pinging @elastic/es-docs (Team:Docs)

elasticmachine commented 2 years ago

Pinging @elastic/es-data-management (Team:Data Management)

smalenfant commented 2 years ago

I updated the other issue. Reverse proxy endpoint can't work when using 2 metadata with a different endpoint. Also the logging for error updating geoip database [GeoLite2-ASN.mmdb] java.net.SocketTimeoutException: Connect timed out doesn't say anything about URL. Where is it trying to download it from?

jeanfabrice commented 1 year ago

As of April 2023, 8.x documentation is still misleading about the use of Use a proxy endpoint

fbaligand commented 1 year ago

By the way, it would be great to have a real "proxy" setting, so that Elasticsearch can download database updates using a http proxy.

yomduf commented 7 months ago

Fixed by https://github.com/elastic/elasticsearch/pull/107094

@fbaligand as a workaround you can configure the proxy via the jvm.options

-Dhttp.proxyHost=https://urloftheproxy.com
-Dhttp.proxyPort=9999
-Dhttps.proxyHost=https://urloftheproxy.com
-Dhttps.proxyPort=9999

This workaround may affect other http call initiated by Elasticsearch

fbaligand commented 7 months ago

Thanks for the workaround @yomduf !

bojian-ben-li commented 2 months ago

Fixed by #107094

Very confused with this fix. It looks the doc fix only appear in 8.13 version doc. Or does any real code fix in 8.14 and 8.15?

(1) On the 8.13 version doc: https://www.elastic.co/guide/en/elasticsearch/reference/8.13/geoip-processor.html#use-proxy-geoip-endpoint

Use a reverse proxy endpoint If you can’t connect directly to the Elastic GeoIP endpoint, consider setting up a secure reverse proxy. You can then specify the reverse proxy endpoint URL in the ingest.geoip.downloader.endpoint setting of each node’s elasticsearch.yml file.

(2) On the 8.11, 8.12, 8.14 or 8.15 version doc: https://www.elastic.co/guide/en/elasticsearch/reference/master/geoip-processor.html#use-proxy-geoip-endpoint

Use a proxy endpoint

If you can’t connect directly to the Elastic GeoIP endpoint, consider setting up a secure proxy. You can then specify the proxy endpoint URL in the ingest.geoip.downloader.endpoint setting of each node’s elasticsearch.yml file.

bytebilly commented 6 days ago

The latest documentation is missing this change and should be updated. @joegallo do you know why this is happening?