elastic / elasticsearch

Free and Open Source, Distributed, RESTful Search Engine
https://www.elastic.co/products/elasticsearch
Other
1.15k stars 24.84k forks source link

elastic not respecting the nonproxyHost variable, breaking use of internal minio/S3 service #80805

Open desnij opened 2 years ago

desnij commented 2 years ago

Description of the problem including expected versus actual behavior:

Elastic correctly respects the proxyPort variables, but does not respect the nonproxyHost variables.

Issue

The repository-s3 plugin fails to work if the S3 bucket can not be reached via the proxy. This could be remedied by accepting the standard nonproxyHost variable.

Environment

  1. Our elastic cluster can only reach the internet via our proxy.
    • So we include all the proxyPort variables ES_JAVA_OPTS, including nonproxyHost (in hope)
    • Our proxy will not connect to internal infrastructure for security reasons.
  2. We have a minio server running inside the network.

Steps to reproduce:

Any activity destine for our minio server

PUT _snapshot/s3a
{
"type": "s3",
"settings": {
  "bucket": "debug",
  "endpoint": "minio.int",
  "path_style_access": "true",
  "chunk_size" : "10m",
  "max_restore_bytes_per_sec" : "10m",
  "delegate_type" : "s3",
  "buffer_size" : "10m",
  "base_path" : "/",
  "max_snapshot_bytes_per_sec" : "10m",
  "protocol": "https"
  }
}

Provide logs (if relevant):

logs

there is no traffic sent to minio.int but we do see traffic heading to our proxy

Tcpdump

$ tcpdump port 3128
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vlan44, link-type EN10MB (Ethernet), capture size 262144 bytes

15:43:00.776617 IP kube-r2-41.42874 > squid.int.squid: Flags [S], seq 487418113, win 29200, options [mss 1460,sackOK,TS val 25352817                                            61 ecr 0,nop,wscale 9], length 0
15:43:00.777502 IP squid.int.squid > kube-r2-41.42874: Flags [S.], seq 3982747686, ack 487418114, win 28960, options [mss 1460,sackO                                            K,TS val 2976999167 ecr 2535281761,nop,wscale 9], length 0
15:43:00.777564 IP kube-r2-41.42874 > squid.int.squid: Flags [.], ack 1, win 58, options [nop,nop,TS val 2535281762 ecr 2976999167],                                             length 0
15:43:00.778189 IP kube-r2-41.42874 > squid.int.squid: Flags [P.], seq 1:114, ack 1, win 58, options [nop,nop,TS val 2535281763 ecr                                             2976999167], length 113
15:43:00.778800 IP squid.int.squid > kube-r2-41.42874: Flags [.], ack 114, win 57, options [nop,nop,TS val 2976999168 ecr 2535281763
...<snip>

Squid logs

==> /var/log/squid/access.log <==
1637182013.043      1 10.4.2.41 TAG_NONE/503 0 CONNECT minio.int:443 - HIER_NONE/- -
1637182013.063      0 10.4.2.41 TAG_NONE/503 0 CONNECT minio.int:443 - HIER_NONE/- -
1637182013.142      0 10.4.2.41 TAG_NONE/503 0 CONNECT minio.int:443 - HIER_NONE/- -
1637182013.346      0 10.4.2.41 TAG_NONE/503 0 CONNECT minio.int:443 - HIER_NONE/- -
elasticmachine commented 2 years ago

Pinging @elastic/es-distributed (Team:Distributed)