hostname on the header is required to forward the request for the proxy server

[rkang]

Hi, Is there a way to setup the proxy server to not require the hostname on the header to forward request?

I created proxy with "java -jar ./mockserver-netty-jar-with-dependencies.jar -proxyPort 1090 -proxyRemotePort 80 -proxyRemoteHost www.mock-server.com" and tested with "curl -v "http://localhost:1090/index.html" --header "Host: www.mock-server.com" to see proxy server successfully forward the request. but when I tried "curl -v "http://localhost:1090/index.html" only, I received:

• Trying ::1... TCP_NODELAY set Connected to localhost (::1) port 1090 (#0) GET /index.html HTTP/1.1 Host: localhost:1090 User-Agent: curl/7.54.0 Accept: /

< HTTP/1.1 404 Not Found < connection: keep-alive < content-length: 0 <

Connection #0 to host localhost left intact

I am trying to hit the proxy server from my mobile application and was hoping not to modify hostname from the request header on the app side.

[jamesdbloom]

The key issue here is not related to MockServer but caused by the AWS CloudFront serving the content for www.mock-server.com which uses the Host header to understand what traffic to serve.

You can test this by using curl directly without MockServer, as follows:

1. retrieve IP address for www.mock-server.com

$ nslookup www.mock-server.com
Server:     192.168.1.1
Address:    192.168.1.1#53

Non-authoritative answer:
Name:   www.mock-server.com
Address: 52.222.232.209
Name:   www.mock-server.com
Address: 52.222.232.161
Name:   www.mock-server.com
Address: 52.222.232.83
Name:   www.mock-server.com
Address: 52.222.232.163
Name:   www.mock-server.com
Address: 52.222.232.11
Name:   www.mock-server.com
Address: 52.222.232.231
Name:   www.mock-server.com
Address: 52.222.232.76
Name:   www.mock-server.com
Address: 52.222.232.114

2. curl www.mock-server.com using an IP address

$ curl -v http://52.222.232.114/index.html >> /dev/null
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying 52.222.232.114...
* TCP_NODELAY set
* Connected to 52.222.232.114 (52.222.232.114) port 80 (#0)
> GET /index.html HTTP/1.1
> Host: 52.222.232.114
> User-Agent: curl/7.57.0
> Accept: */*
> 
< HTTP/1.1 403 Forbidden
< Server: CloudFront
< Date: Tue, 06 Feb 2018 21:03:43 GMT
< Content-Type: text/html
< Content-Length: 551
< Connection: keep-alive
< X-Cache: Error from cloudfront
< Via: 1.1 1b61c3f085e80ef7e1cd6e9ac0cdfa77.cloudfront.net (CloudFront)
< X-Amz-Cf-Id: ks9ZbJX1qclVQ0OsK5266IdG1Tk94PECPTsLpach9ccSAiSYUBEcLQ==

The response returned is identical as when you curl via MockServer. However if you curl adding the Host header then the index.html is returned, as follows:

3. curl using IP address adding Host header

$ curl -v http://52.222.232.114/index.html --header "Host: www.mock-server.com" >> /dev/null
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying 52.222.232.114...
* TCP_NODELAY set
* Connected to 52.222.232.114 (52.222.232.114) port 80 (#0)
> GET /index.html HTTP/1.1
> Host: www.mock-server.com
> User-Agent: curl/7.57.0
> Accept: */*
> 
< HTTP/1.1 200 OK
< Content-Type: text/html
< Content-Length: 23508
< Connection: keep-alive
< Date: Tue, 06 Feb 2018 20:57:44 GMT
< Last-Modified: Mon, 25 Dec 2017 08:03:32 GMT
< ETag: "4ffb02e1c9946a8c0cf5cf45510df61e"
< Accept-Ranges: bytes
< Server: AmazonS3
< Vary: Accept-Encoding
< Age: 473
< X-Cache: Hit from cloudfront
< Via: 1.1 00956c6ec4781959da21fba8cd5dffda.cloudfront.net (CloudFront)
< X-Amz-Cf-Id: MkkDF7_MOsDUkX4Gk9S4abHQUewFndgkEMov_Ctq1SUtZLVssJYzWQ==

I am closing this issue as it is not a bug, but please re-open or open a new issue if you disagree or have more evidence.

[rkang]

@jamesdbloom I am still confused about why proxy didn't forward the request to the remote host. I started mock server with following.

"java -jar ./mockserver-netty-jar-with-dependencies.jar -serverPort 1080 -proxyPort 1090 -proxyRemotePort 80 -proxyRemoteHost ec2-34-211-137-207.us-west-2.compute.amazonaws.com"

And I tried to hit the server with the mobile app, proxy client did not forward the request to the remote host and I got the following.

2018-02-06 15:11:41,038 INFO o.m.m.HttpStateHandler no matching expectation - returning:

{
  "statusCode" : 404,
  "reasonPhrase" : "Not Found"
}

for request:

{
  "method" : "GET",
  "path" : "/manual/auth",
  "queryStringParameters" : {
    "format" : [ "xml" ],
    "api" : [ "0.31" ],
    ":mobile" : [ "true" ]
  },
  "headers" : {
    "content-length" : [ "0" ],
    "Accept" : [ "application/xml" ],
    "Connection" : [ "keep-alive" ],
    "User-Agent" : [ "TableauMobile/2 CFNetwork/893.14 Darwin/16.7.0" ],
    "Host" : [ "localhost:1090" ],
    "Accept-Language" : [ "en-us" ],
    "Accept-Encoding" : [ "gzip, deflate" ]
  },
  "keepAlive" : true,
  "secure" : false
}

[jamesdbloom]

The proxy did forward the request to the remote host. However the remote host responded with 404.

That is because if you send a request to CloudFront without a Host header it will always respond with 404.

Therefore if MockServer forwards a request without the Host header AWS CloudFront responds with 404.

Does that make sense?

[jamesdbloom]

That will also be the same for ec2 instances behind an ELB / ALB because the Host header is used for routing. So without the Host header the routing fails and AWS returns 404.

[rkang]

thanks, that makes sense now.

[xkamil]

I'm facing same issue. I have to add host header to request containing target host. The problem is I can't modify client I'm testing. Is it possible to override host header when proxying it via mockserver?

[celsogbj]

I have the same problem here. It would be nice if we could config the Host header on fowarded request in order to easly solve this issue.

Something like this:

mockserver.proxyRemotePort=443
mockserver.proxyRemoteHost=foo.com
mockserver.overhideHostHeader=true

setting overhideHostHeader to true would set the Host header with the value on proxyRemoteHost property.