Allow Substitutions of Host Header

[jbarratt]

I have a use case that seems like it would be somewhat common, where I am trying to ensure 2 sites are identical before updating DNS, but the sites depend on the Host header being set, as they contain multiple vhosts.

I was able to hack around this running by hand in the simple use case by editing my /etc/hosts between runs, but I'm now looking to do this at a larger scale and that seems like it would be very fragile and hard to orchestrate.

It appears this can be overridden in Phantom driver in the onResourceRequested handler:

page.onResourceRequested = function(requestData, networkRequest) {
    ...
    networkRequest.setHeader('Host', newvalue);
    ...

Would it be reasonable to add a new config stanza for this?

hostHeaderOverrides:
    'elb.amazonaws.com': 'myrealdomain.com'

Then it could be conditionally tested for in the onResourceRequested method in the same way that that resourcesToIgnore is. I'm happy to work on a PR for this, but I wanted to make sure the approach was acceptable first.

[elsigh]

Could the test config just have a headers thing that was generic, e.g. headers = [{'Host', 'myrealdomain.com'}] and then for the ref_config or the run_config you could set that as needed?

[jbarratt]

Yeah, I love the idea of making a more generic header override.

I believe some kind of filter where you match on the hostnames (or some other facet) is still needed, because e.g. you wouldn't want to set a Host: myrealdomain.com header when you load assets that are actually hosted on other domains.

[elsigh]

Ah, now I understand this better - you want to be like a proxy so that when the browser wants to request a.com/b it actually loads b.com/b - yeah? SGTM

[jbarratt]

Yeah, exactly, the live site is on mydomain.com, and I want to make sure the one I just deployed to my-loadbalancer-1234567890.us-west-2.elb.amazonaws.com has the same content before I update the DNS for mydomain.com to point at it.

So I want to be able to run e.g.

./run_url_pair_diff.sh \
    --upload_build_id=1 \
    http://mydomain.com \
    http://my-loadbalancer-1234567890.us-west-2.elb.amazonaws.com

but in the second URL, I want to set Host: mydomain.com when I request it.