Sick hacks for vmini HTTP / HTTPS / ALLOWED_HOSTS / Django mess

[sraustein]

See #628 for background.

The problem in vmini-land is that, to everybody except we sysadmin types, the rest of the world just sees vmini and its ports. Many of these ports magically tunnel to other (virtual) places in wonderous and interesting ways, but this is all a bit much for confused applications and users to understand.

In this case, Django is complaining because some poor user didn't realize that saying https://cache0.vmini.rpki.net/ would really take him to https://ca.vmini.rpki.net/. There's an equivalent problem with equally confused users who try to connect to http://ca0.vmini.rpki.net/ and find themselves at http://cache0.vmini.rpki.net; Django isn't involved in this one, so it doesn't complain, but the user is equally confused.

There are two levels of fix available here.

1. If the goal is just to get Django to stop whining at Randy, we can do that easily enough just by adding cache0.vmini.rpki.net to the list of names Django knows to accept as proper. That is, instead of the current rpki.conf setting of {{{ allowed-hosts = ca0.vmini.rpki.net }}} we would use {{{ allowed-hosts.0 = ca0.vmini.rpki.net allowed-hosts.1 = cahce0.vmini.rpki.net }}} Note that this just silences the whining, it doesn't help the user at all.
2. If we want to things work as expected for the user, we need to figure out what the user was trying to do and issue a redirect if the user ended up in the wrong place. We can do this with the dreaded Apache mod_rewrite module. Details remain to be worked out, but the basic idea is that the https vhost on ca0.vmini would check the client's HOST_NAME header and, if that header indicates that the client was really trying to talk to cache0.vmini, would redirect the user (probably with code 301) to http://cache0.vmini.rpki.net/. Yes, redirecting https to http, guess that's one way to see if the user is awake. Similarly, the http vhost on cache0.vmini would have mod_rewrite hacks to check the HOST_NAME and redirect http://ca0.vmini.rpki.net/ to https://cache0.vmini.rpki.net/.

The second approach is probably the correct one in terms of user experience, although it will take some careful debugging (mod_rewrite is notoriously easy to misconfigure). The price is a bit more complexity for us, the maintainers, but there's not much to be done about that as long as we're trying to run this whole lab-in-a-box universe.

Trac ticket #629 component testbed priority minor, owner sra, created by sra on 2013-09-23T20:38:43Z, last modified 2014-01-26T04:07:04Z

[sraustein]

vmini-land has an additional problem: how is ca0.vmini even supposed to know what vmini's public-IP-address-du-jour is? I suppose we could add some horrible kludge where ca0.vmini runs a script to probe vmini's public interface address and uses that to set an Apache environment variable, or something equally perverse. Note that this would likely require restarting ca0.vmini's httpd every time vmini gets a new DHCP lease with a different address.

Not sure how common this nonsense of sending an IP address instead of a proper hostname is. I would not be surprised if it's purely a malware thing, although I also would not be surprised if Internet Exploder really does do it. The only case we might care about is if legitimate users show up at tutorials with Internet Exploder as their only browser.

For the specific case of the vmini lab, are we willing to live with the restriction that if somebody shows up at a tutorial with nothing but a browser that issues IP address HTTP_HOST headers and it's somebody for whom we really want this to work, we will edit ca0.vmini's rpki.conf to add the public-address-du-jour?

Trac comment by sra on 2013-09-24T01:51:34Z

[sraustein]

It's not the NAT that makes vmini so much fun. It's that the NAT itself travels.

Trac comment by sra on 2013-09-24T01:54:42Z

[sraustein]

The only case we might care about is if legitimate users show up at tutorials with Internet Exploder as their only browser.

they do

if somebody shows up at a tutorial with nothing but a browser that issues IP address HTTP_HOST headers and it's somebody for whom we really want this to work, we will edit ca0.vmini's rpki.conf to add the public-address-du-jour?

survivable.

note that poor littl evmini will find its public address being behind a nat, where the browser clients will likely be there with it.

Trac comment by randy on 2013-09-24T02:06:07Z

[sraustein]

note that poor littl evmini will find its public address being behind a nat, where the browser clients will likely be there with it.

"public" is perhaps not quite nuanced enough for these awful network topologies. In this context "public" just means "approximately the same addressing realm as the browsers that are trying to talk to us".

Trac comment by sra on 2013-09-24T03:06:56Z

[sraustein]

The only case we might care about is if legitimate users show up at tutorials with Internet Exploder as their only browser.

they do

Note that we don't really know that it's Internet Exploder that's the problem, for all we know it's malware masquerading as IE, or a really old version of IE that no sane person uses anymore.

Simplest approach is to play this by ear and hack allowed-hosts on the fly if and only if a legitimate user has problems.

Trac comment by sra on 2013-09-24T03:08:32Z

[sraustein]

So it turns out that we do not need mod_rewrite for this, there's a cleaner solution, albeit a weird one: add a virtual host to ca0.vmini's Apache configuration for cache0.vmini:443, said virtual host containing nothing but a redirection to http://cache0.vmini.

This is now in place and seems to have stopped the Django whining.

Oddity: for reasons which I suppose make sense to the package maintainer, the Ubuntu apache2 default configuration enables name based virtual hosting for port 80 but not for port 443. Our standard configuration doesn't appear to care very much, but for this hack I had to enable it. More on that in another ticket.

Anyway, the redirection seems to work. That is, ca0.vmini successfully tells the browser to go talk to cache0.vmini.

The bad news is that connections from the outside world to cache0.vmini seem not to be working. Works fine within vmini-land (eg, using w3m) but something is blocking port 80. Dunno whether that's vmini or something upstream, quite possibly the latter.

There's a bit of clean-up left to do here, but now that the basic configuration is working, I should be able to finish this without generating further annoying messages.

Trac comment by sra on 2013-09-24T20:00:43Z

[sraustein]

tcpdump on vmini says the block on port 80 is upstream of vmini, probably something on Randy's border.

Trac comment by sra on 2013-09-24T20:15:12Z

[sraustein]

tcpdump on vmini says the block on port 80 is upstream of vmini, probably something on Randy's border.

i intended tls only

Trac comment by randy on 2013-09-24T20:18:26Z

[sraustein]

tcpdump on vmini says the block on port 80 is upstream of vmini, probably something on Randy's border.

i intended tls only

OK, but the way you have things set up on vmini that means that cache0.vmini's web server is inaccessible from the outside, full stop. If that is not also what you intended, you may want to rethink some portion of this.

Trac comment by sra on 2013-09-24T20:38:48Z

[sraustein]

sigh. so what external port should i redirect to vmini to get to cache0 http[s]?

currently, 443 goes to https://ca0.vmini

Trac comment by randy on 2013-09-24T20:45:11Z

[sraustein]

ok, firewall redirects port 80 to vmini which does xinetd to cache0.vmini

Trac comment by randy on 2013-09-24T20:51:42Z

[sraustein]

http://cache0.vmini.rpki.net/rcynic/

works

Trac comment by randy on 2013-09-24T20:54:40Z

[sraustein]

ok, firewall redirects port 80 to vmini which does xinetd to cache0.vmini

And redirect from https://cache0.vmini.rpki.net/ to http://cache0.vmini.rpki.net/ now works as expected. OK, so it looks like this approach does work. Whew.

Remaining stuff to do on cache0.vmini now, theoretically all minor:

• Add alias so that http://cache0.vmini lands at rcynic status rather than default Apache "it worked!" page.
• Add vhost to redirect redirect http://ca0.vmini to https://ca0.vmini.

Trac comment by sra on 2013-09-24T20:59:55Z

[sraustein]

All redirections are now in place on both ca0.vmini and cache0.vmini.

The one thing we should consider before we declare this done is whether we care that people who mistakenly connect to https://cache0.vmini.rpki.net/ will get a certificate warning from their browser when they receive the certificate for ca0.vmini.rpki.net. We could fix this by adding cache0.vmini.rpki.net to the subjectAltName extension for this certificate. Not sure we really care, since

• Using that URL is a mistake in the first place, and
• Anybody but us is probably going to get a certificate warning anyway because the certificate came from Randy's CA rather than one of the big commercial CAs.

Other than that, I think we're done here.

Trac comment by sra on 2013-09-24T22:23:11Z

[sraustein]

Having seen no ringing exhortations that we must do something about the certificate warning, I declare this done.

Trac comment by sra on 2013-09-24T22:44:10Z

[sraustein]

(In #624) Actually, I should have left this depending on #628, as this was one of the numeric HTTP_HOST cases rather than the vmini-specific confusion. But both are fixed at this point, so closing.

Trac comment by sra on 2013-09-24T22:46:15Z

[sraustein]

Looks like we not only lost the entire hand-tweaked Apache configuration on {cache0,ca0}.vmini when we restored the vimages from an old backup, we also managed to revert to a half-assed ancient configuration that prevented apt-get upgrade from installing a normal current configuration. Sigh.

Just barely possible that I have a backup of the Apache configurations somewhere, it's the sort of thing I might have stashed somewhere just in case.

Trac comment by sra on 2014-01-26T00:35:49Z

[sraustein]

Just barely possible that I have a backup of the Apache configurations somewhere

Nope.

Trac comment by sra on 2014-01-26T00:37:56Z

[sraustein]

can we wipe and just restart?

Trac comment by randy on 2014-01-26T00:41:01Z

[sraustein]

can we wipe and just restart?

You'll have to be a little more specific. vmini is a very kinky environment because of what it does and where it goes; blowing away the entire whole environment and starting over would involve serious work (weeks if not months).

Right now I've got rpki-ca and apache shut down on ca0.vmini, am cleaning up old stuff, then will apt-get to get to current software.

Trac comment by sra on 2014-01-26T01:08:28Z

[sraustein]

Right now I've got rpki-ca and apache shut down on ca0.vmini, am cleaning up old stuff, then will apt-get to get to current software.

that is what i meant by wipe and restart. and i think we can easily generate csv files to load.

Trac comment by randy on 2014-01-26T01:20:15Z

[sraustein]

OK, I think this is fixed (again).

Trac comment by sra on 2014-01-26T04:07:04Z

[sraustein]

Closed with resolution fixed