host_matching is in Werkzeug since 0.7 but not in Flask

[constb]

having 'example.com' and 'example.org' running on the same server with the same data backend I thought it would be simple to serve both from a single Flask app, especially since Werkzeug has host_matching but it isn't.

I currently workaround this by 'injecting' hostname into urls with nginx's url rewrites but this is a bit too hackish, you know.

I would like to have syntax like this:

@app.route("/", host="example.org")
def hello():
    return "Hello World!"

[Poincare]

Are example.com and example.org serving the same urls? (I'd assume not)

[constb]

No, it's different sites but with a lot of shared data. So it's reasonable to serve them from the same flask-application. Something similar to vhosts I already have in nginx, but in a back-end.

[Poincare]

Can you check out the above solution and see if it works for you?

[constb]

Um, no. 'host_matching' is a parameter of werkzeug.routing.Map's constructor, not 'add' method's. Also it worth noting that host matching and subdomains feature are mutually exclusive, so keeping it always on in flask code is somewhat backwards incompatible.

[constb]

I think adding a parameter to flask.app.Flask constructor would help. So I could write like:

app = Flask(__name__, host_matching=True)

and when it creates url_map, it would just pass it on.

[constb]

It also seems that code that handles app.route's has to be changed as well. Because this doesn't work either:

from flask import Flask
app = Flask(__name__)
app.url_map.host_matching = True

@app.route("/", host="example1.local")
def hello1():
    return "Hello @ example1!"

@app.route("/", host="example2.local")
def hello2():
    return "Hello @ example2!"

if __name__ == "__main__":
    app.run()

All I see with it is a bunch of 404s.

[Poincare]

The @app.route code just passes on a options dict it gets; I'll try to figure out what's going wrong (and, I had put in a request for the wrong commit; it should have been in the Map constructor).

[kennethreitz]

This would be quite nice.

[kennethreitz]

@mitsuhiko any thoughts on this (and #499)?

[untitaker]

It seems to me that the reason @constb 's example failed was because host_matching also matches the port. host="example1.local" therefore only applies if you access port 80/443. The code in routing.py looks like this is on purpose.

[andrewcrook]

I got this working awhile back host does indeed require the fully qualified domain name and port after host_matching is enabled. Flasks routing API could be updated to take advantage of the underlying setup of werkzeug and provide it in a simpler format.

Note: you can also use pattern matching with host

[untitaker]

I don't consider it a feature, but rather a pitfall that Werkzeug's host parameter implicitly matches the port. IMO it should only match ports if i explicitly specify one.

EDIT: Especially since there's no mention in the werkzeug docs, at least not at http://werkzeug.pocoo.org/docs/routing/

[andrewcrook]

@untitaker Because of this and other reasons with other development platforms when I need a web server running I tend to develop and test using proxy forwarding facing ports 80 and 443, as you say by default Werkzeug makes the assumption your using port 80 unless specified in host (I think that is correct having a job to remember off the top of my head its been awhile).

I don't have a problem with the way Werkzeug' works but perhaps a future version of Flask could break the FQDN/port down into route "host", "port" a optional specific port per route parameter, optional simple HTTPS matching parameter, use a optional global default port variable, if that's not specified use the same port that Flasks web server uses and port 80 as default in development.

Just thinking off the top of my head really need to sit down and think about it....

"""globaldefaultport = 8080 so uses port 8080"""
@app.route("/", host="example1.local")

""" assumes 80 in development otherwise uses Flasks web servers port"""
@app.route("/", host="example1.local")

""" simple per route port matching"""
@app.route("/", host="example1.local", port=8080)

""" simple HTTPS support, port 443"""
@app.route("/", host="example1.local", https=True)

""" and don't break my old code"""
@app.route("/", host="example1.local:8080")

... you get the gist.

[untitaker]

@andrewcrook What did you mean with "pattern matching" before? Could you give an example?

[andrewcrook]

Sure, just like you do with route paths. Goes along the lines of...

using host_matching

example grab subdomain alternative

@app.route("/", host="<mysubdomain>.test.local:8080")
def test(mysubdomain):
    return 'You are reading from subdomain ' + mysubdomain

Works for *.test.local:8080 and returns subdomain

Another...

@app.route("/", host="example<id>.test.local:8080")
def test(id):
    return 'You are reading from example  ' + id

Works for example*.test.local:8080 returns id eg example1.test.local:8080 returns 1

[untitaker]

How did you find that out? I can't find it in the docs... found it. It's there where you expect it the least -- outside a class doc...

[andrewcrook]

Yeah I spent a while looking at the docs and code of Flask and Werkzeug. Only thing I have not tried is a regex map converter. I presume it will work on host like it does with the path.

[yawor]

You can also use pattern matching with port numer:

@app.route("/", host="example<id>.test.local:<port>")
def test(id, **kwargs):
    return 'You are reading from example  ' + id

I've added **kwargs to function definition, but you can use port argument explicitly. Either way you can just ignore it.

Another way to just ignore host matching for some routes is to define host as

'<host>:<port>'

and add **kwargs to function definition. This can be used as a catch all solution for example.

[andrewcrook]

@yawor Yes you can pattern match any part of the host string. Thanks for the catch all idea I'll have to try that next time.

[yawor]

I've forgot to mention that pattern matching for host's port part is the workaround for explicit port problem when you test application on different port than 80/443 :). Thanks to that there is no need to put explicit port number in routing rules as any port will match the pattern.

[miracle2k]

The host handling does have some unfortunate kinks.

• I agree, the port should be implicit. I am doing this right now as a workaround:

  @app.url_value_preprocessor
  def port_catchall(endpoint, values):
    if values:
        values.pop('port', None)
  
  @app.url_defaults
  def add_port(endpoint, values):
    if app.url_map.is_endpoint_expecting(endpoint, 'port'):
        values['port'] = (request.host.rsplit(':', 1)[1:] or ['80'])[0]
  
  @app.route(host='test.localhost:<port>')
  ...

• I am not even sure if I like the fact that host_matching requires an explicit host for all rules. I think I sort of get it, at the same time the ability to add a "host restriction" to a set of rules, and have the rest function as usual feels "right".

It seems to be that a big step to making this usable in practice would be to allow blueprints to be mounted on a domain. I didn't find a simple way to do this without modifying Flask:

https://github.com/miracle2k/flask/commit/55b92e99eec5bcbd15b3404f5335fe13da08b3df

This allows me to do:

if in_production:
    app.register_blueprint(shop, host='localhost:<port>')
    app.register_blueprint(info, host='info.localhost:<port>')
else:
    app.register_blueprint(shop)
    app.register_blueprint(info, url_prefix='info')

Still, there are some issues:

• Because every rule needs a host matching (and to be generated), all app-global rules need one as well. There should be a way to set it once. In particular, there is no equivalent to default_subdomain in werkzeug.routing.
• In particular, the app's auto-generated static rule is a problem. I'm working around this by not using any global static files.

[miracle2k]

Another argument for why ports should be implicit: It turns out that when running the app using lighttpd/fastcgi/flup, server_name does not contain a port, so the rules mustn't contain one either to match.

[andrewcrook]

I actually have an application which requires non-standard ports with a software client app along side ports 80/443. Therefore, there is a need for an option to specify them within rules.

I am always developing multiple domain and sub domain applications as well. I also hit limitations of the current patterns and have to move to regular expressions This is why I think the routing needs to be as flexible as possible. I hate having to use my solution above.

Looked at how some other frameworks written in other languages such as PHP do this. One PHP framework that I really like the routes are all converted to regular expressions and the core of the framework deals with that, however, of course, its not using a web server framework like Werkzeug for this functionality its built in.

As we are using a web server framework, I am wondering if Werkzeug could use a custom regex converter with the full URL (or host to be used with a URL Path custom converter)?

I haven't had time to look deeper at Werkzeug's rule and routing source to see how its working but I know you can already use them with the URL Path as that's Flasks current regex pattern solution you can hack but it's not enough.

Anyway If so, Flask's routing class could be rewritten to convert higher level patterns including optional options for domains, subdomains, ports, extended custom patterns for matching, global and local settings into regular expressions (unless already in regex format) then a dynamic Werkzeug converter could handle them for the actual routing. Flasks router could be made very flexible and even have an option(s) for extending. Other parts of Flask could also benefit, such as blueprints.

If Werkzeug can't use such a converter on the full URL (or host) perhaps we need to post a request?

TBH the issue here is really around Werkzeug as it is inflicting limitations in the routing rules, unless Flask has missed something like the above.

Anyway, I know it would be a lot of work but it would worth it.

(Apologies this was written in a rush)

[andrewcrook]

@miracle2k in your example I agree should be handled by global subdomain/domain/path options or a ruby style block wrapping the blueprint definitions.

[yawor]

I've been working on an application that had to have 3 different behaviors depending on http host:

• a main site without any specific domain (no need to link to it from other two parts),
• a secondary, simple site for specific domain (e.g. example.com),
• sites for users as a subdomains of secondary site (e.g. foo.example.com).

I've tried to do it by defining host arg for routes like this:

• host='<host>:<port>' for main site and then ignore both args,
• host='example.com:<port>' for secondary site and ignore port arg,
• host='<user>.example.com:<port>' for user sites and ignore port arg too.

But I've run into rule ordering problem in werkzeug. For some reason werkzeug routing (which should order rules depending on their specificity level) ordered rules this way:

1. all rules from user sites,
2. all rules from main site,
3. all rules from secondary site, effectively disabling access to the secondary site (requests to example.com where routed to main site). I think that specificity if not detected well in the host part in werkzeug.

So I decided that I'll just use specific domain for main site (e.g. mainsite.com). But then I run into a problem with Flask static routes (I don't remember what exactly was wrong right now). After some digging I've extended the Rule class this way:

from werkzeug.routing import Rule

class CustomRule(Rule):
    def match(self, path):
        if self.host is None:
            path = path[path.index('|'):]
        return super(Rule, self).match(path)

    def build(self, values, append_unknown=True):
        rv = super(Rule, self).build(values, append_unknown)
        if rv is None:
            return None
        domain_part, url = rv
        return domain_part if domain_part else None, url

and then

Flask.url_rule_class = CustomRule

just before creating Flask object.

This way all static routes (and all routes without host arg set), should match for any domain. But be aware that it also matches domains for which you have explicit host rules (so in my example mainsite.com/static/style.css and example.com/static/style.css will match the same rule and use the same style.css file).

[andrewcrook]

@yawor I think that order should be

1. all rules from user sites,
2. all rules from secondary site,
3. all rules from main site,

?

otherwise, the main site rule will activate before the secondary site rule can be matched. Specific rules at the top generic at the bottom, rules are in a dictionary so process in the order that they are defined.

I agree order will be complex with larger applications and complex rules which is why any new routing system needs to work with multiple apps, blueprints, globals or ruby style blocks to separate them out and simplify things.

Thanks for the code example I'll check it out sometime. I got some ideas to try out if successful I will submit to github with a link posted here.

[yawor]

@andrewcrook yes, that should be the correct order, but the default rule ordering in werkzeug got it wrong.

Right after I've sent my previous post I quickly checked how the sorting works. Without analyzing the whole algorithm I think the problem with host rules is that when rule is compiled (Rule.compile method) the weights for sorting are treated the same way for the domain_rule part as for the rule part. I need to look deeper into that. Maybe I can achieve what I need by providing custom logic for Rule.build_compare_key and Rule.match_compare_key methods.

I think that it would be nice to have ability to provide additional value to override rules sorting in special cases (priority?). For example the rule with lower number assigned would be checked before rule with greater number, and rules with same number would be sorted with current algorithm. The priority would have to have some default value if not provided by the programmer, so the logic would be the same whether the priorities are provided or not (group rules by priority, order groups by priority, order rules in groups depending on their content like it's done right now, flatten groups into single list preserving group order).

[untitaker]

Can this finally be closed down? If anybody has objections against the current behavior, a new issue or a discussion in IRC is probably more appropriate.

[mitsuhiko]

I'm all for closing this. If it continues to bug people we can reopen it.

[miracle2k]

I certainly think some changes are necessary here. The API issues can largely be worked around. But unless a blueprint can be mounted under a particular domain, the host matching functionality seems unusable in practice to me - and that part, as far as I can tell, can not be done without Flask changes, which is why I am currently running a patched version with this commit:

https://github.com/miracle2k/flask/commit/55b92e99eec5bcbd15b3404f5335fe13da08b3df

I'd be happy to add tests and make it into a pull request.

[mitsuhiko]

Yeah, if you can add tests and a pull request that would be appreciated.

[justinmayer]

@miracle2k: Hey Michael. Regarding the domain-based blueprint functionality you mentioned, do you think you would be willing to add tests and submit your pull request? Please let me know if there's anything I can do to help.

[miracle2k]

@justinmayer I'm unfortunately not going to find the time anytime soon :/

[orome]

This looks like exactly what I need; but I can't get it working. What am I missing?

[blark]

@miracle2k I was wondering if you could help me out. I'd really like to register Blueprints with a hostname or even host pattern, but for some reason it doesn't seem to be working.

When I do this it works fine:

@foo.route('/', host='<host>.bar.com')

but when I try to remove that host=... and add it to the register_blueprint as you have up in the comments it stops matching. I'm left scratching my head. Any ideas?

[jacobsvante]

Really missing this feature in Flask. Feels like less of a hack compared to only working on the sub-domain level.

[antoine-lizee]

I needed to specify a few routes that do host matching, while having the rest of my app behaves as if there was no host matching enabled. I would love to have it in flask, and first in werkzeug for that matter, but I implemented something out-of-lib for now. This solves the problems of using blueprints for me because the blueprints all need to have the default behavior in my case (matching anything).

The solution (gist with tests) looks like @yawor, but in addition lets me specify hosts that shouldn't be matched if not explicitly specified.