Regular expressions in routing rules and matching to class with matched parameter groups

[gitzdnex]

Hi so I decided to rewrite our apps from Tornado and webpy to Falcon.

While I was trying some setups which we are using all seemed good (generators, asgi,simple returns and classes). Then I have noticed that with normal add_route() and class and method on_get(self,req,resp,value): i can only use some kind of simple rules for validation and not regular expressions with more complex rules which we use a lot. After I found that there is something like add_sink() which allows regular expression matching. After test it seems that there is not method match and matched groups passing? So I tried to make that, but then I found that parameters matched, are not passed to function and only request and response is.

Falcon

class handler_function(object):
    restriction = ["GET","POST"]
    def __call__(self,req,resp):
        if req.method not in self.restriction:
            raise f.HTTPMethodNotAllowed
        return getattr(self,req.method)(req,resp)
    def GET(self,req, resp):
        resp.media = {
            'message': 'msg',
            'path': "",
        }

app = f.App()
app.add_sink(Handler(), re.compile(r'/api/([a-zA-Z0-9_-]{1,40})/(slow|fast)'))

Here is simple Tornado setup where matched groups are passed to functions (I am okay with request/respons as parameters )

class Handler(object):
    def get(self, product_id, action="slow"):
        if product_id not in db:
            raise NotFound(404)
        #use action for something 
        return db[product_id]
url = [(r'/api/([a-zA-Z0-9_-]{1,40})/(slow|fast)', Handler)]

So my question is is this expected and I should make again match and regular expression evaluation and match groups? Or I am completly wrong and I should use something else?

[vytas7]

Hi @gitzdnex! It's true that normal routes don't afford arbitrary regex out of the box. You can implement a custom converter, however, I understand that in some cases it is more convenient to simply use regex. One way of solving this via converters is implementing a generic regex converter, and using that in add_route(). Including such a converter in the framework itself is tracked as https://github.com/falconry/falcon/issues/857.

As you observed, sinks are only matched according to path, not the request's method.

However, named groups should get captured and passed as parameters. The following works for me:

import re

import falcon

class Handler:
    restriction = ['GET', 'POST']

    def __call__(self, req, resp, **kwargs):
        if req.method not in self.restriction:
            raise falcon.HTTPMethodNotAllowed
        return getattr(self, req.method)(req, resp, **kwargs)

    def GET(self, req, resp, **kwargs):
        resp.media = {
            'args': kwargs,
            'path': req.path,
        }

app = falcon.App()
app.add_sink(
    Handler(),
    re.compile(r'/api/(?P<id>[a-zA-Z0-9_-]{1,40})/(?P<action>slow|fast)'))

HTTP/1.1 200 OK
Connection: close
Content-Length: 98
Content-Type: application/json
Date: Thu, 05 May 2022 11:46:11 GMT
Server: gunicorn

{
    "args": {
        "id": "123456-just-testing",
        "action": "fast"
    },
    "path": "/api/123456-just-testing/fast"
}

[gitzdnex]

Hello,

ok this seems to be working. I just find out that I should add $ to mark end as this seems to be matching just from start. Is there actually plan to support regular expressions in add_route()? Also another question is about supporting not named groups?

Anyway thanks for information. I will see and try some another parts and then see.

[vytas7]

Hi again, yes, we are exploring various ways to make Falcon's routing more flexible.

As also alluded to in my previous reply, an easy way is to implement a re converter yourself (something that we are also looking at adding to the framework, itself, in https://github.com/falconry/falcon/issues/857).

You could build this along the lines of

import re

import falcon
import falcon.routing

class RegexConverter(falcon.routing.BaseConverter):
    def __init__(self, expr):
        self._pattern = re.compile(expr)

    def convert(self, value):
        match = self._pattern.match(value)
        if not match:
            return None
        # NOTE: we could also extract a specific group instead of passthrough.
        return value

class Resource:
    def on_get(self, req, resp, resourceid, action):
        resp.media = {
            'action': action,
            'path': req.path,
            'resourceid': resourceid,
        }

app = falcon.App()
app.router_options.converters['re'] = RegexConverter

app.add_route(
    "/api/{resourceid:re('[a-zA-Z0-9_-]+')}/{action:re('slow|fast')}",
    Resource())

Which seems to do the job for GET http://localhost:8000/api/123456-just-testing/fast:

HTTP/1.1 200 OK
Content-Length: 96
Content-Type: application/json
Date: Fri, 06 May 2022 19:08:53 GMT

{
    "action": "fast",
    "path": "/api/123456-just-testing/fast",
    "resourceid": "123456-just-testing"
}

But if the regex doesn't match, e.g., GET http://localhost:8000/api/123456-just-testing/medium:

HTTP/1.1 404 Not Found
Content-Length: 26
Content-Type: application/json
Date: Fri, 06 May 2022 19:10:54 GMT
Vary: Accept

{
    "title": "404 Not Found"
}

There currently seems to be some glitch with converter parameters inside { and } as we don't seem to understand any nested curly braces. I'll investigate and file an issue.

We are also exploring the option of allowing a single field to span multiple path segments, see this in-progress PR: https://github.com/falconry/falcon/pull/1945, possibly in conjunction with regex which could provide even more flexibility.

If you have any other ideas, don't hesitate to chime in this discussion too: Routing improvements! :bulb:

[CaselIT]

We are also exploring the option of allowing a single field to span multiple path segments, see this in-progress PR: https://github.com/falconry/falcon/pull/1945, possibly in conjunction with regex which could provide even more flexibility.

At first this would support only a single regexp though, since the converters that span multiple paths must currently be at the end of the template

[vytas7]

Duplicates #857