How to manage wildcards in the url path

[ThomasTJdev]

I have tried to find a way to manage the wildcards in routes and access their values in a "safe" way, but I don't believe I've achieved it yet. What is the best practices on using wildcards and accessing them? How do you do it? My only solution is by the PR below.

Info:

1. I would like to use named path-parameters as wildcards for visual identification. This should make it easier for me and my team to quickly identify the available path-parameters and route.
2. I would like to be able to implement a secure way to access the named path-parameters which can rely on the path-parameter name instead of the index of the wildcard in the url.

Solution with PR:

I have a PR with a couple of changes to mummy:

1. adding the .parts as a part of the request
2. allowing for named wildcards

Wildcards

Current With mummy I can use * as wildcards but I need to provide comments to identify them.

# Wildcard 1 => projectID
# Wildcard 2 => fileID
router.get("/project/*/file/*", handlerParam)

PR I have hundreds of routes and many of them rely on specific named parameters. An example could be:

router.get("/project/@projectID/file/@fileID", handlerParam)

Access original path

With the named path-parameters and access to the original path ( request.parts), it's possible to match the positions.

Current The request.uri* is public, but the uri changes according to the incoming data. E.g.

# Wildcard 1 => projectID
# Wildcard 2 => fileID
proc handler(request: Request) =
  doAssert request.uri == "/params/19/user/1233"
  let projectID = webby.parseUrl(request.uri).paths[1]
  let fileID = webby.parseUrl(request.uri).paths[3]

# Wildcard 1 => projectID
# Wildcard 2 => fileID
router.get("/project/*/file/*", handlerParam)

PR With the PR the original URI with named wildcard will be available in the new request.parts:

template `@`(str: string): untyped =
  ## Example on how to access path parameters.
  let pathIndex = request.parts.find("@" & str)
  if pathIndex == -1:
    ""
  else:
    webby.parseUrl(request.uri).paths[pathIndex]

proc handler(request: Request) =
  doAssert request.uri == "/params/19/user/1233"
  doAssert request.parts == @["project", "@projectID", "file", "@fileID"]
  echo @"projectID"
  echo @"fileID"

router.get("/project/@projectID/file/@fileID", handlerParam)

[guzba]

In my current project I just use either let split = request.uri.rsplit('/', maxsplit = 1) or let parts = request.uri.split('/'). I can do this since the path is something simple like /abc/*.

I have improving the router and URL wildcards on my "want to do list" so it is something I'm thinking about.

One thing I do suggest is not using parseUrl -- that generally should work but I know of issues with parsing there that may not produce exactly identical results to the routing which is not something I consider good. Unfortunatley that is just another thing that needs improving.

[guzba]

In general I suggest working on this as a routing project. There is no real need to make changes in Mummy. I suggest taking mummy/routers and making a new one that explores supporting the features you'd like. You can then make the callback called by the router have an additional parameter of parts or some richer data structure. This way the routing can be worked on without creating churn and obligations in Mummy.

[ThomasTJdev]

Thanks @guzba . I tried to replicate the routes.nim but my try on different callbacks interfered with the RequestHandler in the mummy.nim file.

But, by using the custom handler I could hack around it. This allowed me to use named paths for the wildcards and indexing params. Code below for reference for other:

import mummy, mummy/routers

import std/httpcore, std/strtabs, std/strutils

from webby import decodeQueryComponent

type
  RouteType* = enum
    Get
    Post

  Details* = object
    urlOrg*: string
    urlHasParams*: bool
    params*: StringTableRef

  CallbackHandler* = proc(request: Request, details: Details) {.gcsafe.}

template routerSet(
    router: Router,
    routeType: RouteType,
    route: string,
    handler: proc(request: Request, details: Details)
  ) =
  ## Transform router with route and handler.
  ## Saving the original route and including the `Details` in
  ## in the callback.

  # Saving original route
  var
    rFinal: seq[string]
    urlParms: bool = false
  for r in route.split("/"):
    if r.len() == 0:
      continue
    # Got @-path, replace with *
    elif r[0] == '@':
      rFinal.add("*")
      urlParms = true
    else:
      rFinal.add(r)

  # Generating routes
  case routeType
  of Get:
    router.get(
      "/" & rFinal.join("/"),
      handler.paramCallback(Details(
        urlOrg: route,
        urlHasParams: urlParms,
        params: newStringTable()
      ))
    )
  of Post:
    router.post(
      "/" & rFinal.join("/"),
      handler.paramCallback(Details(
        urlOrg: route,
        urlHasParams: urlParms,
        params: newStringTable()
      ))
    )

proc paramCallback(wrapped: CallbackHandler, details: Details): RequestHandler =
  ## Callback where the `Details` is being generated and params
  ## are being made ready.
  return proc(request: Request) =

    let uriSplit  = request.uri.split("?") #.split("/")

    # URL query: ?name=thomas
    if uriSplit.len() > 1:
      for pairStr in uriSplit[1].split('&'):
        let
          pair = pairStr.split('=', 1)
          kv =
            if pair.len == 2:
              (decodeQueryComponent(pair[0]), decodeQueryComponent(pair[1]))
            else:
              (decodeQueryComponent(pair[0]), "")
        details.params[kv[0]] = kv[1]

    # Body data: name=thomas
    if "x-www-form-urlencoded" in request.headers["Content-Type"].toLowerAscii():
      for pairStr in request.body.split('&'):
        let
          pair = pairStr.split('=', 1)
          kv =
            if pair.len == 2:
              (decodeQueryComponent(pair[0]), decodeQueryComponent(pair[1]))
            else:
              (decodeQueryComponent(pair[0]), "")
        details.params[kv[0]] = kv[1]

    # Path data: /project/@projectID/user/@fileID
    if details.urlHasParams:
      let
        urlOrg  = details.urlOrg.split("/")
        uriMain = uriSplit[0].split("/")
      for i in 1..urlOrg.high:
        if urlOrg[i][0] == '@' and urlOrg[i].len() > 1:
          details.params[urlOrg[i][1..^1]] = uriMain[i]

    wrapped(request, details)

template `@`(s: string): untyped =
  details.params.getOrDefault(s, "")

template resp(httpStatus: HttpCode, resp: string) =
  request.respond(httpStatus.ord, @[("Content-Type", "text/html; charset=utf-8")], resp)

proc indexCustom(request: Request, details: Details) =
  for k, v in details.params:
    echo $k & " = " & v
  echo "projectID:  " & @"projectID"
  echo "userID:     " & @"userID"

  resp(Http200, "Hello, World!")

var router: Router
router.routerSet(Get, "/project/@projectID/user", indexCustom)
router.routerSet(Post, "/project/@projectID/user", indexCustom)

[guzba]

Having put up with the split / rsplit stuff for a while now without liking it, I think named parameters are great so this looks very comfortable. Thanks for sharing.

[guzba]

A prototype of my own on this and some other improvements: https://github.com/guzba/mummy/pull/111