Improve developer ergonomics with easier access to query parameters, cookies, etc

[ajusa]

I recently ported over a small project of mine (https://github.com/ajusa/250) from Jester to Mummy, and ended up creating a file (https://github.com/ajusa/250/blob/master/src/mummy_utils.nim) which has several utility functions that I thought were "missing" from mummy itself.

It'd be nice if some of these could be added to mummy itself - or if there were easier ways to set cookies, do redirects, and parse objects from query parameters/post bodies. Those were my main pain points.

I believe you mentioned on Discord that you were exploring a pattern to handle parsing query parameters and path parameters, if that's the case that would probably fix most of the issues I've described above.

[guzba]

Thanks for linking to this file, it is useful to see. I think me continuing to make more examples will go a long way to help in these cases.

I consider it a great thing you were able to add those conveniences in just a handful of lines.

Why I'm less than eager to build these in directly:

1) Redirects -- they are very simple, it can be 1 line. It is also the case that people need to specify 301 vs 302 and may want to add additional headers. For these reasons, I currently think this is best shown in a new example instead of trying to create a proc that ends up taking everything as an optional parameter.

2) Cookies -- this should be its own little import in webby so you can choose to use that one or choose to use the Nim stdlib version. I like keeping the choice open.

3) URL / URI handling -- again, it is a webby or Nim stdlib option. Request will have .uri on it and you can choose to parse it in your handler. This is important because parsing that .uri could fail and I consider it important that the programmer has the option to choose how to handle that.

4) getOrDefault is a totally reasonable improvement, should add that to webby etc.

I don't write this as a dismissal, just sharing how I'm thinking now. All of this will get better, it takes a long time to really nail all the aspects of server QoL and I am not for sure not there yet!

[guzba]

I have added a couple examples, one showing redirecting, the other showing an approach to custom handler types with additional parameters https://github.com/guzba/mummy/pull/78

[ajusa]

I looked through #78 and I like the custom handler approach, I'll have to try it out. Never thought to use callbacks in that way.

For the points you raise though:

1. I'm a bit split on this - on one hand I agree, on the other hand redirects are so common that I feel that including a slightly opinionated way of doing them (while still allowing additional headers like you mentioned) would be useful. I can see myself misspelling "Location" and not realizing it, it'd be nice if I didn't have to worry at all about the specific header value.
2. Agreed, once webby supports cookies that would be pretty nice to have.
3. Fair enough, mummy shouldn't be too opinionated here
4. Awesome, I'll look into sending a PR for that soon.

Feel free to close this issue as I think #78 addresses most of my concerns (unless there's something else you think can be added).

[ajusa]

I've updated my code over at https://github.com/ajusa/250 and deployed it to https://www.arhamjain.com/250, just as an FYI. Seems to be running behind Apache without any issues so far, I just had to add a content type of HTML to get everything working on an actual web server rather than my machine.

Thought you'd like to hear about someone else actually deploying an app built on Mummy :)

[ThomasTJdev]

I have added some of the ergonomics in my previous issue with focus on the parameters - both named but also query params - https://github.com/guzba/mummy/issues/98 (code: https://github.com/ThomasTJdev/mummy/issues/2).

For the response and redirect I have something like:

template resp(resp: string) =
  request.respond(200, @[("Content-Type", $ContentType.Html)], resp)

template resp(httpStatus: HttpCode, contentType: ContentType, resp: string) =
  request.respond(httpStatus.ord, @[("Content-Type", $contentType)], resp)

template resp(resp: JsonNode) =
  request.respond(200, @[("Content-Type", $ContentType.Json)], $resp)

template resp(httpStatus: HttpCode, resp: JsonNode) =
  request.respond(httpStatus.ord, @[("Content-Type", $ContentType.Json)], $resp)

template redirect(httpStatus: HttpCode, path: string) =
  request.respond(httpStatus.ord, @[("Location", path)])

[ThomasTJdev]

I have updated my personal package with utilities. For reference for others: https://github.com/ThomasTJdev/mummy_utils

[guzba]

I redid url parsing in webby and have decided to go further with mummy/routers features in this work-in-progress PR: https://github.com/guzba/mummy/pull/111

Relevant to this issue since it would make query parameters much easier to access as suggested.

[ajusa]

This is really great to see, thank you guzba! Two questions:

1. Will there be any work to be able to extract a list of query parameters out? I see that the underlying implementation is just seq[(string, string)], so do I just want to loop over the keys/values and add them to my own seq if I have a key with multiple values? Maybe this is more of a webby issue than a mummy issue.
2. Will you also be adding a similar field to parse a form URL encoded body? This one is a bit more opinionated (as the body is really just a string), it's just convention that we format it a certain way based on the browser implementations of form encoding and multipart encoding.

[guzba]

Will there be any work to be able to extract a list of query parameters out? I see that the underlying implementation is just seq[(string, string)], so do I just want to loop over the keys/values and add them to my own seq if I have a key with multiple values? Maybe this is more of a webby issue than a mummy issue.

The API support for the QueryParams type can certainly be improved but yeah you can iterate the (key, value) pairs and do whatever you'd like. The choice of seq internally was done to enable multiple values for the same key, though the easy [] accessor will not be what you want in that case.

Will you also be adding a similar field to parse a form URL encoded body?

I agree that it is a bit less clear how to improve things for the request body. The body can be form-encoded, could be JSON, could be mulitpart etc and these all have different implications. I do think the body being a string is a good baseline not a height of convenience haha.

Even if it is not an ideal name since it was intended for parsing the URL query string, parseSearch in webby will decode a form-encoded body. Just a note in case I've not showed that anywhere. let parsedBody = parseSearch(request.body)

[guzba]

Revisiting the initial 4 topics that were discussed as of today:

1. Redirects -> nothing to report here
2. Cookies -> still need to be added to webby
3. URL / URI handling -> ended up changing my mind here and went ahead and added a parsed and decoded .path and .queryParams to Request in release 0.4.0
4. getOrDefault was added ina PR a while back