Feature request: form input validation

[stephenbenedict]

Form input validation helper functions and examples would be very useful to have.

[pkulchenko]

@stephenbenedict, how would the API for that look like? fullmoon already provides a way to apply a pattern or a regex to the request parameters, so something like this should work:

fm.setRoute({"/myroute/:value", value = {regex = "^(\d\d|\w\w\w)$", otherwise = 400}},
  function() something using `value` end)

In this case the handler will only be called if value has two digits (\d\d) or three word characters and return 400 otherwise. Is that what you're looking for? You can get arbitrary complex with the checks applied and the result returned (for example, you can add a message to the returned status). Is that what you're looking for?

[stephenbenedict]

Thanks! What I had in mind is something like what Lapis does:

https://leafo.net/lapis/reference/input_validation.html#example-validation

Using regular expressions of course would work, but to make it easier and clearer, it would be nice to have distinct functions (exists, is_integer, min_length, etc.) for validation.

That, and a way to capture any resulting error messages to render into a real view/template (not just a message with a 400 status). Having not used Lapis, I’m not sure how it accomplishes this. The errors would need to be part of a table, I think, which is automatically rendered inside the form HTML.

With IHP (Haskell), after performing validation, there is a left/right expression to handle error/success, like so: https://ihp.digitallyinduced.com/Guide/validation.html#adding-validation-logic

In the error case, you render the "New" view with errors from the Create action.

For reference, I have separated my routes from my controller actions like:

-- app.lua

local PostsController = require "PostsController"

fm.setRoute("/posts/new",
    function() return PostsController.newAction() end)

fm.setRoute(fm.POST"/posts/create",
    function(r) return PostsController.createAction(r) end)

-- PostsController.lua

function PostsController.newAction()
    return fm.serveContent("posts/new")
end

function PostsController.createAction(r)
    -- Dummy code like what is used in Lapis
    validate.assert_valid(r.params, {
        { "title", exists = true },
        { "body", exists = true }
    })

    -- If above succeeds, create post and redirect to /posts, otherwise render new post view with errors
end

[pkulchenko]

Yes, I'm familiar with how it's done in Lapis, I'm just not sure how you want the API to look like.

Maybe something like this:

local errors = fm.validate(r.params, {
        { "title", exists = true },
        { "body", exists = true }
    })
if errors then return fm.serveContent("form", {errors = errors}) end

and you can then refer in the form to individual errors by using errors.title or errors.body. Would something like this work?

The main advantage would be the ability to report all errors together; it may still be possible to throw the very first error with some 400 message, as it may be sufficient in many cases when a partial html payload is returned (for example, when something htmx is used).

[pkulchenko]

BTW, if your response content doesn't depend on the request, then you can simplify the post/new route as the following:

fm.setRoute("/posts/new", fm.serveContent("posts/new"))

I understand that there may be something else to do in the handler, but if not, the above should work.

In fact, the handler can be replaced by other functions, like fm.setRoute("/posts/new", fm.serveAsset) (assuming there is an asset at that URL) or fm.setRoute("/posts/new", fm.serveAsset("/another/asset"). serveResponse, serveRedirect and other serve* methods should work as well (again, assuming they don't depends on anything in the request itself).

[stephenbenedict]

Maybe something like this:

local errors = fm.validate(r.params, {
        { "title", exists = true },
        { "body", exists = true }
    })
if errors then return fm.serveContent("form", {errors = errors}) end

I actually did not have a clear idea of the kind of API I wanted, but what you suggest looks great. I was actually confused by the Lapis validation example since it is not apparent where you handle the response to the validation function. Yours is much clearer.

[stephenbenedict]

BTW, if your response content doesn't depend on the request, then you can simplify the post/new route as the following:

fm.setRoute("/posts/new", fm.serveContent("posts/new"))

Great to know! Thanks.

[pkulchenko]

@stephenbenedict, I've implemented the changes and pushed to https://github.com/pkulchenko/fullmoon/tree/validation branch. Here is the spec/documentation:

local function serveSigninErr(error)
  return fm.serveContent("signin", {error = error})
end
local validator = fm.makeValidator{
  {"name", minlen = 5, maxlen = 64, msg = "Invalid %s format"},
  {"password", minlen = 5, maxlen = 128, msg = "Invalid %s format"},
  otherwise = serveSigninErr,
}
-- r is a special request parameter to be passed to the validator
fm.setRoute(fm.POST{"/u/signin", r = validator}, function(r)
    -- something useful with name and password
    return fm.serveRedirect(fm.makePath("index"), 303)
  end)

-- another option is to call the returned validator directly instead of using it as a filter:
fm.setRoute(fm.POST{"/u/signup"}, function(r)
    -- something useful with name and password
    -- validator returns `true` or `nil,error`
    assert(validator(fm.getRequest))
    return fm.serveRedirect(fm.makePath("index"), 303)
  end)

Supported validator checks:

• minlen = integer
• maxlen = integer
• test = function returning true or nil,error
• oneof = value | { table of values to be compared against }
• pattern = "lua pattern"

I didn't add exists, as it can be checked with minlen=1 and the fields are required by default, so I didn't see a reason for it. Also, oneof=value can be used instead of equal.

Supported rule options:

• opt = true: makes a parameter options; it's required by default; rules are still applied if parameter is not nil.
• msg = "message for parameter %s": adds a customer message for this parameter; will overwrite messages from individual checks

Supported validator options:

• key = true: return errors as keys by parameter names
• all = true: return all errors instead of the first one
• otherwise = function: allows to specify error handling when validation fails; will receive error(s) as the first value and the actual value that the validator was called for as the second parameter (the request object in this case); it's not used if the validator is called directly.

Using either of this options changes returning a string with the error message to returning a table with one or more error messages.

One thing I don't like is that it does accept a request object instead of an arbitrary table with parameter values. I can change that, but using a request object is convenient, even though it's now available through a method too. I'd have to add a special filter option to retrieve the list of parameters.

Actually, thinking about it, I'll probably change the logic a bit to use _ = validator to pass an empty value, so I can either use r.params or a table, which will make it a bit more convenient to be applied against arbitrary tables, but I'll wait for the rest of your feedback before making this change.

Let me know how this works for you.

[stephenbenedict]

@pkulchenko Thank you very much! Questions and comments:

1. In this example:

-- r is a special request parameter to be passed to the validator
fm.setRoute(fm.POST{"/u/signin", r = validator}, function(r)
    -- something useful with name and password
    return fm.serveRedirect(fm.makePath("index"), 303)
  end)

What is happening with the r = validator part? Is the validator function being called with r as one of its arguments? Sorry, I am a novice with Lua so my confusion is probably due to my lack of knowledge.

2. The supported validator checks look great and sufficient for my use.

3. I agree that it would be more intuitive to pass in r.params rather than the request object. Most of the time, the request table is used in routes, so having to use a different request object incurs some cognitive overhead and looks confusing. Though, how is the request object typically used vs. the request table?

4. Could you provide examples of how the validator options key = true and all = true would be used?

5. With the new validation functions, the most intuitive way for me to use them would be like:

fm.setRoute(fm.POST{"/u/signup"}, function(r)
  local validator = fm.makeValidator{
    {"name", minlen = 5, maxlen = 64, msg = "Invalid %s format"},
    {"password", minlen = 5, maxlen = 128, msg = "Invalid %s format"},
    all = true,
  }
  local valid = validator(fm.getRequest) -- or ideally `validator(r.params)`
  if (valid) then
    return fm.serveRedirect(fm.makePath("index"), 303)
  else
    return fm.serveContent("signin", {error = valid.error})
  end
end)

Is this possible? As a novice with Lua, I'm not sure if the above is even correct Lua

[pkulchenko]

What is happening with the r = validator part? Is the validator function being called with r as one of its arguments? Sorry, I am a novice with Lua so my confusion is probably due to my lack of knowledge.

Normally yes; there is nothing to apologize for, as it's Lua-based, but fullmoon-specific syntax to add filters to routes. In this case, you're adding a filter to take r and check it with the result of makeValidator call. I updated the syntax as was discussed earlier, so you can call it with _ = makeValidator(...) and the validator will get request.params table.

I agree that it would be more intuitive to pass in r.params rather than the request object. Most of the time, the request table is used in routes, so having to use a different request object incurs some cognitive overhead and looks confusing.

I agree and the most recent update already includes this change.

Though, how is the request object typically used vs. the request table?

It's the same thing; both request object and request table can be used interchangeably.

Could you provide examples of how the validator options key = true and all = true would be used?

Sure; there are makeValidator tests you can check for this usage. Basically, by default the result will be a string "Invalid name format" or something like that. If all=true is specified, then (1) all errors will be reported instead of the first one and (2) errors will be stored as a table (even if there is only one), so the result will be {"invalid name format"}, in case the user wants to display all form errors as a list. if key=true is specified, then the error is returned as a hash table, so in this case it will be {name = "Invalid name format"}, so it can be positioned next to the field in the form. I have an error field that looks like this in the template {%& errors and errors.name %} next to the name field. If both all and key is specified, then you get all errors all stored as key = value format where the name of the field is the key.

With the new validation functions, the most intuitive way for me to use them would be like:

yes, this is exactly how it's going to work with one small tweak:

fm.setRoute(fm.POST{"/u/signup"}, function(r)
  local validator = fm.makeValidator{
    {"name", minlen = 5, maxlen = 64, msg = "Invalid %s format"},
    {"password", minlen = 5, maxlen = 128, msg = "Invalid %s format"},
    all = true,
  }
  local valid, errors = validator(r.params) -- you can now pass r.params
  if (valid) then
    return fm.serveRedirect(fm.makePath("index"), 303)
  else
    return fm.serveContent("signin", {error = errors})
  end
end)

errors returned as the second value from the validator. Since you used all=true, you'll get all values, so error will be a table in the template and you'll have to iterate over it or use table.concat to build the value you need. This is why in my templates I use key and just access error by field name.

you can also move makeVlidator outside of the route handler (and reuse across multiple handlers), as there is nothing request specific in its configuration.

The advantage of doing this as a filter (like I showed earlier) is that you can use the otherwise value to send the error response and your route handler will only have the "happy" path, but both options are available to you.

[stephenbenedict]

Thank you very much for the explanations. I understand now how the route filter works as well as key = true and all = true. This looks perfect for what I need. I now need to add this validation to the app I’m building. I will let you know if I run into any issues.

[pkulchenko]

Thanks @stephenbenedict! Added documentation and pushed the changes.

[stephenbenedict]

@pkulchenko Fantastic documentation. Everything you need to know in the order you need to know it.

[pkulchenko]

Great to hear it looks useful; thank you for the feedback!