Question: pre-validation parsing

[ivan-kleshnin]

Hi, guys. I'm trying to replace custom ad-hoc validators in my project with functional applicative-based validation like you suggest. The piece I don't get is a form/json parsing.

Like there's a date as a string (say "2012") in the payload and I want to validate it, so I have to convert "2012" to new Date("2012") before validation is applied. Currently, I do that imperatively but, since I have declarative validation, I'd like to have the same level of abstraction for that parsing stuff.

Is there any standard tool or appoach to this in functional programming? Any clues or tips will be appreciated.

[robotlolita]

Eventually the validation module will be a higher-level library that will provide such standard tool, but while that doesn't happen you can build your own on top of the validation structure.

The way I've been doing it is to have a set of primitive validations (e.g.: isString, dateFormattedAs, ...), and a set of validation combinators (e.g.: and, or, arrayOf, ...).

The primitive validations will generally perform both validation (checking if the structure matches what you expect) and normalisation (transforming that value into something easier to work with). So you could have:

const isType = (type) => (value) =>
  typeof value === type ? Success(value) 
  : Failure([`Expected a value of type ${type}`])

const isString = isType('string');

const dateFormattedAs = (format) => (value) => {
  const date = moment(value, format);
  return date.isValid() ? Success(date.toDate()) 
  : Failure([`Expected a date with format ${format}`]);
}

const dateBetween = (start, end) => (date) =>
  date >= start && date <= end ? Success(date) 
  : Failure([`Expected a date between ${start} and ${end}`]);

Failures should be something that can be concatenated, arrays are the simplest form of that, but it does make combining error messages a bit more awkward.

Anyway:

dateFormatedAs("YYYY")("2012") 
// would return Success(new Date(2012))

Combinators make working with multiple validations a bit simpler:

const { chain } = require('folktale/validation');

// here each function is a (value) => Validation function.
const seq = (fn, ...fns) => (value) => fns.reduce((v, f) => chain(v, f), fn(value));

// And you can use the combinators to combine validations
const birthYear = seq(
  isString, 
  dateFormattedAs('YYYY'), 
  dateBetween(new Date(1990), new Date(2018))
);

With this all of your combined and primitive validations will be a (value) => Validation function, so you can use:

birthYear("2012").matchWith({
  Success: ({ value }) => `handle successful validation here (value is a Date object)`,
  Failure: ({ value }) => `handle errors here (value is an array of errors)`
});

Multiple validations can be done with collect:

const { collect } = require('folktale/validation');

collect(
  birthYear(user.birthYear),
  isString(user.username)
).matchWith({
  Success: ({ value: [birthYear, username] }) => 
    `${username} born ${birthYear.getYear()}`,
  Failure: ({ value }) => 
    `Invalid data: ${value.join(', ')}`
});

[ivan-kleshnin]

@robotlolita thank you for the explanation! Very informative and totally makes sense to me.

Let me ask a few more (and thanks for your time):

1) In case you just print all messages "above the form" Validation [message1, message2] works perfectly. In case you want to log messages below their corresponding input elements (or both) – you need to filter them. And input ids can be quite different from their visible labels. For example Password should contain numbers can correspond to the repeatPassword field so filtering by startsWith("Password") won't help.

I ended up writing my own Failure {id: [messages]} monoid and then Validation typeclass for learning purposes. It wasn't hard because you did such a great job explaining concepts in Folktale docs (big thanks for that – your docs are among the best I've seen!). Now I think I could also use something like:

Failure [{message, elementId}]

which is a bit less performant to use, but arguably easier to support and it keeps the order of original messages (may be necessary). I suspect you're using this version but it would be great to hear your opinion.

2) On the page: https://folktale.origamitower.com/api/v2.1.0/en/folktale.validation.html it's said that:

Result and Validation are pretty close structures. They both try to represent whether a particular thing has failed or succeeded, and even their vocabulary is very similar (Error vs. Failure, Ok vs. Success). The major difference is in some of their methods.

A Result is a data structure that implements the Monad interface (.chain). This makes Result a pretty good structure to model a sequence of computations that may fail, where a computation may only run if the previous computation succeeded. In this sense, a Result's .chain method is very similar to JavaScript's ; at the end of statements: the statement at the right of the semicolon only runs if the statement at the left did not throw an error.

I wonder what's a problem to add a chain method to the Validation and get this piece of functionality? I tried that and it works. I even think it's necessary because you sometimes want to avoid getting extra error messages:

["Password must have a length of 6", "Password must contain a special character"]
vs
["Password must have a length of 6"] then possibly ["Password must contain a special character"]

let isPasswordValid = (value) => {
  return collect([
    minLength("Password", 6, value),
    matches("Password", /\W/, value).failMap(_ => ["Password must contain a special character"]),
  ])
}

let isPasswordValid2 = (value) => {
  return minLength("Password", 6, value).chain(_ => {
    return matches("Password", /\W/, value) // if the prev. validation succeded
  })
}

3) Optional fields (validate if and only if it's filled). I made it like this:

// as said above, I reimplemented this stuff for learning purposes
let fromValue = R.curry((error, value) => {
  return value == null ? Failure([error]) : Success(value)
})

function optional(fn) {
  return R.pipe(
    fromValue("!"), // Success(value) | Failure(["!"])
    (v) => v.cata({ // this is from "fantasy-land/Daggy" equals your `matchWich`
      Success: (value) => fn(value),
      Failure: (value) => Success(null),
    })
  )
}

The usage is predictably:

validateFn(data)
optional(validateFn)(data)

Do you approach this differently?

4) Why it's called "applicative based validation" when it's all about monoids?

[robotlolita]

1) Our errors are usually a bit more structured:

{ 
  code: 'some-unique-id', 
  message: 'natural language description of the error',
  payload: { /* structured description of the error */ }
}

So if the error message is associated with an element it'll have code: 'field' and payload: { id: '...', error: <error> }

2) There's no .chain method in the Validation instances because the behaviour of .ap and .chain are incompatible. Any implementation would end up violating the Monad laws, and generic code would behave weirdly when receiving a Validation instance (more details here: https://folktale.origamitower.com/docs/support/faq/#why-is-there-no-chain-for-validation)

That said, there's now a chain function in the validation module that does that, it's just not a method (and thus not an implementation of Monad).

3) The only optional validations we do at work right now is "this field was not provided, so we won't validate and just use a default value instead". This is implemented as:

const optional = (validation, default_) => (value) =>
  value == null ? Success(default_) : validation(value);

optional(isString, "a")("foo"); // ==> Success("foo")
optional(isString, "a")(1); // ==> Failure(["Expected a string"])
optional(isString, "a")(null); // ==> Success("a")

Though that's not as generic and compositional as it could be. It's something I still have to think about how to solve in a better way.

4) The core of the Validation library is based on applicative functors (https://github.com/origamitower/folktale/blob/master/packages/base/source/validation/validation.js#L90-L108). The monoid instance is simpler, but less general, and could be written in terms of the applicative instance (but the converse isn't true).

You can do this with the applicative instance:

Success(a => b => ({ name: a, age: b }))
  .apply(Success("Alice"))
  .apply(Success(12));

But you wouldn't be able to cover this use case with the monoid instance because it loses information at each .apply (previous success values are discarded).

[ivan-kleshnin]

Got it, thank you! Closing this one...

Any plans to add something like: https://github.com/krisajenkins/remotedata/blob/4.5.0/src/RemoteData.elm to FolkTale?

[robotlolita]

It's not on the roadmap right now (see https://github.com/origamitower/folktale/blob/master/ROADMAP.md), but that does sound like a good idea. I'll keep it in mind in the future :>