Separate out semantics v. HTML/HTTP-specific bits of forms

[japhb]

The semantic bits of the forms implementation are valuable separately from the HTML/HTTP-specific bits. For example, it would be useful to be able to:

• Validate that stored data still remains valid according to updated display form rules (or from a different point of view, that updates to the display form definitions do not make existing stored data invalid)
• Validate that stored data has not become corrupt, or that proposed database changes will not make it so
• Validate the same data sent via JSON or YAML instead of via HTTP form submission
• Allow rendering the form and parsing the result using an alternate renderer or parser:
  • an HTML renderer that supports i18n and l10n
  • a JSON or YAML description passed to another microservice
  • a GTK or TUI interface

Please break the existing monolithic form implementation into semantic versus HTML/HTTP-specific layers to support the above.

[jnthn]

The semantic bits of the forms implementation are valuable separately from the HTML/HTTP-specific bits.

I'm not sure what's left if you remove everything in some way web-connected. I mean:

• The range of traits for configuring form fields control types are drawn directly from HTML 5 forms
• The validation model follows the HTML 5 validation model pretty closely

Taking some specific points:

Validate that stored data still remains valid according to updated display form rules (or from a different point of view, that updates to the display form definitions do not make existing stored data invalid)

You can make an instance of the form with .new like any other object, and then call .is-valid. So I think this one is already handled?

Validate that stored data has not become corrupt, or that proposed database changes will not make it so

I think the same answer as above applies.

Validate the same data sent via JSON or YAML instead of via HTTP form submission

We're not even doing HTML forms very completely yet: there needs to be a parse overload for multipart/form-data, but nobody got around to it yet. It's entirely reasonable to have a parse method that takes a Hash too;, that way, it should Just Work with both JSON (automatically) and YAML (if you add a body parser for that to the router).

Allow rendering the form and parsing the result using an alternate renderer or parser

This is technically possible today, although we should probably define a rather more proper DOM rather than the current array of hashes thing, and document it.

an HTML renderer that supports i18n and l10n

The aim is for Cro::WebApp to provide support for this; I already thought it through during the API design, but there wasn't time to implement it in the first release (admittedly, the things the initial release can do are, for better or worse, centered around what we needed for a customer delivery).

a JSON or YAML description passed to another microservice

Hm, that's an interesting scenario; got any more concrete examples?

a GTK or TUI interface

As noted at the start, it's not immediately clear to me what could be extracted for a context this different. I mean, I think taking the design idea/approach and applying it in those contexts could end well, but I don't know what code re-use could sensibly happen.

[japhb]

• The range of traits for configuring form fields control types are drawn directly from HTML 5 forms
• The validation model follows the HTML 5 validation model pretty closely

That may be true now, but it's not clear it needs to be limited by design. We might want to later add additional layers of compound or custom controls and validation, e.g. allow a single attribute of the top-level form to have a type indicating it is a group of input controls (presumably represented by its own class), or even a list of same.

A use case for this off the top of my head might be a financial app that needs a common concept of a "beneficiary" to be used for anything that might be paid out or transferred upon death, such as insurance, retirement plan, trust, etc.

Top-level forms that need to include a list of beneficiaries and payout percentages don't really care if each entry is rendered individually as a div containing a select from a dropdown of known persons, plus a text entry for a percentage; or alternately a single big table with columns for name, birthdate, tax-id, payout-percent. The top-level form just wants to say has Beneficiary @.ben; and be done with it, and leave the details of how Positional fields are laid out (and the UX of how to add/remove items) to the form renderer, and the details of what fields are needed within each item to the Beneficiary (sub-)form class.

Validate that stored data still remains valid according to updated display form rules (or from a different point of view, that updates to the display form definitions do not make existing stored data invalid)

You can make an instance of the form with .new like any other object, and then call .is-valid. So I think this one is already handled?

I thought the docs said this could fall afoul of is required being added to a field, or a new required field being added?

Validate the same data sent via JSON or YAML instead of via HTTP form submission

We're not even doing HTML forms very completely yet: there needs to be a parse overload for multipart/form-data, but nobody got around to it yet. It's entirely reasonable to have a parse method that takes a Hash too;, that way, it should Just Work with both JSON (automatically) and YAML (if you add a body parser for that to the router).

OK, that makes sense to me.

Allow rendering the form and parsing the result using an alternate renderer or parser

This is technically possible today, although we should probably define a rather more proper DOM rather than the current array of hashes thing, and document it.

This is part of what I meant about breaking the layers apart -- having a defined, documented interface I could use to add my own rendering or parsing interface. I might have gone XY on the actual problem though, because for me it looked like there was enough promiscuous knowledge that there had to be a disentanglement pass first -- though I freely admit that could have been just a mistaken first impression.

an HTML renderer that supports i18n and l10n

The aim is for Cro::WebApp to provide support for this; I already thought it through during the API design,

YAY! Are you the translation files going to be compatible with some existing toolkit, or are we going for "level up the industry" here too? :-)

but there wasn't time to implement it in the first release (admittedly, the things the initial release can do are, for better or worse, centered around what we needed for a customer delivery).

Yeah, I can understand that completely.

a JSON or YAML description passed to another microservice

Hm, that's an interesting scenario; got any more concrete examples?

This is more looking for a general mechanism/hook than something I had a particular list of ideas for, but here are a few:

• Active fraud/abuse detection systems looking for patterns in form use or errors, which may want to respond with or otherwise initiate an anti-abuse response (including telling the web app to start treating the user differently)
• Passive audit trail recorders that track every user interaction with sensitive data (PII/medical/financial/etc.)
• Integration/acceptance test systems wanting to test semantics rather than scrape the served HTML
• A proxy for serving a consistently "new look" redesigned interface when not all of the old interface has been converted over yet, so that the proxy is wrapping the old interface UX into the new interface look

a GTK or TUI interface

As noted at the start, it's not immediately clear to me what could be extracted for a context this different. I mean, I think taking the design idea/approach and applying it in those contexts could end well, but I don't know what code re-use could sensibly happen.

I was thinking recently about how to build a method for configuring my client/server games that would work no matter the client-side UI chosen, and I realized I needed to create some sort of standard schema for defining the things a player would need to choose -- a selection for difficulty level, a bounded Int for max number of players joined into a single game, a list of friends to invite, a short Str for game instance name, an invite message, etc. These all make semantic sense whether the UI type is CLI, TUI, GTK, WebSPA, WebCanvas, etc.

And then I saw Cro::WebApp::Form and went "THAT. I need exactly that. Now how do I separate the semantics, including attribute traits, validation model, etc. from the HTML renderer without massive copy/paste?"