Closed timneedham closed 9 months ago
edent
commented
7 years ago Thank you for this detailed proposal. I'm going to ask @psd to see how this fits in with the work we're doing on forms.
Regarding the two schemas you've mentioned (Mozilla's and Angular) - are there any compelling reasons to develop your own competing schema? We usually encourage people to work with existing bodies, rather than creating a new standard.
Both have open source licences, which is a good start for us.
timneedham
commented
7 years ago Hi @edent
Thanks for your quick response!
Yup, we used the Angular project in recent prototypes: and it's worked really well... we got the base capabilities up-and-running in no time.
However, there are many areas where we feel both these projects lack the capabilities of our existing DSL (and certainly some of the ideas we'd like to introduce in the next!)
Both projects are also tied to the strengths/weaknesses of a particular front-end technology; fracturing things from the outset?
We were hopeful to find a fuller/richer standard that is technology-independent too?
edent
commented
7 years ago Our first questions on standards like this is....
Those aren't meant to be aggressive - just need to understand where it makes sense to collaborate.
arkie-boy
commented
7 years ago I'm the editor of an ISO standard for the representation of the design of forms as metadata that is relevant to these requirements. Standards from our working group are freely available - although I think you can still pay for a copy if you like! - so it meets a practical definition of 'open'. The standard describes a model for registering forms, sections, questions and their answers together with the form's logic in a platform independent way that may be used - with suitable templating code - to generate web forms for data entry. Personally I have transformed form definitions into W3C XML Schema and then into XForms, but I don't see any reason why the same cannot be achieved for other form languages.
We're about to move from a technical specification to a full blown IS: I'd be delighted to talk about your application and (hopefully) ensure that the standard accounts for your use cases, I would be very grateful if we might use one of your forms as an example if there are no copyright issues. 19763-13-PDTS.pdf
Best place to start is reading appendix B
edent
commented
7 years ago Thanks @arkie-boy - could you give us a link to your working group?
The licence for West Midlands Fire Service appears to be MIT https://github.com/wmfs/tymly/blob/master/LICENSE
arkie-boy
commented
7 years ago I'll have to check how the MIT license works with ISO.
The working group is ISO/JTC1/SC32/WG2. We're best known for ISO11179 which is a standard for metadata registries (enterprise data dictionaries, basically).
https://www.iso.org/committee/45342.html
Steve
timneedham
commented
7 years ago To answer your initial questions @edent ...
Can you contribute to the existing standard(s)?
If your new DSL is useful to you, it will be useful to other people.
Of the standards that you've seen, do any have a good track record of accepting changes?
Do any of the existing standards support extensions? That is, you use their standard with your own, well-defined, addons.
Do you have the capability to run a standard better than the existing bodies?
Those aren't meant to be aggressive - just need to understand where it makes sense to collaborate.
psd
commented
7 years ago Hi Tim, that's very interesting and we're definitely 'fellow travellers'. I'm away on holidays for the rest of August, but I'll definitely take a look at the tymly code and see how it works. It would be great to see if we can work on a challenge together in the Autumn as we have a similar need for an open standard.
We (GDS) recently ran a discovery into how teams design, publish and process offline forms across government, some of our findings will be made public soon.
During the course of our research we spoke to a number of different teams who are mechanically generating online forms across government using a number of different technologies, including de facto formats such as Adobe XFA, and a number of DSLs in JSON and YAML, notably the Home Office Forms (HOF) and Digital Marketplace.
We're in the process of working out what ideas we could test in an alpha (should we go ahead), but something I definitely see as being useful is a format for describing a form which we can use to consistently model forms and possibly transpose to and from one of the many other formats used to describe existing forms.
Some of the initial needs we have for such a format include:
but there are many other needs which we imagine will emerge during an alpha, including some of those you enumerate.
Paul
timneedham
commented
7 years ago Hi all, thanks for all your input, sounds exciting!
Just a quick question: as above, we've a few other interrelated challenges waiting-in-the-wings:
| Challenge | Description |
|---|---|
| Data model | A discussion about how best to extend JSON Schema to include extra attributes (such as which properties are good for indexing and similar). |
| Workflow | We're finding a lot of benfit in using Finite State Machines to run workflow, but there doesn't seem to be a standard way to define machines/states/events in JSON? Full-fat BPMN, BPEL etc. seems overkill for our needs. |
edent
commented
7 years ago @timneedham Separate challenges are fine - but we will ask you to take on some of the discovery work.
jazz-b
commented
7 years ago Hi everyone, this is the workbook schema file mentioned above...
timneedham
commented
6 years ago Hi everyone,
A quick (and surprisingly emotional :flushed:) update...
In the end we delivered our MVP using React Schema Form as the basis of specifying UI content declaratively. It went OK, but it's not a great fit. We've had to extend much more than we expected and it's already starting to creak with pretty simple UI descriptions. That said, our MVP has really helped us discover what it is we really need...
We've recently started turning our requirements (for a language capable of describing UI content for use within a digital service) into a specification along with a supporting SDK. We've called it...
Maybe it's because we've been at this for ten years (and Viewscript was always hiding just under the surface, like an open standards truffle)... or perhaps we've some leftover endorphins from getting our MVP shipped... but we're really quite excited about all of this. It feels like a lot of good things can come from something like a Viewscript specification.
Viewscript links
Note that things are less than a month old at the moment and so very much incomplete!
Apologies if this is bad form...
Kind regards
Tim
timneedham
commented
6 years ago We're seriously considering using AdaptiveCards as the basis for our declarative front-end UI.
The Guiding Principals, especially 7 are quite heartening and suggest we can extend towards something akin to Viewscript:
The format needs to be extensible
Hosts should have the freedom of adding support for custom elements or custom actions that go beyond what the format is capable of This is particularly important for actions, as various hosts don't necessarily support the same set of actions These additions are at the discretion of the host They are not a de facto addition to the Adaptive Card specification As such, they make a payload that uses them incompatible with the mainstream Adaptive Card format They can however be presented to the Working Group and proposed as new features for a future version of the format
Does anyone have any thoughts/experience re. AdaptiveCards at all?
timwis
commented
5 years ago 👋 hi folks, this is a topic I've researched a bit before for a few projects as well. I found that the existing implementations out there weren't all that standardised, and often worked around JSON Schema, which makes sense for validation, but doesn't support things as essential as ordering the fields, conditionally displaying them, etc. which resulted in essentially duplicate/redundant configs (one for the "schema" and one for the "display"). Maybe there's no way around that, but I'd love to dive deeper to find out if there is. (Part of me does wonder whether at the end of the rabbit hole it's just HTML :O)
Anyway, just wanted to connect this thread to @dmethvin-gov from the US Digital Service, as they were working on usds/us-forms-system which also used the mozilla approach.
timneedham
commented
5 years ago Hi @timwis , thanks for this!
We'll take a look at the US Forms System.
Not sure if it's of interest, in the end we had a go at the stuff you've mentioned... conditional visibility, validation etc. Based on a heavily-extended AdaptiveCards spec.
We'd love to know your thoughts about Cardscript?
Playpen: https://wmfs.github.io/cardscript/
WIP docs https://wmfs.github.io/tymly-website/reference/#cardscript
(take a look at the complex and Kitchen sink examples?)
galund
commented
5 years ago It's worth mentioning that react-based systems aren't readily compatible with the service standard. The service manual has just been updated with regards progressive enhancement: https://www.gov.uk/service-manual/technology/using-progressive-enhancement
timneedham
commented
5 years ago Good point @galund! 😄
In our particular case, the Vue framework we're currently using with Cardscript (Quasar) supports Server Side Rendering, which may help allay some concerns?
This standard should obviously be technology-agnostic (i.e. it has absolutely no relation to any front-end technology such as React, Angular or any other)
Of course, any declarative [JSON] standard could also be used as the basis to generate flat HTML too... the separation of a declarative standard and the technology which may subsequently visualise it is a core principle of projects such as AdaptiveCards and Cardscript.
timwis
commented
5 years ago @timneedham Cardscript looks awesome! Really similar to what I had been toying with. I think I gave up whilst trying to safely evaluate user input for conditional visibility expressions which I see you've managed to wade through :P
dmethvin-gov
commented
5 years ago Thanks for the ping @timwis!
I can't get the CardScript site to load, I'm getting this error and a blank screen:
vendor.373f220d.js:6 Error: Loading chunk 149bcf13 failed.
(error: https://wmfs.github.io/cardscript/js/149bcf13.dc65cbf2.js)
at HTMLScriptElement.l (runtime.6446a868.js:1)After quite a bit of hands-on experience with forms, I think that the approach taken by React-JSONSchema-Form (RJSF) is not really a good one. Although JSON Schema is great for defining the resulting data of a form and the constraints on its data, it doesn't have the expressive power to define all the semantics that are needed in the front-end. Concepts like this:
RJSF tried to do this with additional rules "alongside" the JSON Schema but this tended to make the definition very redundant because many fields were mentioned in two (sometimes three or four!) places. Also, it is much too tied to React when it doesn't need to be, as already mentioned. So much of the definition of the form has nothing to do with whether it's in React, Angular, jQuery, or plain JavaScript.
There is prior art here that seems pretty close in concept, see https://github.com/formio/formio and its video at https://www.youtube.com/watch?v=d2gTYkPFhPI . It's not perfect but it does have a visual form builder and creates a "schema" for the form in JSON. From there you can run a builder that generates the resulting JSON Schema for the backend and components for a framework like React, Vue, or Angular.
Beyond that I'd love to see any form system be able to pull in different widget sets via some "adapter" mechanism. It's waayyy too heavy a lift to define all the different types of widgets that people need, and there are several good sets out there already for the popular frameworks. To me that would be the primary reason to use a framework and not just have the form system do its own state and form management in plain JS. I don't want to build yet another set of date inputs, date picker, date time picker, etc. This is especially painful in React where there isn't even agreement on how to manage CSS in an app.
I feel like our ideal here should be to avoid the need for experienced programmers for something as rote and repetitive as form creation. In the US federal government there are more than 25,000 paper forms. We can't expect to automate those in reasonable time if we require a programmer to write each one.
timwis
commented
5 years ago Well summarised @dmethvin-gov - I've run into the same issues! The cardscript playpen loaded fine for me - give it another shot? I think you'll find it more closely aligns with what you've described.
DidacFB-CDDO
commented
9 months ago As per housekeeping practices we are closing this suggestion down since there has been no comments since 2019.
The Forms Challenge
Category
Challenge Owner
West Midlands Fire Service
We're a small in-house team within West Midlands Fire Service that has over 12 years experience in developing dynamically-rendered input forms (i.e. a technique that separates the definition of what a form should collect from the code that controls how the user interface is delivered).
This separation-of-concerns has worked really well for us over the years and continues to bring unexpected benefits. We've now accumulated a library of around 100 form definitions which cover all aspects of Fire Service activity.
As we begin work on our next-generation platform we'd like to align to a standard that helps widen the uptake of this approach.
Short Description
This is a challenge to produce an open standard to express user-facing input forms.
The standard should cover several facets:
Along with our prototype DSL here are a couple of established reference-points to start:
Note!
User Need
As this is a more general challenge, it doesn't relate to one particular group of user.
That said, what is important from a user-perspective is that the specification be expressive enough for dynamic form renderers (be them running inside web pages or mobile apps) to deliver rich and efficient user experiences.
Expected Benefits
Assuming this standard could be used throughout government to define user-facing forms...
When quantifying the impact of introducing our initial dynamic-form platform, we internally estimated that it was saving 15,000 person-hours per year (as compared with the overheads incurred with traditional "discreet system" approaches and training). If such estimates still hold true today, then the cumulative impact of supporting a switch to dynamic-form-rendering across government would be significant.
Functional Needs
We consider the proposal should be: