Feature for embed UISchema into JSONSchema

[qcho]

Prerequisites

• [x] I have read the documentation;

I'm proposing this feature before doing any coding or PR to see what the community and maintainers think about the change. I can create a PR with the changes (seems not a big feature) if it's not against any Design principles.

Description

I want to propose a new feature where the UISchema is embedded within the JSONSchema.

Let's say we have these two definitions from Simple example

JSONSchema:

{
  "title": "A registration form",
  "description": "A simple form example.",
  "type": "object",
  "required": [
    "firstName",
    "lastName"
  ],
  "properties": {
    "firstName": {
      "type": "string",
      "title": "First name"
    },
    "lastName": {
      "type": "string",
      "title": "Last name"
    },
    "age": {
      "type": "integer",
      "title": "Age"
    },
    "bio": {
      "type": "string",
      "title": "Bio"
    },
    "password": {
      "type": "string",
      "title": "Password",
      "minLength": 3
    },
    "telephone": {
      "type": "string",
      "title": "Telephone",
      "minLength": 10
    }
  }
}

UISchema:

{
  "firstName": {
    "ui:autofocus": true,
    "ui:emptyValue": ""
  },
  "age": {
    "ui:widget": "updown",
    "ui:title": "Age of person",
    "ui:description": "(earthian year)"
  },
  "bio": {
    "ui:widget": "textarea"
  },
  "password": {
    "ui:widget": "password",
    "ui:help": "Hint: Make it strong!"
  },
  "date": {
    "ui:widget": "alt-datetime"
  },
  "telephone": {
    "ui:options": {
      "inputType": "tel"
    }
  }
}

Is there any reason why we can't just have one "extended" JSONSchema object (this is actually allowed by the JSONSchema spec) with the UI definitions instead of replicating the entire mapping?

The result would be JSONSchema + UISchema:

{
  "title": "A registration form",
  "description": "A simple form example.",
  "type": "object",
  "required": [
    "firstName",
    "lastName"
  ],
  "properties": {
    "firstName": {
      "type": "string",
      "title": "First name",
      "ui:autofocus": true,
      "ui:emptyValue": ""
    },
    "lastName": {
      "type": "string",
      "title": "Last name"
    },
    "age": {
      "type": "integer",
      "title": "Age",
      "ui:widget": "updown",
      "ui:title": "Age of person",
      "ui:description": "(earthian year)"
    },
    "bio": {
      "type": "string",
      "title": "Bio",
      "ui:widget": "textarea"
    },
    "password": {
      "type": "string",
      "title": "Password",
      "minLength": 3,
      "ui:widget": "password",
      "ui:help": "Hint: Make it strong!"
    },
    "telephone": {
      "type": "string",
      "title": "Telephone",
      "minLength": 10,
      "ui:options": {
        "inputType": "tel"
      }
    }
  }
}

Expected behavior

This proposed feature resembles how HTML work. We have CSS (like UiSchema) and we can also have some styling inline for some objects. The expected behavior is therefore easy to understand.

JSONSchema's ui:* properties override UiSchema ones the same way style=* attributes have precedence over css in html.

Actual behavior

This is not a breaking change. If you replace the JSONSchema from the sandbox with the JSONSchema+UiSchema one it will continue to work as expected (because the UiSchema is still in place)

If you remove the UiSchema all the ui: definitions are lost but we need to make the change so that it will work with inline UiSchema too.

[glasserc]

Personally, I am not crazy about the change. While I guess it is true that the JSON Schema spec allows arbitrary additional keys, to me the JSON Schema describes a data format, and UI considerations really belong elsewhere. I guess I would want to know why this feature is valuable to you and what you would use it for.

I know @n1k0 is mostly doing other things these days, but I'd be curious to hear his opinion.

[bkeating]

If it's not a breaking change, I'd say it would be great to support . there may be situations where a single payload makes better sense. Would personally like this more than two separate payloads, at least for the simple use cases.

[qcho]

@glasserc I'd like to use the JSONSchema + UISchema definition to expose/consume a custom-forms api to the outside world. Instead of sending {schema:{...}, ui:{...}} with lot's of shared mapping sending only one extended json makes sense.

Also having the "override" functionality adds some extra functionality when trying to get form definitions from elsewere having a default ui with the availability to overide it's default behaviour from server.

Also it seems some more complex json-schema features where fields get a bit messy with conditions, flow control, etc (specially over the last drafts) would greatly benefit from having inline "ui:" instead of trying to remap there conditionally.

[glasserc]

OK, I guess it would make sense to allow using a combined schema instead of/in addition to the current separated format.

[bkeating]

Definitely in addition to. Either use case should be supported imo.

[llamamoray]

I'm not too keen on this unless there was a configuration option when rendering the form that turns this feature off.

My use case for the library allows users to define data structures using JSON schema and then my code decides the best way to display this. It's quite important that there is this separation and allowing a merged schema would mean I'd loose control of what generated forms look like.

[bkeating]

Very insightful, @llamamoray. Thanks for keeping me grounded.

There are a lot of scenarios where either-or would be a good choice. I've been egging this change on because in my current situation, we own the whole thing internally and would enjoy having less objects to manage. I haven't felt like I've gained anything from the separation but again; for this current project of mine only. While a form is just a form, they continue to surprise me in how unique they often end up being. ❄️

Can we just reference the same formSchema object for the uiSchema prop? Filter for ui-relevant definitions only? While you end up repeating yourself, the explicitness might help ensure whats going on:

<Form 
  schema={myFormSchema} 
  uiSchema={myFormSchema}  
/>

and maybe as a safe-guard, provide an explicit way to prevent the formSchema from defining your uiSchema:

<Form 
  schema={myFormSchema} 
  uiSchema={myUiSchema}  
  disableInlineUiSchema
/>

This could compare the two objects and make sure they aren't the same(?)

Or maybe we just need to keep 'em separated.

[llamamoray]

I understand the issue with the UISchema having to replicate the structure of the data schema, but allowing the combing of them just adds needless complexity to what is currently a very easy to understand API.

At the moment we have a clear separation of concerns: the data structure you wish to collect, and any tweaks to how you wish to display your form.

Allowing one schema to override the other introducing a hierarchy like the style vs css example is exactly what causes headaches for code maintainers: in order to know there is a hierarchy you have to read the API rather than the interface being self explanatory.

It would be pretty easy to write a function that parses this combined schema and splits the schema out into a data and ui schema that you pass into the relevant props. If how you store and maintain your schemas is your only concern, I would personally go down this route.

[bkeating]

Great point. Separating out a single payload beforehand is likely much cleaner than supporting this internally.

Realizing now that I’d be happy with just that but obviously didn’t think of it myself. Could we entertain the idea of adding a section to docs to satisfy this? It’s not a direct responsibility nor suggested approach, but boy, having it as a small section in the docs would help prime creativity.

On Mon, Feb 12, 2018 at 7:29 AM Alex Taroghion notifications@github.com wrote:

I understand the issue with the UISchema having to replicate the structure of the data schema, but allowing the combing of them just adds needless complexity to what is currently a very easy to understand API.

At the moment we have a clear separation of concerns: the data structure you wish to collect, and any tweaks to how you wish to display your form.

Allowing one schema to override the other introducing a hierarchy like the style vs css example is exactly what causes headaches for code maintainers: in order to know there is a hierarchy you have to read the API rather than the interface being self explanatory.

It would be pretty easy to write a function that parses this combined schema and splits the schema out into a data and ui schema that you pass into the relevant props. If how you store and maintain your schemas is your only concern, I would personally go down this route.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/mozilla-services/react-jsonschema-form/issues/701#issuecomment-364922436, or mute the thread https://github.com/notifications/unsubscribe-auth/AAJo5H3BZRxdwikwNDEa1wmHWslV_skNks5tUDzEgaJpZM4Pcq_o .

[glasserc]

Thanks for talking this out, you two. I'd be happy to review/merge a PR that added to the documentation as long as it was reasonably generic (something like "here are some suggested ways to use rjsf" would be OK with me; something like "here's one very specific way to use rjsf with combined schema and uiSchema" would not be).

[handrews]

Hi folks- just a heads up that the vocabulary support planned for draft-08 (see json-schema-org/json-schema-spec#561) is specifically designed to facilitate this sort of combination.

We are also splitting the subschema keywords (e.g. properties, items, allOf, etc.) out of validation and into their own vocabulary in order to encourage non-validation vocabularies to use them (from the perspective of writing a validation schema, there is no change for this- it will be backwards-compatible).

[jorge2013]

Hi, I do think that keeping it simple is the way to go, but maybe if the uiSchema information inside the JSONSchema is thought as a default value it makes sense. In fact angular-schema-form does exactly that: https://github.com/json-schema-form/angular-schema-form/blob/development/docs/index.md#form-defaults-in-schema and personally I think it is a nice feature to have. In my case I generate the JSONSchema on the fly based on the fields definition that I handle in my system, and the Form information is constant. I'm thinking into migrate from angular-schema-form to this library, and for sure I could use it as it is and generate both JSON on the fly, but I still think that been able to specify the default form information inside the schema is a nice feature, but instead of the proposed idea of just adding multiple fields with the ui prefix, I think it is cleaner just adding a new field like: ui-schema-form: { .... } again, the idea comes from angular-schema-form not me :)

[brettz9]

I love the inline approach exactly as allowed in the proposal. I have yet to see any code--including JavaScript--which is fully concern-separated, and if there is concern separation, it can make things unnecessarily cumbersome. Does everyone write, or want to write, code like this in all situations? Be honest...

function domBridge (dataset) {
    location.href = dataset.url;
}

$('#myButton').addEventListener('click', function () {
    domBridge(this.parentNode.dataset);
});

or don't most of us live by the fact that we don't always want to be troubled to do this (or distract ourselves and others by doing this) and just do this?

$('#myButton').addEventListener('click', function () {
    location.href = this.parentNode.dataset.url;
});

Libraries like jQuery are so popular in large part because they even facilitate a (concise) mixing of concerns (attach a listener to this element and build some HTML on another element with a listener, etc.).

Separation of concerns can be valuable (and failure to separate a headache) but it depends on what is of "concern" to your project. Otherwise, separation actually adds overhead: whether readability, searchability, performance, and often even intelligibility.

Yes, one can envisage situations where the UI elements become so overwhelming as to drown out the data structure (and vice versa). But it also can help some projects to quickly see intent and rapidly build their applications based on the ability to quickly interpret the source in a single place.

I constantly find the most successful and readable solution for another area relating to separation of concerns, HTML templates with JavaScript, is to start with model-view separation but allow myself to add behavioral listeners inline within my markup (I use my own library Jamilih to express HTML as JavaScript/JSON, mixed in with inline listeners) and only as it becomes cumbersome, move such behaviors out of the view. This is not out of laziness but out of experience with how it conduces to maintainable and readable code. I see it like organ development. The fetus begins undifferentiated and only progressively develops organs out of that mass.

I hope we can be tolerant of both approaches, and recognize that good projects may even want to use both approaches simultaneously in some cases or progressively wean themselves from one to the other.

[brettz9]

Oh, and part of the reason I think inline JavaScript (or CSS) is so anathema is because HTML syntax highlighters have not typically provided inline syntax highlighting so finding a bunch of spaghettei code without such highlighting, typically not broken up with newlines, can be very unpleasant to deal with. Mix in with that the syntax mismatch, with inline code needing to escape quotes (another reason I prefer using JavaScript exclusively for HTML as well), there is reason it became so widely rejected. None of these relate to separation of concerns, but even while that is part of it, we have to admit that once we see a valid concept like "separation of concerns", the principled/disciplined among us tend to become code police in enforcing it, often without seeing enough nuance about it.

[epicfaace]

Revisiting this ... it seems like one big advantage of combining the uiSchema and schema into a single object would be that the class names / widgets / etc. can be conditionally modified based on dependencies / anyOf / oneOf. Right now there is no way to do that unless one makes custom widgets. See #1236 and #1206.

This point wasn't brought up in the earlier discussion in this issue, so I'd love to hear your thoughts.

[handrews]

@epicfaace this sort of thing is exactly why we are making it easier to extend schemas with recognizable keywords in the next (yes, I know, long-delayed but it is getting close) draft of JSON Schema. We have also formalized how different sorts of keywords interact.

I suspect most if not all of your uiSchema keywords would be considered annotations- data that is attached to a JSON instance if it passes validation. That would be the mechanism for leveraging dependencies/anyOf/if etc. (although we also split up dependencies into two keywords because its two variations do different things).

In the next draft, there is a formal recommendation for how to collect such annotation information, which will hopefully make it easier to build features such as form generation out of JSON Schema.

[LucianBuzzo]

@handrews That sounds really interesting, Is there a place we can see an in progress version of this draft?

[qcho]

@epicfaace I think the point was always there. I agree one of the best advantages of allowing this kind of inline metadata is to avoid complex (unneeded) mapping between the two specs.

Quoting myself earlier:

Also it seems some more complex json-schema features where fields get a bit messy with conditions, flow control, etc (specially over the last drafts) would greatly benefit from having inline "ui:" instead of trying to remap there conditionally.

conditions, flow control, etc (specially over the last drafts) means things like That would be the mechanism for leveraging dependencies/anyOf/if etc.

Also i think the spec as proposed in the first comment is simple enough for anyone to understand pragmatically.

[handrews]

@LucianBuzzo it's what's checked into master at the json-schema-spec project (you can build an HTML version if you install the xml2rfc python package (see the .travis.yml file for exact version to install).

In the next couple of weeks we should post a rendered version for final review. We're still working through the last few issues right now.

[felixfbecker]

Perhaps the Form component could just accept a prop like resolveUiSchema that gets passed the schema of a field (with resolved anyOf etc) and can return the uiSchema to be used for that field? I am not particularly interested in inlining entire uiSchema into JSON schema, but I'd really like to modify the UI based on some hints encoded in the JSON schema, e.g. something like a custom "multiline": true property or "format": "search" for a custom search syntax I want to render an autocomplete widget for.

[arjan]

I have implemented a basic method to add a $ref syntax within the UI schema to resolve references.

The idea is that you preprocess the UI schema before passing it in to the Form component:

<Form uiSchema={resolveUiSchemaReferences(uiSchema)} ...>

[FunkMonkey]

@epicfaace

I think @felixfbeckers proposal is a little more flexible / elegant, especially when conditionally changing the UISchema. Any chance we could convince you to try this approach in #1638?

Your idea to store it exactly in the JSONSchema could easily be extracted into a helper function like this:

function resolveUiSchemaFromJSONSchema(schemaForField: JSONSchema7, pathToField: string[]){
  return schemaForField['ui'];
}

<Form schema={someSchema} resolveUiSchema={resolveUiSchemaFromJSONSchema} />

What do you think?

[rkorrelboom]

Just getting started with this library but I'd also love this feature. Until this is supported I think wrapping components that accept a schema and uiSchema with a HOC that extracts ui: fields from the schema should work (or am I missing something?):

export function useUiSchemaFromSchema(schema, uiSchema) {
  return useMemo(() => {
    const extracted = {};
    for (const key in schema) {
      if (key.startsWith('ui:')) {
        extracted[key] = schema[key];
      }
    }
    return { ...extracted, ...uiSchema };
  }, [schema, uiSchema]);
}

export const withUiSchemaFromSchema = (WrappedComponent) => (props) => {
  const uiSchema = useUiSchemaFromSchema(props.schema, props.uiSchema);
  return <WrappedComponent {...props} uiSchema={uiSchema} />;
};

const fields = {
  StringField: withUiSchemaFromSchema(StringField),
  // etc...
}

[epicfaace]

think @felixfbeckers proposal is a little more flexible / elegant, especially when conditionally changing the UISchema. Any chance we could convince you to try this approach in #1638?

@FunkMonkey Sure, I think that would also be work, would be glad to review that approach in a PR.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Please leave a comment if this is still an issue for you. Thank you.

[adjenks]

Hey @stale bot can you re-open this?...

[adjenks]

I was already mixing the two schemas together for a while. I would just pass jsonSchema.properties as the uiSchema, but this only works for one layer because if you want to customize anything within a second level object you need to put it under the properties key of jsonSchema, but uiSchema does not expect that. If we just made the UI schema require the properties key for objects it would work.

Since the uiSchema for arrays already requires that you use the "items" key to define uiSchema information, it would actually make it more consistent if objects required the "properties" key as well, instead of just writing the properties directly under the parent.

Just because they would follow the same structure and would be mergeable, wouldn't mean you would have to merge them. You could still keep them separate if required for your application.

I vote to at least mirror the structure of the schemas so that they can be combined if desired. There might be other keys I'm not thinking of, but I wouldn't mind having a slightly deeper structure to mirror the jsonSchema.

[jasongill]

I also would like to see this added if possible; since JSON Schema now allows for this type of custom addition, it just makes things cleaner than having to generate and parse out two schema (especially when the schema is extremely complex and has many nested levels).

[modellking]

We have a use case where data may be given from the UI or taken from an API. we support multiple formats for the API, but the UI-Form shoud only use one Format. This schema is programmaticly constructed in the specific View, so we either have to offer twice the utility or own an additional abstraction layer to split the unified Schema into two.

Our Workaround; We use this very simple implementation: https://gist.github.com/modellking/d44060795fcbc24a0cf1eefb824b9863

[emmalcg]

I was also looking to add custom annotations to json schema and just came across this, and that json schema supports adding any custom annotations and just prefixing them with x https://json-schema.org/blog/posts/custom-annotations-will-continue#how-did-we-arrive-at-as-the-prefix-of-choice

[handrews]

@emmalcg that's a proposal for an unpublished future version. Draft 2020-12 recommends treating any unrecognized keyword as an annotation, no x- required.

[emmalcg]

@handrews oh really that must explain why it is not documented! I was wondering about this advice at the end of the post then? should it not be followed?

Moving forward, prefix your custom annotation keywords with x-.

Another nice thing about this solution is that you don't have to wait for the next version of JSON Schema to come out. You can start updating your schemas today. x- keywords are compatible with all versions of JSON Schema that are currently published; they'll still just be collected as annotations. And when the next version comes out, you'll already have migrated!

[handrews]

I haven't been directly involved with the JSON Schema org for a while so I can't speak to what they intend (AFAICT, at the moment they're debating changing direction. But since current implementations already treat unrecognized keywords (x--prefixed or not) as annotations, you can use x- prefixes if you want. I find the prefixes unsightly, but that's just a stylistic preference.

[Yashsharma1911]

Is this proposed change implemented? it would be nice to have

[PSU3D0]

This would greatly simplify server-side schema form generation with libraries like Pydantic.