Need a human friendly multi-line "description" in json-schemas

[darshakthakore]

I'm opening this one here to continue the discussion in #226 from the old repo. Basically the "request" is that there should be some mechanism that somewhat acts as jsondoc for elements being defined in json-schema, mainly useful in cases where enums and tuple arrays are being defined. The current "description" property is a string only, can it be made into either a string or a string array so that it can be multi-line. This would help in cases where the json-schemas are being used for modeling and are consumed equally by humans and machines.

[awwright]

JSON usually represents newlines with the \n sequence, I'm not sure how wise it would be to add a new construct for the sole purpose of overcoming this limitation in the format.

[handrews]

I'll throw in again in favor of humans writing schemas in YAML and producing JSON as a build step to be consumed by machines (most YAML/JSON parsers have some sort of hook you can use to preserve ordering if you want the JSON to look like the YAML for human readability).

[Relequestual]

What do you want beyond using \n for newline? Would you want the fact that \n is available for new line to be part of the specification?

[erosb]

I would also advice using YAML instead of complicating the schema spec.

[Relequestual]

@erosb right. sure. I'm not familiar with the specifics of converting YAML to JSON. Are you saying you can create YAML, you can do so with line breaks, and it will convert to single strings with \n ?

[handrews]

@Relequestual YAML has formats for both "folded" block text (where it removes all newlines, so you can write long strings wrapped to be easier for humans to read in source) or "literal" block text (where you can wrap it and it preserves newlines). Both forms strip off uniform indentation and have various chomping options for terminal newlines, trailing blank lines, etc.

So the folded form would just be a long string in JSON, while the literal form would be a long string with embedded \n newline characters.

[erosb]

I'm not familiar with the specifics of converting YAML to JSON

You can think about YAML as a different "dialect" of JSON. They differ only in lexical elements, but they represent the same structure, so there is a 1:1 mapping between YAML and JSON.

[handrews]

there is a 1:1 mapping between YAML and JSON

Not quite- YAML can be used to instantiate specific types or classes when it is loaded, while JSON has no syntax to do so. An obvious difference is that YAML can specify an ordered map with !!omap or an unordered set with !!set, neither of which can be done in JSON.

[awwright]

As long as it can represent the full object model defined in JSON Schema, and people don't use more than that, I don't see a problem.

Using a more generic Javascript syntax (allowing unquoted keys and trailing commas) is also common.

[handrews]

As long as it can represent the full object model defined in JSON Schema, and people don't use more than that, I don't see a problem.

I see it as totally outside the scope of JSON Schema. To take an extreme example, if you've got a prose format that you can somehow read in and spit out valid JSON Schema in JSON, the JSON Schema specification doesn't care one way or the other. If a specific implementation wants to directly read (YAML, JavaScript, mythical prose format) and support it without explicitly producing JSON first, that's an implementation detail and not addressed by the spec one way or the other.

The only thing the spec requires is that if you want something to be recognized in an implementation-independent way as JSON Schema in a hypermedia system, it must be served as application/schema+json, so documents in those situations MUST be JSON.

[Relequestual]

(@erosb I'm semi familiar with YAML (I really like RAML), but I haven't used it much.)

I mostly agree with @handrews on this. http://json.org defines \n is new line character in string. Therefore the description field already supports multi line content because it is json.

If you're wanting to view the json in json format with line breaks, then no, json doesn't support that, and you'll need to use a text editor that can wrap text. I see this as a non issue and should be closed.

[handrews]

I see this as a non issue and should be closed.

Agreed, JSON's shortcomings are an issue to be raised with the JSON people (good luck), and are not problems we can fix here. At most we could add a note that other forms such as YAML may be easier for humans to use for working with schemas, but are outside of the formal standard.

[handrews]

@Relequestual , @awwright , @darshakthakore , it looks like this should be closed as out of scope for JSON Schema.

[awwright]

It's only 8 days old, let's wait for @darshakthakore to give an opinion

[handrews]

It's only 8 days old, let's wait for @darshakthakore to give an opinion

er... good point. Sorry, it's been a long week and I didn't realize it was that recent somehow.

[darshakthakore]

Oh wow, I wasn't expecting such a quick discussion on this and thanks @awwright and @handrews for the opportunity to respond back. First of all, let me state that I agree that we shouldn't be trying to fix JSON's shortcomings in JSON Schema. Having said that, I wanted to at least show the use case that we are trying to address that relates to my request. We are using JSON Schema for specifying data models for IoT scenarios. Here's one specific example. To elaborate a bit, an organization called OCF is heavily using JSON Schemas to publish IoT data models for interoperability. This is sort of a schema.org for IoT models (using JSON Schemas) where the community can browse/search for existing data models that they can reuse. That's why i mentioned in my initial request about human consumption.

As shown in the example i linked above, when we use enums, there is a need to specify/explain the values of these enums for human consumption. I can certainly writhe them in the "description" by using a single string with \n inserted but that doesn't help the human reading/searching the schema as such. One option is to allow a string array (like the "detail-desc" we used). I'm certainly open for alternatives to achieve this. I'm actually interested and want to understand the YAML option better (we actually use RAML currently as the API definition language with JSON Schema being the data model language).

Its likely that our use of JSON Schemas is somewhat of a side case (btw we also do use JSON Schemas for actual payload validation and machine parsing/tooling).

[darshakthakore]

Also just wanted to mention that although enum is the most obvious use case, this also applies to other elements as well like this example

[pdl]

I would say this is actually a different problem.

What I'd really want to do if I were writing this schema is document each of the enum values separately. This means a) there's a usable relationship between the actual value and the documentation, which is good for tooling; b) it's much less likely that the values will get out of sync with the documentation.

enum: [
  {
    "literal": "oic.sec.svc.doxs",
    "description": "Service for establishing a device owner"
  },
  ...
]

At the moment, I don't think this is possible. According to the current spec as I read it, the enum field is just an array of literal values:

An instance validates successfully against this keyword if its value is equal to one of the elements in this keyword's array value.

This is understandable as you just want to do enum : [ "foo", "bar" ]. But it hampers extensibility. It means you cannot say anything about those enum values themselves, and it means making things harder for extensions which can express concepts like 'The value is oic.sec.svc.doxs but if you're presenting a form, the user-facing text you should show is Service for establishing a device owner'.

How much of an issue would it be to alter enums so that objects could be used in this way? Does anyone actually use object literals in enums?

I would imagine the spec text would read something like:

The value of this keyword MUST be an array. This array SHOULD have at least one element. Elements in the array SHOULD be unique.

Elements in the array MAY be of any type, including null.

Elements in the array which are objects MUST have a key "literal".

An instance validates successfully against this keyword if:

• it is not an object, and it is equal to one of the elements in this keyword's array value
• it is equal to one of the values of the "literal" key of the elements in this keyword's array value which are objects

... although at this point, I am wondering whether oneOf would be suitable (and if so, under what circumstances enum is better).

[handrews]

@pdl the issue you want for enum stuff is #57

Let's keep this issue tightly focused on string wrapping.

[handrews]

@darshakthakore I like @awwright 's solution to the enum thing shown at https://github.com/json-schema-org/json-schema-spec/issues/57#issuecomment-247855548

It's a bit awkward when built from enum alone, but it's still pretty compact. And the version using constant is even more clear.

Beyond that, issues of presenting things for humans to browse/read starts getting into #67 (UI Schema). UI Schema is talking a bit more about forms, but could be viewed as any sort of user presentation (for further discussion, comment on that issue :-)

@awwright how do you feel about closing this now? Fundamentally, this is still a limitation of JSON and not something that JSON Schema can change.

[epoberezkin]

@awwright how do you feel about closing this now? Fundamentally, this is still a limitation of JSON and not something that JSON Schema can change.

hear, hear

[paradj00]

"What do you want beyond using \n for newline?" would really like for \n or \\n to work in the places it should, you know in js, python, and c++ something really needs to be interpreted correctly and using YAML for this is like going to the store for the same bottle of water you get from your own freakin sink.

[handrews]

@paradj00 this is still a limitation of JSON, meaning the application/json media type as described in RFC 8259. Per RFC 6839, a +json media type MUST still conform to the JSON RFC.

Once again THIS IS NOT SOMETHING JSON SCHEMA, OR ANY +json MEDIA TYPE, CAN CHANGE.

You might want to consider writing schemas in JSON5 if YAML is not to your taste. Depending on your programming language, there may be libraries that allow this available.

[askirmas]

Was implemented before I was born