Closed netmilk closed 8 years ago
So, wouldn't omitempty
be a superset of optional
?
Optional allows a key to be omitted from the resulting JSON, similarly omitempty
allows a key to be omitted, but would fail if the key was present with its "empty" value.
In Go, which this suggestions seems to have been influenced by you are looking at these rules in the JSON package:
The empty values are false, 0, any nil pointer or interface value, and any array, slice, map, or string of length zero.
Which essentially map this way..
JSON Types | Empty Value For Omission | Go Type That Encodes To Value |
---|---|---|
Array | [] |
[]slice |
Object | {} |
struct{} |
Number | 0 |
int/float.. |
String | "" |
string |
Bool | false |
bool |
Null | null |
nil pointer |
@zdne null as a primitive type is not hiding the true type, because null is a valid JSON type along with number, bool, and string. Allowing null in place of another object is only part of the issue with MSON, the bigger issue in my opinion is the lack of null as a type.
@obihann null
may be a type in JSON but no so in many other programming languages or serialization formats. As such it does not convey any information about the type of the value besides it is "not set".
As discussed in https://github.com/apiaryio/mson/pull/35#issuecomment-134181674
I would like to come up with solution for following:
null
(not set) while retaining information about the actual type of the value.""
, []
, {}
, 0
, etc.)Does everyone involved here agree that solving this will be solution for his/her respective problems?
@zdne Good point, I keep look at this from the issue if null as a value, not null as a type and what that may or may not represent.
And yes I believe we need:
null
""
, 0, non-nullSomething that may need to have some thought though, is 0 truly empty? If one wants an optional number field would 0 be representative of a empty value or an actual 0 value? Perhaps that is not really relevant here.
Going back to null
as a allowed value...
- key1: null (string)
- key2 (string)
{
"key1": null,
"key2": ""
}
✅
{ }
✅
let key1: String?
let key2 = String()
✅
By default, optional values with null
or default
values MAY but DOESN'T
HAVE TO be ommitted. To force "omit empty" use omitempty
type modifier.
- key1: null (string, omitempty)
- key2 (string, omitempty)
{
"key1": null,
"key2": ""
}
❌
{ }
✅
A type placeholder – generics support.
- key1 (any)
{
"key1": null
}
✅
var key1: AnyObject
✅
NOTE: The generics sections of specification needs tighten up.
Thoughts?
Looking at the first MSON example:
- key1: null (string)
- key2 (string)
I like it, however it seems to me this would mean by default we allow null, is this your thought? If so can specify a rule to inforce no null, or would required
handle that?
--edt--
On second thought I don't believe required being used for non-null is a good idea, required is for when a field has to exist, which is very different than non null in therms of schema.
@obihann
by default we allow null, is this your thought?
No, default is the default (erm) empty value – that is "", {}, [] etc
.
If you would like to override it I would suggest
- key1: Hello! (string)
- default: null
Which is equivalent to
- key1 (string)
- default: null
- sample: Hello!
Which brings me to...
- key (any)
For the hypothetical any
type, in JSON representation the default value would
be null
. Hmm.
So maybe we do have it?
That is
- key (any)
{ "key": null }
I guess it is all just one big mental exercise for me (and hopefully others)
in that null
type is just "any" type.
Sorry, I didn't mean default in the sence of a default value. What I was trying to say was are we accepting null as an acceptable value without any additional flags? Here is an example of what I'm trying to say:
null
is ALWAYS accepted as a value for any type
- key (string)
Accepted
{"key": "cat"}
Also Accepted
{"key": null}
null
is accepted as a value for any type WHEN notset
is applied
- keyA (string, notset)
- keyB (string)
Accepted
{"keyA": "cat"}
{"keyB": "cat"}
Also Accepted
{"keyA": null}
{"keyB": "cat"}
Not Accepted
{"keyA": null}
{"keyB": null}
I think your "option B" is what was discussed above with nullable
type modifier.
Please disregard my previous comments on "any type", albeit we may need it for generics in this scenario it is not applicable. Too late I've realized that its equivalent in JSON world would be all 7 primitives types (from the JSON Schema perspective).
So I feel like we are back in square one :)
Based on all the suggestions, heres what I suggest:
Applies to basic data structure and type definitions. [Types][] that are sub-typed from Primitive Types MUST NOT contain a [Member Type Group][] or [Nested Member Types][].
null
Specifies a null type.
boolean
Specifies a type with allowed values of true
or false
.
string
Specifies any string.
number
Specifies any number.
Defines extra attributes associated with the implementation of a type.
nullable
- The value may include {}
, null
, or []
required
- instance of this type is requiredoptional
- instance of this type is optional (default)fixed
- instance of this type structure and values are fixedsample
- Alternate way to indicate a [Value][] is a sample. See [Sample][].default
- Alternate way to indicate a [Value][] is a default. See [Default][].A sample
Type Attribute is mutually exclusive with default
.
Update: Removing
null
as a type, as @pksunkara pointed out with the addition of anullable
rule it is not needed
If we are using nullable
type attribute, we don't need a null
Type Attribute.
@pksunkara great point, I'll remove that!
nullable - The value may include {}, null, or []
I think that this would be misleading and wouldn't allow to differentiate between null
and {}
unless you limit it to their respective "notset" value:
string, boolean, number
-> null
Array[xxx]
-> []
Object
-> {}
nullable
would mean "not-set" or "empty" value of their respective type (I wouldn't go with nullable
as name.)
- keyA (string, nullable)
- keyB (Array[string], nullable)
{"keyA": "cat"}
{"keyB": ["cat", "dog"]}
{"keyA": null}
{"keyB": []}
{"keyA": null}
{"keyB": null}
null
is an existing type
nullable
would mean "empty" value of their respective type only for Array and Object (I wouldn't go with nullable
as name.)
- keyA (Enum[string, null])
- keyB (Array[string], nullable)
{"keyA": "cat"}
{"keyB": ["cat", "dog"]}
{"keyA": null}
{"keyB": []}
{"keyA": null}
{"keyB": null}
I think option B would better capture the intention to both provide a way to specifiy possible null
values and to allow "empty" values for Arrays and Objects.
Update:
Option B would also allow:
- keyA (Array[Enum[string, null]], nullable)
{"keyA": ["cat", "dog"]}
{"keyA": []}
null
type{"keyA": [null]}
Good point, perhaps we need to define nullable
and empty
, however would empty
to better differentiate between things like []
or {}
and null
. What about this:
nullable
- The value may be null
allowempty
- The value may be {}
, []
, 0
, or ""
I'm starting to think we don't need a primitive type null because it would likely never be used outside of an enum
to allow the additional value, I can't see this ever being required:
- key null (null, required)
@obihann : Did you take my update into account? I agree that there probably won't be much use outside enums, but I think that the combination with enums alone is very powerful and desirable on its own.
@hobofan I may have missed it, so your feeling is that we use null as a primitive, and nullable as an option to define something like [null]
?
So if I wanted a string with that allows null I would do enum(string, null)
however if I want an array that allows null I would just do array, nullable
, and it would all look like this:
- keyA null (enum[string, null], required) - accepts strings or nulls
- keyB [null] (array, nullable, required) - accepts arrays or arrays containing null
- keyC null (enum[array, null], required) - accepts arrays or null
{
"keyA": null,
"keyB": [null],
"keyC": null
}
I agree the combination of a null
type and a nullable
option is a very powerful combination, however as you mentioned using nullable
as a way to define [null]
as an acceptable value is not straight forward to me. Perhaps allowempty
would be best and then it could be like this:
allowempty
- The value may be {}
, []
, [null],
0,or
""`
Sorry for the misunderstanding. I meant to say that something like allowempty
should be used to express {}
, []
, 0
, ""
and that Arrays like [null]
and [null, "cat", null]
should be able to be expressed by something like:
- keyA: (Array[enum[string, null]], required) - accepts Arrays whose items are either strings or nulls
Ahhh, that makes sense. So this is where we are at then as I understand:
null
primitive
the addition of an
allowempty` optionApplies to basic data structure and type definitions. [Types][] that are sub-typed from Primitive Types MUST NOT contain a [Member Type Group][] or [Nested Member Types][].
null
Specifies a null type.
boolean
Specifies a type with allowed values of true
or false
.
string
Specifies any string.
number
Specifies any number.
Defines extra attributes associated with the implementation of a type.
allowempty
- The value may include {}
, 0
, '""', or []
required
- instance of this type is requiredoptional
- instance of this type is optional (default)fixed
- instance of this type structure and values are fixedsample
- Alternate way to indicate a [Value][] is a sample. See [Sample][].default
- Alternate way to indicate a [Value][] is a default. See [Default][].- keyA (enum[string, null], required) - Accepts `null` or `string`
- keyB (enum[array[string], null], required) Accepts `null` or array of **only** `string`
- keyC (array[enum[string, null], required) Accepts an array of `null` or `string`
- KeyD (object, allowempty, required) - Accepts a populated or empty (`{}`) object
- KeyE (string, allowempty, required) - Accepts a populated or empty (`{}`) object
Accepted:
{
"keyA": null,
"keyB": null,
"keyC": [null],
"keyD":{},
"KeyE": ""
}
or
{
"keyA": "jeff",
"keyB": ["jeff"],
"keyC": ["jeff"],
"keyD":{"a":123},
"KeyE": "jeff"
}
Incorrect:
{
"keyB": [null],
"keyC": null,
"keyD": null,
"KeyE": null
}
@obihann +1
On Aug 24, 2015, at 5:48 AM, Z notifications@github.com wrote:
I would like to come up with solution for following: allow a value of any type to be null (not set) while retaining information about the actual type of the value. allow a value of any type to be a default empty value ("", [], {}, 0) I think there are three states -- four, really:
- not even present in a given instance of the object type (an "optional", typed field)
- present, but set to null, with full information on what the type would be if there were a value (a "nullable", typed field)
- present, but set to a type-appropriate empty value (a "defaultable", typed field)
- and of course the boring, ordinary "set to some actual value of appropriate type"
Perhaps you were assuming #1 (optional) and #4 (actual), since they're already well-specified?
In the null case, I'm not certain where this type information would be retained. The MSON document itself (a document written in MSON; not the spec, written about MSON) should specify the type (like "(string, optional, nullable, defaultable)"). Were you also implying that the serialized form (JSON, XML, etc.) would carry some type info?
And I think maybe the "retained type info" problem exists even in some default-empty-value cases. Certainly it does in more pedantic languages, like Java, that can force you to type-cast a value before you're even allowed to check nullity. But if "actual type retention" applies only to the MSON itself, then the example immediately above covers it.
Jack Repenning Jack@netgate.net
@hobofan I've created a pull request with the latest changes, let me know if you still think its a good idea lol.
@jrep I believe null should be a primitive type, not a option such as nullable. This is because a) (as you stated) in JSON null does not reflect the alternative (string, object, etc) type that could be used as a value, and b) null is itself a valid type both from virtually every programming language, and JSON itself.
Also regarding the omitance of optional and actual, you are correct this is because they are already defined.
I think there are three states -- four, really:
- not even present in a given instance of the object type (an "optional", typed field)
- present, but set to null, with full information on what the type would be if there were a value (a "nullable", typed field)
- present, but set to a type-appropriate empty value (a "defaultable", typed field)
- and of course the boring, ordinary "set to some actual value of appropriate type"
We need to make sure MSON can be used to render the above four states.
But I agree with @zdne's comment that we shouldn't lose the information about the actual type. But that leads to the question whether @zdne meant it in the MSON or the rendered JSON/XML?
@obihann The problem with having a null
type is that since MSON is just a modelling language, it is not tied to JSON directly. Yes, XML contains a method to represent null
too, but it is not definitely an actual type. This is why I am completely against null
primitive type.
If we can use the tags like nullable
to represent the above 4 states, we can find a solution to this problem.
@pksunkara The very description of MSON is that is a markdown language for describing JSON and JSON Schema, if you read the specs for JSON schema section 3.5 you will see null
is a primitive type. If you are choosing to ignore this fact and steer MSON into a different direction then fine, however the very description of MSON should be changed to reflect this.
@obihann In MSON's README I see
Markdown Syntax for Object Notation (MSON), a Markdown syntax compatible with describing JSON and JSON Schema.
and the very second sentence is
MSON is a plain-text, human and machine readable, description format for describing data structures in common markup formats such as JSON, XML or YAML.
From my POV, MSON is definitely trying to be suitable for describing JSON and JSON Schema, but it has much higher ambitions than to be just a syntax sugar for those two.
If the only way we can resolve this issue is to use a nullable
option and not a null
primitive then I will gladly outline the new options, and submit a pull request for it, I don't want to delay this any longer as the lack of null
support be it either a primitive or a option is hindering a project I am currently working on.
That being said, I would much prefer a null
primitive.
Just to see if were on the same page, this is what I'm thinking:
Defines extra attributes associated with the implementation of a type.
nullable
- The value may be null
allowempty
- The value may include {}
, 0
, "
, or []
required
- instance of this type is requiredoptional
- instance of this type is optional (default)fixed
- instance of this type structure and values are fixedsample
- Alternate way to indicate a [Value][] is a sample. See [Sample][].default
- Alternate way to indicate a [Value][] is a default. See [Default][].- keyA (string, nullable, required) - Accepts `null` or `string`
- keyB (array, nullable, required) Accepts `null` or array of **only** `string`
- keyC (array[enum], required) Accepts an array of `null` or `string`
- (string, nullable, required)
- KeyD (object, allowempty, required) - Accepts a populated or empty (`{}`) object
- KeyE (string, allowempty, required) - Accepts a populated or empty (`{}`) object
Accepted:
{
"keyA": null,
"keyB": null,
"keyC": [null],
"keyD":{},
"KeyE": ""
}
or
{
"keyA": "jeff",
"keyB": ["jeff"],
"keyC": ["jeff"],
"keyD":{"a":123},
"KeyE": "jeff"
}
Incorrect:
{
"keyB": [null],
"keyC": null,
"keyD": null,
"KeyE": null
}
I am holding off on creating another pull request however I do have a branch with the above attributes with examples available and ready
@obihann Thanks for all the work you are doing. But please do hold off on creating the PR since this discussion is not yet finalised. :smile:
@pksunkara no problem, however at some point we are going to have to make a discussion since this conversation has circled several times, and the author @netmilk has not been a part of the conversation in quite a while.
Since you are actually a contributor and a member of the apiaryio organization, how do you feel about the current suggestion?
@obihann The current solution does look good but we need to think of a few things before committing to it. MSON can be quite complex and we need to think of all the scenarios. I feel like we need to consider some related things which haven't entered into the discussion like Samples
and Default
.
As I said earlier, this problem is not as simple as it looks.
@pksunkara I see Samples
being larger of an issue than Default
, specifically because Default
really only gives you one option. Based on that the following could be written:
- name: null (string, nullable, required) - Name can be a string or null, default is null
{
"$schema": "http://json-schema.org/draft-04/schema#",
"id": "",
"type": "object",
"properties": {
"name": {
"id": "/name",
"anyOf": [
{"type": "string"},
{"type": "null"}
]
}
},
"required": [
"name"
]
}
{
"name": null
}
{
"name": "jeff"
}
Samples
shouldn't be much more difficult, the fact that you cannot combine a Sample
with a Default
might even simplify it further.
- name (string, nullable, required) - Name can be a string or null, default is null
- Sample
- null
- "jeff"
- "bob smith"
I believe the the Optional
attribute would be unaffected, and Fixed
would be very similar to the above example for Sample
.
Lots of interesting discussion going on here!
@obihann thanks for leading the push. Do not worry about @netmilk – I am sure he is happy :smile: (as he is on vacation ATM)
So. I think one of the problems here is we do not have clearly described the problem and trying to solved to many things at once. Let me try to summarize what is going on here, and lets first agree on that.
null
@neonichu having a body JSON key with null value. There is no way how to express it in MSON at this moment. Can you please add support for null primitive type in MSON?
How to describe a member type with null
value?
Introduce null
as a possible value for any sample.
- key: null (string)
Introduction of a new keyword may affect existing MSON documents.
A null
value would translate into the null
TYPE.
A null
value would translate into an element with xsi:nil="true"
ATTRIBUTE.
A null
value translates to the nil
VALUE.
A null
value translates to NULL
(or 0
) VALUE.
A null
value translates to None
VALUE of NoneType
TYPE.
A null
value translates to nil
VALUE of NilClass
TYPE.
A null
value translates to undef
VALUE.
This – seemingly straightforward issue – brings many related issues on the table.
{ key: null }
vs.
{ key: "" }
null
" typeOf:
NOTE: By rendering is meant representing what was described in MSON in the respective language / serialization format.
any
" typeBefore we look at possible solutions – are there any objections against what is written in https://github.com/apiaryio/mson/issues/26#issuecomment-134632896 ?
None whatsoever, you hit the nail on the head when you said:
Rephrased Problem
How to describe a member type with null value?
Good! We do not have to, necessarily, solve all the related problems here. If we focus on the main thing. However, it will be good to keep them on mind as we are trying to solve the main issue so we wont block or complicate the solution of the related issues later.
We could focus specifically on null
in this issue, resolve that, then create additional issues for the other issues / ideas raised. I would be glad to help work through all of them. I fell like family here now :P
He he. Agree. Let's focus on the null
. First without closing paths to the other issues.
So hypothetically, if we do introduce null
as a keyword and a possible value:
- key: null (string)
What needs to be solved?
Does the example render as
{ "type": [ "string", "null" ] }
or
{ "type": "string"}
or
{ "type": "null"}
Does the example render as (remember it is optional in the example)
{ "key": null }
or
{ }
NOTE: If you would like null
as a text you would have to backtick escape it - key:
null(string)
.
Sorry, I must have blinked: what is "fixed"? (Update: for the record, I guess I meant "what is fixed
?". But both forms of the question are now answered;-)
@zdne I believe the correct JSON schema would be something like this:
{
"$schema": "http://json-schema.org/draft-04/schema#",
"id": "",
"type": "object",
"properties": {
"name": {
"id": "/name",
"anyOf": [
{"type": "string"},
{"type": "null"}
]
}
},
"required": [
"name"
]
}
Regarding 2, I'm not a huge fan of {} === null
I think that is a good place for an allowempty
attribute however as we discussed we should focus on null
only, so I think we just ignore empty values for now as the introduction of null
shouldn't directly effect them.
All that being said @pksunkara is not a fan of null
as a primitive type and though it is my preference, I am comfortable with a nullable
attribute.
@jrep Nothing really was fixed, @zdne and I just felt that the conversations of things like empty values should be left out of this issue because they are not directly related to the original issue, the lack of null
support.
Mostly we are trying to circle back to the beginning and resolve what was originally reported as an issue.
@jrep all the values are considered samples. If you have fixed
a thing it means it has to appear exactly as it was written.
Consider array of strings with two sample values
- key (array[string])
- one
- two
vs. a tuple (pair) that always has "one" and "two" literals in it
- key (array[string], fixed)
- one
- two
fixed
also fixes the order and count so
- key (array, fixed)
- (string)
- (number)
says the array is a pair of a string and a number in this order.
ah, fixed or fixed
, bounded context ;)
In this Dredd issue is @neonichu having a body JSON key with
null
value. There is no way how to express it in MSON at this moment. Can you please add support fornull
primitive type in MSON?