JSON literal expressions

[AnthonyDGreen]

Despite being out of vogue, XML is still in use in tons of businesses and many VB customers I talk to continue to enjoy the feature. It's natural to ask whether we can duplicate the wins here with JSON. I was a little hesitant on JSON until I saw that SQL Server 2016 has built-in JSON support.

I've implemented a working prototype here though it's out of date and needs updating to work with the latest Roslyn sources/VS2017. It's build around JSON.NET. The syntax is just JSON:

Dim obj As JObject = { "messageType": "", "content": "SomeText" }

And like XML it supports embedded expressions:

Any value not native to JSON (string, number, JSON object, JSON array, true, false, null) is an embedded expression:

Dim obj As JObject = { "name": <expression here> }

Any key not a string is an embedded expression.

Dim obj As JObject = { <computed name>: "" }

Any member not a key-value-pair is an embedded expression.

Dim obj As JObject = 
{
  "key1": "value1",
  from kvp in dictionary select (e.key: e.value)
}

The big "win" uses I imagine are copying and pasting a JSON example/message from some website into your code and then substituting in your own dynamic values with embedded expressions. I hope this could be a powerful tool for communicating to REST services since users won't have to create separate POCO objects and serialize them or manually construct JSON.NET JObjects:

The second big win use is pairing JSON literals with pattern matching, enabling the developer to decompose a JObject off the wire without deserializing to a .NET Object.

Select Case message
    Case Match { "messageType": "ack",
                          "timestamp": Date(d) }

        Console.WriteLine($"Message receipt received at {d}")
End Select

No need for any special member access syntax since the ! operator works natively with JSON.NET objects. We should consider some sort of pattern-based support so the feature could target either JSON.NET or Windows.Data.Json but if we can only do one, JSON.NET is preferred.

[craigajohnson]

JSON.NET is fine. And not to be overdramatic but we would use this every. single. day.

[AnthonyDGreen]

Then I should get my prototype working again so you can play with it and validate that hypothesis! :)

[AdamSpeight2008]

@AnthonyDGreen How do you build and run the prototype? I'm using VS2017 Preview Version 15.3.0 Preview 3.0

[AnthonyDGreen]

Yeah, I should probably just rebase it on the latest sources.

[AdamSpeight2008]

I had selected the wrong start up project. Needed to use VisualStudioSetup.

[AnthonyDGreen]

Have been working on getting my branches up-to-date to put out an updated prototype of this. Have it working but still need to fix a few little bugs and I'll upload a VSIX:

[KlausLoeffelmann]

Did you rebase on current bits?

[AnthonyDGreen]

I rebased on master as of two days ago, I think.

[aarondglover]

Love it. Great idea

[tmat]

@AnthonyDGreen What is the syntax for comments? ', // or comments not allowed?

[AnthonyDGreen]

I've been using http://json.org/ to define the standard. JSON itself doesn't have a notion of comments and unlike XLinq, I don't think JSON.NET provides a way to encode comments (or whitespace in general). We should discuss forbidding it.

Whether a string is a JSON string or a VB String is also an open design question.

[reduckted]

Whether a string is a JSON string or a VB String is also an open design question.

If you're referring to how escaping is done (i.e. "" or \"), then I would suggest going with JSON strings. Right now, I can copy an element from an XML document and paste it into the editor and it's a valid XElement without needing to make any changes to it. I would expect to be able to do the same thing with a JSON document.

[franzalex]

@tmat:

What is the syntax for comments?

As @AnthonyDGreen stated, there is no defined standard for comments in JSON. This is quite logical, since the JSON format is mainly intended for data transfer.

I am, however, of the notion that this is the greatest injustice done the format. Being a text-based format which is largely human-readable, it is not uncommon to find people creating or tweaking JSON files by hand. For this singular purpose, I believe that having some comments in the format would have been rather helpful.
Many other text-based formats (e.g. XML) support comments so I see no reason why the JSON 'spec' fails to include this.

In my case, when editing JSON by hand, I use the C-style multi-line comments (i.e. /* */) to include comments where appropriate since it has a clearly-defined termination marker which isn't a line break.
I do not recommend using the single-line comment marker (i.e. //) since a poorly-written JSON minifier could unintentionally comment out all KV pairs that follow the comment marker.

[AnthonyDGreen]

@reduckted absolutely in agreement that the copy/paste principle is paramount for this feature. The problem is that the way the feature is designed right now there is no special syntax that denotes what's a VB expression; whatever isn't a JSON expression is a VB expression. That leads to some weird interactions while typing/exploring:

? { "key": "Line1\r\nLine2" } ' This is a JSON string.
' BUT
? { "key": "Line1\r\nLine2".ToString() } ' This is a VB string.

So dotting off of the string, parenthesizing it, etc, could silently make it no longer JSON. We could report a warning on some such cases. Other cases are worse:

? { "key": "Inline \"quotes\" use \"" } ' Everything is fine.
? { "key": "Inline \"quotes\" use \"" } ' Chaos ripples throughout the remainder of the file.

Note: If a user does attempt to use "" escapes in the JSON string we can report and error and have the IDE fix it. That's probably better than supporting "" escapes in JSON strings.

Of course, the scenario might be very unlikely and I'd hate to spoil the simplicity of the syntax with some sort of (%= %) syntax to denote VB expressions. Technically the IDE will colorize VB strings and JSON strings differently so this might not even be a thing at all.

[VBAndCs]

I like the idea, and in general I like to use this value pair json syntax to intialize any dectionary or expando object. I suggest that you auto generate a a filed or a property for each key and add it to the JObject, so we can hard type the key names. this will solve the fatal mistake when a typo adds a new key to the collection. Another solution is making the JObject an expando object, and auto add the properties to it from the initializer behind the sense so they can be available in intellisense in design time We can use some roles to rename invalid key names, like converting each space to _ in property names... etc.

[DualBrain]

Love the idea... love the video… would definitely use this. I also want to confirm the comment at the very start of this proposal "despite being out of vogue, XML is still in use in tons of businesses and many VB customers I talk to continue to enjoy the feature." as being very true. I'm literally working on a project at this moment where 1/3rd of the third-party API's I'm working with are XML only, 1/3rd are JSON only and 1/3rd give the option of choosing between one or the other. So having the ability to easily interact with JSON (as we have with XML) would be a huge timesaver.

Of course, the scenario might be very unlikely and I'd had to spoil the simplicity of the syntax with some sort of (%= %) syntax to denote VB expressions. Technically the IDE will colorize VB strings and JSON strings differently so this might not even be a thing at all.

Although there is serious geek cred associated with the implementation you've done where the line is blurred between JSON and VB; I will say that having this sort of syntax makes it clear that you are "doing something" above and beyond the JSON syntax. The "ugly" way is clearly visible and the action doing so doesn't get lost is the sea of JSON. So some more thought should be given that.

If I were looking for a "flaw" (as it is really, really, really minor) hiding in what I see it is in the reliance on having to package another assembly. Meaning that if you were to make a small single executable utility, you couldn't do so if utilized this. As I said, a minor issue... but one that doesn't exist with regards to XML literals. ;-) With that said, I'd rather have the "penalty" of have a multiple assembly distributable then not having the choice at all for the ease of use.

All-in-all this is really awesome!

[VBAndCs]

Please look at this proposal https://github.com/dotnet/roslyn/issues/38274 . It will allow strongly typed JSON and XML settings.

[bandleader]

Re: comments, I assume we aren't suggesting that we produce JSON with comments in it (this would be invalid JSON). I assume the question is how to add VB comments in middle of a JSON literal.

I assume this would be done in the regular way?

Dim name = {
  "first": "Anthony", 'Former VB PM
  "last": "Green"
}

Of course there would be no way to do this all in one line, but that's a longstanding limitation of VB's not having inline (/* ... */) comments (which also means that other constructs such as array literals, CallExpressions, etc. can't be done in one line with comments).

[paul1956]

With _ ' Comment you can get very closer and the feature could be expanded to work in more places but if you want a single line solution I have not thought of a non-breaking way since anything is allowed after a ' until EOL