JSONTag: Tagged JSON

JSON has won the battle for universal data interchange format. There are still some holdouts using XML or SOAP, but if you start a new online API today, the default is JSON. Even the linked data proponents have made a JSON version, called JSON-LD.

However, JSON has a problem. It is too restricted. There are too few basic data types. This means that if you do need to specify a specific data type, like Date, you must go out of your way to either define a JSON Schema, or, another external definition, like JSON-LD does. This leads to unnecessary complexity.

Instead of creating another Something-in-JSON format, JSONTag enhances the JSON format with additional type information inline. Every JSON file is valid JSONTag. Every JSONTag file can be easily stripped of type information to get valid JSON.

JSONTag looks like this:

<object class="Person">{
    "name": "John",
    "dob": <date>"1972-09-20"
}

Type information is inserted immediately in front of a value, enclosed in < and > characters. There is a restricted list of types. But you can add extra attributes to the types for more information.

For example, though there is only one type object, just like in JSON, you can enhance it with attributes. Here we've added the class attribute. The type syntax is based heavily on the HTML tag syntax. There is one difference, attribute names are more restricted. You aren't allowed to use the - character in an attribute name. This is because it leads to less readable code in most programming languages.

Install / Usage

npm install @muze-nl/JSONTag

In the browser:

<script src="https://github.com/muze-nl/jsontag/raw/main/node_modules/JSONTag/dist/browser.js"></script>
<script>
    let p = JSONTag.parse('<object class="Person">{"name":"John"}')
    let s = JSONTag.stringify(p)
    let type = JSONTag.getType(p) // 'object'
    let className = JSONTag.getAttribute(p, 'class') // 'Person'
</script>

In node:

import JSONTag from '@muze-nl/jsontag'

let p = JSONTag.parse('<object class="Person">{"name":"John"}')
let s = JSONTag.stringify(p)
let type = JSONTag.getType(p) // 'object'
let className = JSONTag.getAttribute(p, 'class') // 'Person'

Changing and building dist files

I've been using parcel to create the simplest build config that I could. Simply run this command to create the dist files:

npx parcel build

API Reference

parse

JSONTag.parse(stringValue, reviver, meta)

JSONTag.parse works identical to JSON.parse and is backwards compatible with JSON strings.

It adds a meta parameter, which can be omitted. The format of meta is:

{
    index: {},
    unresolved: [],
    baseURL: "https://localhost/"
}

The meta index is updated to add { id: <value reference> } pairs, for each value that has a tag with an id attribute. For each <link> tag found, which has a URL value not found as an id in the meta index, an entry in the unresolved array is added, like this:

{
    src: <parent object>,
    key: <property name or index>,
    val: <link URI>
}

On each parse() call, if you pass the meta object, all unresolved entries will be checked to see if there is now a corresponding value. If so, the <link> objects will be automatically replaced with a reference to that value and the entry removed from the unresolved array.

The baseURL value is used to parse and validate link and URL values. It will also be used to match id attribute values with link values in the future. This will allow you to automatically link together jsontag documents with different baseURLs.

stringify

JSONTag.stringify(value, replacer, space)

JSONTag.stringify works identically to JSON.stringify. But in addition it also can stringify circular references and it will also codify any types or attributes set on values with the setType and setAttribute methods.

getType

JSONTag.getType(value)

This function will return the type of the value. It will return the normal JSON types, e.g: string, number, boolean, array, object. But if the value has been annotated with setType, it will return that instead.

setType

JSONTag.setType(value, type)

This will annotate the value as being of type type. Valid types are: object,array,string,number, decimal,money,uuid,url,link,date,time,datetime, interval, timestamp, text, blob, color, email, hash, phone, int, int8, int16, int32, int64, uint, uint8, uint16, uint32, uint64, float, float32, float64

getAttribute

JSONTag.getAttribute(value, attributeName)

This will return the attribute value as a string, or undefined.

let p = JSONTag.parse('<object class="Person">{"name":"John"}')
let className = JSONTag.getAttribute(p, 'class') // 'Person'
let id = JSONTag.getAttribute(p, 'id') // undefined

setAttribute

JSONTag.setAttribute(value, attributeName, attributeValue)

This will set the attribute attributeName to the given attributeValue. If you pass an array of strings as attributeValue, the array will be joined using a space character and the result set as attributeValue.

let p = JSONTag.parse('{"name":"John"}')
JSONTag.setAttribute(p, 'class', 'Person')
let s = JSONTag.stringify(p) // '<object class="Person">{"name":"John"}'

addAttribute

JSONTag.addAttribute(value, attributeName, attributeValue)

This method differs from setAttribute in that it will append the attributeValue to an existing attribute, if the attribute with that attributeName is already present.

let p = JSONTag.parse('<object class="Person">{"name":"John"}')
JSONTag.addAttribute(p, 'class', 'Employee')
let s = JSONTag.stringify(p) // '<object class="Person Employee">{"name":"John"}'

removeAttribute

JSONTag.removeAttribute(value, attributeName)

This will remove the entire attribute from the tag for the given value.

let p = JSONTag.parse('<object class="Person">{"name":"John"}')
JSONTag.removeAttribute(p, 'class')
let s = JSONTag.stringify(p) // '{"name":"John"}'

getAttributes

JSONTag.getAttributes(value)

This will return an object with all attributes and attribute values.

let p = JSONTag.parse('<object class="Person">{"name":"John"}')
let attrs = JSONTag.getAttributes(p) // { class: "Person" }

isNull

JSONTag.isNull(value)

This will return true if the value is either null or an instance of the JSONTag.Null object. JSONTag adds a Null class, to allow you to add attributes to it. The javascript and JSON null value is special in that all null values are identical. This doesn't allow you to add attributes to a specific null.

let p = JSONTag.parse('<object class="Person">null')
let className = JSONTag.getAttribute(p, 'class')
if (JSONTag.isNull(p)) {
    console.log('null object with class '+className)
} else {
    console.log('normal object with class '+className)
}

JSONTag Types

The list below is preliminary. The aim is to have a good coverage of most used or useful types, without creating an unwieldy specification. It should be only slightly harder to implement JSONTag support compared to JSON support. Some inspiration was taken from HTML, PostgreSQL and RED

JSON derived

• number
• string
• object
• array

Lowlevel scalars

• text
• blob
• int (uint, int8, uint8, int16, uint16, int32, uint32, int64, uint64)
• float (float32, float64)

Semantic types

• color
• date
• decimal
• email
• hash
• interval
• link
• money
• phone
• range
• time
• timestamp
• url
• uuid

Boolean values

JSONTag supports only direct boolean false and true. You cannot set a tag or add attributes to them. This is because it is impossible in javascript to create an object that evaluates to false. Even new Boolean(false) evaluates to true. There is simply no way to add metadata to a specific false value.

Circular data, or references

One shortcoming of JSON is that it cannot represent data with internal references. JSONTag solves this by introducing the <link> type. Here's an example:

<object id="source">{
    "foo":{
        "bar":"Baz"
    },
    "bar":<link>"#source"
}

When parsed the property bar is a reference to the parent object of that property. The current stringify implementation (javascript) automatically add id attributes and link values when a reference to a previous value is found.

This allows for complex graphs to be serialized to JSONTag and revived correctly. The <link> type does not specify a specific format, other that a string. It is implied that the URL format is explicitly supported as the default.

Typed Null values

JSON only supports a single null value. Each null is identical to each other null. This makes it impossible to add type and attribute information to it. So JSONTag uses a Null class. Each instance of the Null class is unique. The Null class has no properties or methods, except for:

• isNull: property, always true
• toString: returns ''
• toJSON: returns 'null'
• toJSONTag: return typeInfo + 'null'

Any access to other properties or methods results in an exception

Because JSONTag now can keep type and attribute information, you could create your own typed Null object when reviving JSONTag data.

JSONTag.parse() only creates a JSONTag.Null object if the stringified version has a tag, otherwise it will just return a normal null.

Monkeypatching

Upgrade a program to use JSONTag by:

JSON.parse = JSONTag.parse
JSON.stringify = JSONTag.stringify

Since the API is identical this just works. Use at your own risk however...

Reviver extension

JSON.Parse has a reviver parameter, and JSONTag.parse has it too. It is fully backwards compatible, but it adds an extra parameter:

let result = JSONTag.parse(JSONTag, function(key, value, meta) {

})

The extra meta parameter is an object with an ids property, which is an object with all id attribute values found in the JSONTag text. As well as a unresolved property, which is an array with all <link> values, which haven't yet been resolved.

TODO

• figure out if the URL parser must become strict, to avoid server-side request forgery using differences in the parsing of invalid URLs

• resolve id attributes and link values using the meta.baseURL value

• if jsontag.parse links together objects from different baseURLs, then stringify will generate a single new document from multiple seperate documents. Should we keep track of which object is part of which original document, so that we can recreate a specific document with a specific baseURL? Similar to how Quads work in Linked Data?

• add a streaming parser

• change the id attribute to always be a valid URI, so the value and id attribute value can become identical

• only allow id attribute on