Proposal: JSON datatype spec

[whatisgalen]

Arches can use resource instance data for almost any purpose. Internally, the system settings are resource instance data. If a developer wanted to leverage the Arches Resource Model pattern to work with arches functions in a certain way, interact with the map layer manager in a novel way, or just pass information to some other resource instance, there are times when having a json datatype (on the frontend, not to be confused with the official JSON) would be the most convenient and logical place to store, for example, metadata about a tile with file-list data, or map configuration information, etc.

In theory, most use cases are addressable by existing backend solutions; the purpose of this proposal is that a frontend solution exists to manage data that is most appropriately stored as json rather than string.

The datatype "json" then would work as follows:

• selectable in the node config in graph designer like any other datatype
• validated solely for correct json formatting
• excluded from indexing (for the time being)
• simple json editor as widget

[whatisgalen]

@philtweir this may be of interest to you. My original impetus for this was to create a node that could store map-related configuration information directly in json, however given that your permissions work has leveraged creating permissions entities out of Arches resource models, such a configuration field existing as a node in that resource model might be useful for governing or informing permissions infrastructure that is already interrogating tile data

[philtweir]

Certainly of interest - we have hit a couple of examples where we would like to store machine-readable data that is more complex than basic types but may not warrant a new datatype - as we are doing more API-driven work, both on mobile and Vue frontends, this is increasingly applicable.

Possible examples, expanding on @whatisgalen 's list:

• configuration overrides, such as persistent (shared) display preferences for a monument
• data structures for custom front-end-specific widgets, like chart layouts or colour palettes (which are not relevant to e.g. API, and do not seem to justify constructs like DataTypes in the data layer)
• multi-modal or contextual values, such as phone-compatibility for a digital resource
• reference/API-relevant data (non-frontend), such as original cached values from an upstream data source, whose own schema is inconsistent, undocumented or third-party-controlled (useful for replication, recovery and audit)

Admittedly some of these could be implemented another way. I could also see the risk of including a handy type that does not enforce a schema into core, tempting implementers into the habit of not creating proper datatypes or functions when they should, and being dissuaded from making Arches-supported widgets to code around it on the frontend - i.e. they could undermine their Arches instance as a semantically-aware data platform by poking too many holes, before they get the benefits.

If so, perhaps a middle way might be to allow a JSON type that requires a schema validation class to be chosen in dropdown (from a Python search path) - perhaps pydantic, attrs or even dataclasses class - or the frontend configuration widget requiring JSON-Schema with the JSON - not necessarily to reject incompatible data, but to make the path of least resistance involve a planned structure, and perhaps the default JSON type could also store whether the validation has passed (in the tile data) on save? Of course, for a familiar implementer, who gets the tradeoff, subclassing to create a "Schemaless JSON" widget would be pretty trivial. Although maybe I am overthinking!

Either way, definitely see the merits.