ADAPT / Standard

ADAPT Standard data model issue management
https://adaptstandard.org
MIT License
8 stars 1 forks source link

Serialization Format #49

Open knelson-farmbeltnorth opened 2 years ago

knelson-farmbeltnorth commented 2 years ago

In both the ADAPT Technical Call and ADAPT Standards call on August 3, the question arose whether we have an adequate standards candidate without defining a viable serialization (exchange) format for the high density spatial data.

To date, we've treated this as a separate problem that lay in the future. We did discuss it at the Midyear Meeting and agreed that such an exchange format should utilize geojson, without defining further details.

This issue is to facilitate discussion on it.

knelson-farmbeltnorth commented 2 years ago

The current ADM Plugin serializes the Catalog as json, and serializes the remainder as protobuf. In addition to the fact that any additions to the model must be manually duplicated in the protobuf schema within this plugin, there is a desire among a number of ADAPT contributors to move away from spatial formats that are not human readable, with an eye toward geojson. File size, however, is a key consideration, and certainly played a role in the initial selection of protobuf.

One possible approach, assuming we continue to serialize "header" data in json (to include OperationData, DeviceElementUse and Working Data/Unit of Measure), each of these objects carries an instance id that could be useful for optimizing the size of the geojson. Thus each feature could contain something like: {properties:{ "1234" : 12.34, //OperationData1-sprayer-row1-vrAppRateVolumeActual - gal/ac (comments to be omitted from the actual payload) "5678": true}} //OperationData1-sprayer-row1-dtRecordingStatus There would be a question if this format compresses to a manageable file size, and if enough of those calling for human readability find the need to cross reference ids in the header as any improvement on the protobuf.