provide protobuf schema

[maltegrosse]

As mentioned two years ago, a protobuf would be more helpful in sense of data model instead of having an outdated c# implementation. any news on this?

see https://github.com/ADAPT/ADAPT/issues/331

and

https://github.com/maltegrosse/ADAPT-protobuf

[knelson-farmbeltnorth]

@maltegrosse The intent of the ADAPT Standard is to be a platform/language agnostic data model, whereas the earlier ADAPT Framework (github.com/adapt/adapt) is a .net plugin architecture for proprietary file translation.

The json schema (alpha) for the model is in the root of this repo, and documentation is a work in progress at adaptstandard.org. At a high level, most objects are representable as json, with vector layers represented in GeoParquet and raster layers as GeoTiff.

[maltegrosse]

@knelson-farmbeltnorth thank you for clarification. To be honest, it is really hard to follow/implement ADAPT if you not directly involved in internal the development, even I really like the idea and structure the data model provides. Just mentioning few items

• adaptframework.org
  • there is no information regarding this fundamental change to adaptstandard.org - I would really appreciate, for me and others, to mention it more clearly somewhere and not let me struggle to go into this c# valley of tears :)
  • also talking about API is very misleading in that sense (as its mainly a data model, not an application interface)
  • I assume the "framework" is deprecated and not longer maintained?
  • some clearification should be added to my open issue on adapt https://github.com/ADAPT/ADAPT/issues/331 because my development was exactly done because myself and others faced the issue which will now be solved with adaptstandard.org, as far as I understood.
• adaptstandard.org
  • this website should somehow mention the current stage including some timeline and more important the actual goal which it tries to solve
  • I would really appreciate if the "json" like schema is not finalized and the (nearly industry stadard) protobuf will be used instead:

Compared to json

• it provides type safety
• auto generation in multiple programming languages (as at the end of the day the data model needs to be implemented in some way)
• clear object/entity structures and references between them, which leads also to easy diagram generation ( https://github.com/maltegrosse/ADAPT-protobuf/blob/master/diagrams/ , e.g. Adapt Common )
• protobuf is highly extensible. (only mention some examples: create ontop of the protobuf an API using GRPC, even a REST proxy is available for someone who prefers it, also generate database schemas is possible and documentation generation ).
• and if you really want to stick to json, all protobuf implementation offer one easy method ".toJson"

[zwing99]

Hey @maltegrosse, your feedback about the development of this model is accurate. We have struggled how to share the progress. I totally agree that the two websites should start to reference each and give a little clearer picture of what progress has been made. @jim-wilson-kt put the new site together and might be able to help us get that done sooner rather than later. We should lay out some timelines or link to a timeline in this repo or something.

On the topic of protobuf vs json schema, I wonder if you are hearing just "json" and not "json schema". JSON Schema defines a strict schema for json (including types). For what it is worth I personally prefer protobuf over json schema, but a tool called SCORE is being used to model and document ADAPT. SCORE emits json schema as a default. Json Schema has type generators for all sorts of languages (like protofbuf does). I want to be clear you feedback that protobuf might be better is fair and reasonable but it might be better than you think. Take a peak at the links I dropped.

The final thing to think about with relation to your feedback is that ADAPT is not meant to be your internal data model. It is a data model that is sufficient and complete enough to pass around data inside of the Ag Industry. So things like a database emitter, while they could be valuable have some limited value. IMHO. ADAPT standard is meant to be written as a json file (that obeys the linked schema) and a few linked files for the dense geospatial data.

[knelson-farmbeltnorth]

@maltegrosse I agree with @zwing99 that there is still work to do on documentation. Coincidentally, adaptstandard.org was updated today. I hope you find it slightly more informative; there is definitely still much to add.

Re the Framework, it is not deprecated, but it is rather stable at this point. The updates are minor and infrequent. It is widely used to exchange geospatial field operations data between equipment manufacturers and farm software providers. Unfortunately it was never properly documented, and there are a more than a few entities in the model that never got use (or consistent use). We've actively sought to avoid those problems in the Standard.

[maltegrosse]

Thank you both for more detailed explanation, few final thoughts:

• great that adapt-standard website has been updated, perhaps adaptframework.org should mention something too?
• regarding json schema & protobuf: even by going over the links @zwing99 provided, I still dont understand it, but seems already been decided (even the framework had some initial protobuf implemented) :)
• as json-schema seems so universal, (still mention its javascript object notation) perhaps you can also provide auto-generate profobuf files (as initially requested on the github issue), so i am more than happy :-)
• I understand that adapt should not be considered as an internal data model, but certain parts need to be consistent with you own internal business data model, so storing data structed (in a rel. db) is always preferred than just dumping ADMs into a data swamp (lake). e.g. internal / external ID management.
• final thoughts: would it be possible to provide a simple end to end example, where data is exchange, like from extraction from a farm management system until reading it in another farm management system. I think it would be helpful for me and others, to get an idea how the makers of ADAPT thought about this.

[knelson-farmbeltnorth]

@maltegrosse We do intend to provide examples for each scenario, we just haven't got there yet.

[knelson-farmbeltnorth]

@maltegrosse The first six scenarios on the website defining Work Records and Work Orders now contain examples. https://adaptstandard.org/docs/scenarios/

[knelson-farmbeltnorth]

Closing this item for housekeeping of issues on the project page. Please open a new issue for anything additional.

In the case it wasn't clearly stated above for those finding this issue at a later time, the team intentionally chose json-schema for the definition of the standard vs. protobuf.