Open derberg opened 1 year ago
I really like it and would go with relying on schemaFormat as well
How do you think about this Szenario:
components:
messages:
firstLevelNestedMessage:
payload:
$ref: '#components/schemas/LearnXInYMinProtocolBuffer#NestedMessages.FirstLevelNestedMessage'
schemas:
LearnXInYMinProtocolBuffer:
schemaFormat: application/vnd.google.proto;version=3
schema:
$ref: https://gist.githubusercontent.com/shankarshastri/c1b4d920188da78a0dbc9fc707e82996/raw/49e733499bfb302d9ecf320f2eca2f752f7e257b/LearnXInYMinProtocolBuffer.proto
components.schemas.LearnXInYMinProtocolBuffer
pointing to a whole proto|xsd file containing multiple schemas.components.messages.firstLevelNestedMessage
pointing to components.schemas.LearnXInYMinProtocolBuffer
and picking NestedMessages.FirstLevelNestedMessageDo a schema needs to be distinct or can it be multiple schemas. My opinion is it should be distinct. Either needs to contain an anchor pointing to a single schema or the file is only allowed to contain a single root level schema
sorry I do not get what you mean by distinct
in this context. Not sure how it relates to your example
payload:
$ref: '#components/schemas/LearnXInYMinProtocolBuffer#NestedMessages.FirstLevelNestedMessage'
this will not work, right? according to your PR, afaik, payload.$ref
means it is pointing to AsyncAPI Schema. So normal resolution will be applied and fail
you would need to do
payload:
schemaFormat: application/vnd.google.proto;version=3
schema: '#components/schemas/LearnXInYMinProtocolBuffer#NestedMessages.FirstLevelNestedMessage'
@dalelane @char0n @smoya can you have a look 🙏🏼
basically the understanding is that this is a non breaking change that we could introduce in 3.0 and release as 3.1
but if it is breaking, then better know it sooner 😆
I'm not super confident in making statements on this, as this is getting much deeper into the weeds of JSON specs than I've got before!
If I've understood the description above correctly, then it seems reasonable to do this in 3.x as adding behaviour for new schemaFormat values.
Has anyone tried spiking a prototype along the lines of the custom resolvers described above to convince ourselves that this is workable?
This issue has been automatically marked as stale because it has not had recent activity :sleeping:
It will be closed in 120 days if no further activity occurs. To unstale this issue, add a comment with a detailed explanation.
There can be many reasons why some specific issue has no activity. The most probable cause is lack of time, not lack of interest. AsyncAPI Initiative is a Linux Foundation project not owned by a single for-profit company. It is a community-driven initiative ruled under open governance model.
Let us figure out together how to push this issue forward. Connect with us through one of many communication channels we established here.
Thank you for your patience :heart:
Still valid? @derberg @dalelane @GreenRover
Yes, definitely a 3.1 candidate forme.
This issue has been automatically marked as stale because it has not had recent activity :sleeping:
It will be closed in 120 days if no further activity occurs. To unstale this issue, add a comment with a detailed explanation.
There can be many reasons why some specific issue has no activity. The most probable cause is lack of time, not lack of interest. AsyncAPI Initiative is a Linux Foundation project not owned by a single for-profit company. It is a community-driven initiative ruled under open governance model.
Let us figure out together how to push this issue forward. Connect with us through one of many communication channels we established here.
Thank you for your patience :heart:
This issue aims to kinda summarize idea mentioned dunring Adding support for non-JSON schemas (April 18th 2023) that could be a good solution for using JSON Reference for non-JSON structures like Protobuf or XSD.
JSON Reference spec
Current JSON Reference used in AsycAPI spec is JSON Reference v0.3.0.
In general, JSON Reference focuses on defining how a reference object should look like and requires only the following:
$ref
$ref
is a string value of URIKey words used to describe requirements, like SHALL or SHOULD follow https://datatracker.ietf.org/doc/html/rfc2119
Even thought the JSON Reference says in the intro
Later in spec in
Resolution
section it says in only SHOULD:Also, for fragments that start after
#
, JSON Pointer is not required but recommended:Conclusion is
$ref
pointing to.proto
or.xsd
for example.proto
for example, because JSON Pointer is just a recommended solution. So we can for example follow:.proto
, when there are nested types that need to be referenced, we can follow standard Protobuf approach for pointing to nested types. So for$ref: "https://gist.githubusercontent.com/shankarshastri/c1b4d920188da78a0dbc9fc707e82996/raw/49e733499bfb302d9ecf320f2eca2f752f7e257b/LearnXInYMinProtocolBuffer.proto"
if someone wants to referenceFirstLevelNestedMessage
type, fragment would look like, probably :smile:,#NestedMessages.FirstLevelNestedMessage
. So the final reference would be:.xsd
it is much easier, as there is already a XPath in place that allows to create a fragment pointing to a specific part of the XML/XSD. That requires further exploration and some input from experts but in theory, if we have https://www.w3.org/2001/XMLSchema schema, the xpath fragment toformChoice
simple type would probably bexs:schema/xs:simpleType[@name='formChoice']
so the resulting reference is something like:Now the question is how on a spec level specify that dereferencing for given
$ref
should not use JSON Pointer but XPath or some other solutionSolutions on a speck level
schemaFormat
basedAs discussed during Adding support for non-JSON schemas (April 18th 2023) meeting, we can say that dereferencing mechanism depends on the
schemaFormat
.schemaFormat
specifies AsyncAPI Schema, JSON Schema, Avro or any other JSON structure, we follow JSON Reference + JSON Pointer in fragmentsschemaFormat
specifies Protobuf, we follow JSON Reference + Protobuf Nested Types Reference in fragmentsschemaFormat
specifies XSD, we follow JSON Reference + XPath in fragmentsAlternative
Something I mentioned before somewhere in an issue or Slack, but can't find reference.
We can just follow example from the JSON reference resolver that we use, that explains how custom resolvers work -> https://apitools.dev/json-schema-ref-parser/docs/plugins/resolvers.html. So for example they had a use case that someone keeps schemas in a MongoDB and want to reference directly to the database. The solution is that
$ref
value starts with"mongodb://"
because plugins can discover that likecanRead: /^mongodb:/i
.so we can do:
"$ref": "xsd|https://www.w3.org/2001/XMLSchema#xs:schema/xs:simpleType[@name='formChoice']"
"$ref": "proto|https://gist.githubusercontent.com/shankarshastri/c1b4d920188da78a0dbc9fc707e82996/raw/49e733499bfb302d9ecf320f2eca2f752f7e257b/LearnXInYMinProtocolBuffer.proto#NestedMessages.FirstLevelNestedMessage"
Neverheless, sharing just for reference as when I thought about it, I did not take
schemaFormat
into consideration. So relying onschemaFormat
is better imho.Tooling
json-schema-ref-parser
that we use is pretty flexible:And writing them is not hard same using. So, again in theory, in current JS Parser we can do
$RefParser.dereference(mySchema, { resolve: { proto: ourProtoResolver }});
whenever we encounter a proto reference. This will of course require some refactor in Parser as dereferencing happens once, globally for entire AsyncAPI document, and in new approach that should be done one by one, applying different resolver depending onschemaFormat
I think I covered entire discussion and summary. Lemme know if something is missing
pinging meeting participants: @GreenRover @jonaslagoni @fmvilas