search

oasis-tcs / odata-specs

OASIS OData TC: Markdown sources for OData specification drafts. https://github.com/oasis-tcs/odata-specs
https://oasis-tcs.github.io/odata-specs/
Other
6 stars 2 forks source link

Inconsistencies for transient entities handling in JSON format #1928

Open falbert-ptc opened 2 months ago

falbert-ptc commented 2 months ago

Both 4.0 and 4.01 seems to have inconsistencies regarding handling of transient entities in the JSON format.

Null ID for transient entities in JSON:

However, it seems to be incompatible with at least 2 other use cases:

  1. In the case of recursive navigations ($expand=Nav($levels=max)), it is stated you must replace the entity instance by a reference, using its id:
  2. In the case of navigationLink, it is stated it must contain the read URL appended with the navigation property name:

In the case of the XML format, this is not an issue since even transient entities have an id. The client can use those transient ids when parsing and make sense of the references.

ralfhandl commented 2 months ago

@falbert-ptc Thanks for reporting this, we will discuss it in one of the next OData TC meetings.

ralfhandl commented 1 month ago

Regarding use case #1 - recursive navigation between transient entities: I don't think we have a problem here because 11.2.5.2.1.1 Expand Option $levels only requires that services

MUST solve circular dependencies by injecting an entity reference somewhere in the circular dependency

Transient entities don't have an identity, so there can't be circles consisting only of transient entities.

@falbert-ptc Do you have use cases for circular references involving transient entities?

ralfhandl commented 1 month ago

Regarding use case #2 - navigation links of transient entities: with 4.02 4.3 Transient Entities may have an explicit read URL, which means they also have default navigation links (read URL plus navigation segment).

If they don't have a read URL, they may have explicit navigation links.

@falbert-ptc: Do you have use cases for transient entities with navigation links?

falbert-ptc commented 1 month ago

We do have use cases with circular references between transient entities.

We have an endpoint that returns structured data (parent-child relationship) of transient entities. Each transient entity represent an information aggregate of multiple data sources, and cannot be addressed individually outside of the current response. Each transient entity has multiple navigations to said data sources.

While rare, there are cases where there could be circular references (direct or indirect) between an ascendant and a descendant in the structure. In that case, we are supposed to insert an entity reference.

The issue is with the JSON format, where transient entities are specifically stated to not have an entity id. So it's impossible to have an entity reference or a read URL + navigation segment. Were that not the case, the above 2 use cases would not be a problem. For XML, transient entities have an entity id so there is no issue with what the protocol prescribes w.r.t. navigation links or entity references.

Note that in my research I've only looked at 4.0 and 4.01. I wasn't aware that 4.02 had been published. Although, I did not find any update to the JSON format in 4.02 (https://docs.oasis-open.org/odata/odata/v4.02/).

ralfhandl commented 1 month ago

@falbert-ptc Thanks for the explanation!

How would you recognize circular references between transient entities? Do these entities have an internal id that you can use to determine whether two of them are really the same and not just coincidentally have the same representation?

ralfhandl commented 1 month ago

For XML, transient entities have an entity id

With XML, do you mean the OData Atom Format Version 4.0?

If yes, do you use this in "production" software?

falbert-ptc commented 1 month ago

Correct. Each transient entity has a unique ID within that response. This is generated in the format described in the Atom format (odata:transient:{some-generated-unique-identifier-to-not-break-atom-parsers})

That way, the client can use those unique ID when finding an entity reference.

falbert-ptc commented 1 month ago

For XML, transient entities have an entity id

With XML, do you mean the OData Atom Format Version 4.0?

If yes, do you use this in "production" software?

Yes, that's what I meant. We do use it in production, commercial software. Although our support for XML is less extensive than for JSON. Because of market demand we are focusing on JSON.

ralfhandl commented 1 month ago

Each transient entity has a unique ID within that response.

Could you use this to construct a non-canonical URI to use in the @odata.id, pretend that your transient entities are not transient, and return 410 Gone if someone actually tries to request such a URI?

That way you could use entity references to express cycles within a response.

ralfhandl commented 1 month ago

We do use it in production, commercial software.

Did you encounter any gaps in the Atom format?

falbert-ptc commented 1 month ago

Each transient entity has a unique ID within that response.

Could you use this to construct a non-canonical URI to use in the @odata.id, pretend that your transient entities are not transient, and return 410 Gone if someone actually tries to request such a URI?

That way you could use entity references to express cycles within a response.

Yes, that would be the easiest and preferred way. The issue is the JSON format currently specifies that the "@odata.id" value must be null if it's a transient entity: "If the entity is transient (i.e. cannot be read or updated), the odata.id annotation MUST appear and have the null value." . We had this reported as a defect in our software (where odata.id wasn't null for transient entities in JSON). When we fixed it to set it to null, it made it compliant with the spec but we didn't realize the impacts on the 2 use cases mentioned in this ticket.

ralfhandl commented 1 month ago

We had this reported as a defect in our software (where odata.id wasn't null for transient entities in JSON).

Which (kind of) client (software/app) did stumble over this, and how could they figure out that the entities were transient?

I'd like to know how much chaos we may cause if we relax the specification and allow "temporary" or unresolvable URIs in addition to null for transient entities.

falbert-ptc commented 1 month ago

We do use it in production, commercial software.

Did you encounter any gaps in the Atom format?

What we're finding challenging with Atom is that there are less examples, or examples are a lot more abstract (as opposed to the Order/Customer/etc. examples). One use case that comes to mind is action invocation: we found it hard to understand the correct way to pass action parameters in the request body. But I can't say that we've found gaps.

Because this is not a widely requested feature, our investment has mostly been on the JSON format.

ralfhandl commented 1 month ago

read URL + navigation segment

With 4.02 we explicitly allow transient entities to have read URLs, which means they also can have default computed navigation links that can be omitted from the payload. This of course requires the service to correctly answer requests to {navigationLink}/{navigationProperty}.

Already with 4.0 it was possible to have explicit navigation links in transient entities, if one doesn't mind the resulting verbosity of the JSON responses.

falbert-ptc commented 1 month ago

We had this reported as a defect in our software (where odata.id wasn't null for transient entities in JSON).

Which (kind of) client (software/app) did stumble over this, and how could they figure out that the entities were transient?

I'd like to know how much chaos we may cause if we relax the specification and allow "temporary" or unresolvable URIs in addition to null for transient entities.

Honestly, I do not remember the specifics. In our records, I could not find a mention of the actual client software that reported the issue. I think it was a middleware like Boomi, but I'm really not sure. Also, at that time our OData implementation would not correctly generate a unique transient entity ID (the entity ID in Atom XML was also not unique and wasn't prefixed with "odata:transient:"). I think the client was trying to use that ID as a read URL, but it would fail because it could not determine it was a transient entity.

I think the problem of an unresolvable URI exists today in the Atom format because it always includes the entity id. ... <a:id>odata:transient:MyType:123456</a:id> <a:link rel="edit" href="odata:transient:MyType:123456"/> <a:category scheme="http://docs.oasis-open.org/odata/ns/scheme" term="#Domain.MyType"/> ...

{ "@odata.context": "http://<serviceRoot>/$metadata#Domain.MyType", "@odata.type": "#Domain.MyType", "@odata.id": null, ... }

But, if the entity id is in the specific "odata:transient:" format, then the client would know that it is not resolvable.

falbert-ptc commented 1 month ago

Another way to say this is: if I could reuse the same generated transient ID in both XML and JSON, that would resolve all our problems :). My main problem is with the verb "MUST" for the entity ID being null in the JSON format. If it was a "MAY", that wouldn't be an issue. Or something along the lines "the odata.id annotation MUST appear and MAY have either the null value or a unique transient id following the pattern odata:transient:{some-generated-unique-identifier}"

ralfhandl commented 1 month ago

@falbert-ptc Please check & review PR #1942