Option to retrieve posted statement as response body?

[vbhayden]

Has there been any discussion on alternative response bodies for submitting an xAPI statement?

Currently, the only response for submitting any number of xAPI statements is an array of those statements' id properties in their original submission order. With the LRS potentially assigning timestamp, stored, id, authority, and version, there are properties of the statement (specifically authority) that either can't be or probably aren't known to the sender. As statement forwarding isn't part of the spec, a second HTTP request is the only spec-ensured way to get those final properties.

As background, ADL is looking at scale-ability and LRS federation as topics of interest -- specifically, how something like an Apache Kafka cluster might be used for notifying interested systems of new xAPI statements. The most modular plan seems to be standing up a Kafka-enabled proxy in front of an LRS and producing validated statements to a topic.

The problem then is that the xAPI statement should be broadcast as it appears in the LRS, likely requiring that second HTTP request. In the case above and in controlled ecosystems, we can infer most of the properties in question at the proxy layer, but there might be cases where that isn't possible.

So I was curious if there's been any discussion on the response body structure (like a header specifying the LRS to return the statement in its final form), or if someone could summarize whatever discussion led to the current id-only response?

Thanks!

[bscSCORM]

Hi @vbhayden, I don't recall not having these properties as the LRS had set them being discussed as an issue in the early days of the spec, but I think the important point to keep in mind here is:

The LRS MUST include the header "X-Experience-API-Consistent-Through", in ISO 8601 combined date and time format, on all responses to Statements Resource requests, with a value of the timestamp for which all Statements that have or will have a "stored" property before that time are known with reasonable certainty to be available for retrieval. This time SHOULD take into account any temporary condition, such as excessive load, which might cause a delay in Statements becoming available for retrieval. It is expected that this will be a recent timestamp, even if there are no recently received Statements.

And note stored is defined as:

The time at which a Statement is stored by the LRS. This can be any time between when the LRS receives the Statement and when it is written to storage.

This is a round about way of saying it, but the LRS is not required to have finished storing the submitted statements at the time it responds to a request to submit xAPI statements. This "Eventual Consistency" was something we discussed as an important use case for xAPI, in order to allow distributed systems. The "stored" time the LRS sets may well be after that response is done returning, and it's possible the other properties are determined at that time. All the LRS really has to determine is that the statement is valid and it will store the statement, and the ID it will assign.

So it would be a big breaking change to require LRSs to even determine the stored time, and to a lesser extent those other parts of a statement it might set, at the time it responds to a request to store statements.

Although "statement forwarding" is not part of the spec, we did discuss the concept of "statement polling" a good bit during the development of the spec, to ensure that would be a reasonable way to get statements out. If you are able to take that approach, you can request large batches of statements at one time, so it shouldn't add many additional HTTP requests.

[vbhayden]

Yeah the storage process would inevitably complicate this, as you don't want the successful storage of a statement holding up its response.

The idea of doing something like this 200% riddled with breaking changes, so I tried to phrase it as a request for information / insight to convey my appreciation for that.

Thanks for the info!

[vbhayden]

With the X-Experience-API-Consistent-Through header, the only potentially uncontrollable property would be authority, but a first stab can make that property's fallibility known to players. Going to close this unbloat the issues.