search

adlnet / xAPI-Spec

The xAPI Specification describes communication about learner activity and experiences between technologies.
https://adlnet.gov/projects/xapi/
905 stars 405 forks source link

If no statements are received, does X-Experience-API-Consistent-Through update? #956

Closed garemoko closed 8 years ago

garemoko commented 8 years ago

Currently the spec is does not say if an LRS should update its X-Experience-API-Consistent-Through date if it receives no statements.

e.g. 2:30pm a statement is received 3pm statements are fetched.

is X-Experience-API-Consistent-Through 2:30pm or 3pm (or a little before either of these)?

This question was causing some problems in development of the test suite.

Consensus on today's call was that it should be 3pm. Testing against SCORM Engine and Wax supports this (they give the more recent time).

It was proposed on the call that we add a SHOULD* or make equivalent changes to the wording to add this requirement.

fugu13 commented 8 years ago

This seems to be a misunderstanding of what the definition of x-experience-api-consistent-through is. It is some time the LRS strongly believes it is consistent by, which will generally be a lag on the current time. The choice of what and how to signal is up to the LRS -- if, at 3pm, the LRS has reason to believe it is only consistent through 1pm (perhaps some queue is lagging a huge amount for some reason), it might return 1pm, and that would be the right response -- even if statements have been received after, that might be present in the response. Since it really can't be anything other than a best guess most of the time it doesn't even need to be monotonically increasing, as it would if it were precise.

Also, any LRS for which it isn't at least a bit before the current time for the LRS is implementing the header incorrectly, as no LRS I'm aware of both 1) uses a fully linearizable database (none do even that) and 2) ensures the complete system is fully linearizable, including web servers. Without those guarantees, every LRS is eventually consistent. So if an LRS is queried at exactly 3:00.00, the time returned better be at least a hair before that (really, probably at least 20 or 30 seconds to account for server drift in any LRS that involves multiple servers, before we're even talking about how many databases can take quite a few seconds to reach full consistency under heavy load).

garemoko commented 8 years ago

The header is only mentioned in once place (in 1.0.2 at least) where it says:

The LRS MUST include the header "X-Experience-API-Consistent-Through", in ISO 8601 combined date and time format, on all responses to Statements requests, with a value of the timestamp for which all Statements that have or will have a "stored" property before that time are known with reasonable certainty to be available for retrieval. This time SHOULD take into account any temporary condition, such as excessive load, which might cause a delay in Statements becoming available for retrieval.

By that definition, at value of 2:30pm for the example above would satisfy the requirements. I think we just need to add "latest possible", so the new text would read

... value of the latest possible timestamp for which all Statements...

garemoko commented 8 years ago

Actually, we don;t need "possible". "latest timestamp" is fine

fugu13 commented 8 years ago

I've replied in the PR. This is a much more complicated topic than the model being used to reach the conclusions you have.