Which Mobility properties need to be versioned?

[wrygiel]

Initially, we have planned for entire Mobility object to be versioned. Meaning that we would have a history of changes of each single element and attribute in the XML response.

In Warsaw, Kai proposed a much simpler model: Perhaps we need to version LA components only?

I don't have an opinion on this thus far, but it's surely worth considering. Once we have the "snapshot" section of the model, we should go through all the elements and attributes and verify for which of those we need to track the history of such changes. For example: Should the sending institution track the event that student's name has changed, and should it notify the receiving institution about such changes?

@erasmus-without-paper/all-members

[wrygiel]

I will briefly explain how this was supposed to work in my "early vision" of a model:

I suspected that the sending HEI will be required to keep a kind of materialized view of the Mobility object, along with all the previous revisions of this view. We would also have a library which - given a list of timeline revisions - would be able to give us the exact contents of the Mobility object in any given point in time. This would include all the changes in the view (such as names). In other words, it would require both parties to handle a simple, non-relational XML document versioning system.

This was the only solution I could think of which would allow the sending HEI to properly keep track of all the changes. Whenever something was changed, the server would be able to generate this view again, produce a diff of the changes, and store it to the timeline. This might require running a periodical sweep on all the mobilities in order to detect and materialize all such changes.

However, do we need to keep track of all changes? Maybe not. My vision requires some algorithmic background in order to be implemented properly. I might find it simple enough, but I can see that many developers wouldn't agree with me on this. The definition of the <timeline> element would need to be very strict, and we would be required to design some common tools for handling such XML diffs.

[kaiqu]

I suspected that the sending HEI will be required to keep a kind of materialized view of the Mobility object, along with all the previous revisions of this view. We would also have a library which - given a list of timeline revisions - would be able to give us the exact contents of the Mobility object in any given point in time. This would include all the changes in the view (such as names). In other words, it would require both parties to handle a simple, non-relational XML document versioning system.

Well - it could be stored relationally, with a Revision Number as part of the Mobility key: Any change on any level would result in a new Revision, and the current version would by definition be the latest Revision. This is basically extending the proposed versioning mechanism from LA to the whole Mobility.

[...] Whenever something was changed, the server would be able to generate this view again, produce a diff of the changes, and store it to the timeline. This might require running a periodical sweep on all the mobilities in order to detect and materialize all such changes.

However, do we need to keep track of all changes? Maybe not. My vision requires some algorithmic background in order to be implemented properly. I might find it simple enough, but I can see that many developers wouldn't agree with me on this. The definition of the element would need to be very strict, and we would be required to design some common tools for handling such XML diffs.

Or, if stored relationally (as discussed above), it would be a matter of SQL diffs.

[wrygiel]

I think we should try your proposal first. It sounds like it might work, and it is simpler. Once we have some more XML examples and use cases, it will be easier for connector developers to determine if these are compatible with their local systems.

[mikesteez]

Could we describe why the revisions has to be part of the API? Is it not at the discretion of the node itself how the mobility has evolved?

[wrygiel]

We have discussed it during the previous meetings. There are also some conclusions in the existing documents, but it needs to be document in the schema and Mobility API specification in detail. In short, this is needed in order to:

• Be able to extract a document compatible with EU regulations (it has three sections, each describing LA in different points in time).
• Be able to accept particular revisions of mobility attributes. Only after both partners accept the same revision it becomes operational. After this, both partners may submit draft (not yet accepted) versions of future revisions.

[georgschermann]

I think we should only version the LA on API level I don't think that the rest was meant to be versioned over the network

[kaiqu]

I think we should only version the LA on API level I don't think that the rest was meant to be versioned over the network

LA ended up only containing an optional document reference - the rest of the information was in Mobility. That's why I suggested to remove LA as a separate entity in another issue. (I have done this in the new version of the model)

[wrygiel]

I think we should only version the LA on API level

Still not sure on this. It depends on what you mean by "LA" exactly: 1. the "LA paper document" representation, 2. the LA entity in the model (the one that has been just removed).

The problem is: Many of the other Mobility attributes are present on the LA document, when it is being signed in paper form, as it is now. Once we have all of these different attributes written down and decided upon, we will need to look through them, one by one, and answer this question for each attribute X:

• If an LA document is already signed, and later attribute X is changed, does the LA need to be signed again?

If the answer is YES for any X, then we will need to version entire Mobility entity, I think.

[kaiqu]

I agree with @wrygiel.

[kaiqu]

This needs to be resolved asap.

[StefanJahnke]

Hello,

there is indeed a need to get the confirmation of a change of learning components by at least the other institution. In reality, the student would usually be the one suggesting changes but we agreed in earlier discussions that the institutions act on behalf of the students in EWP. There is nonetheless the need for approving a changed Learning Agreement somehow, as it serves as a contract between the institutions and the student. It is a basis of possible legal dispute and therefore we should be sure that our technical solution caters for understanding who is the institution suggesting the changes and which is the latest version approved by both institutions (one acting ALSO on behalf of the student). We would furthermore need to define that the one proposing the changes always acts on behalf of the student.

For attributes related to the basic information there would probably not be a change needed. As Kai mentions, once we have an updated technical data model, it will be relatively straight forward to mark the elements that would require an approval of sorts.

[kaiqu]

we should be sure that our technical solution caters for understanding who is the institution suggesting the changes and which is the latest version approved by both institutions (one acting ALSO on behalf of the student).

This is a separate point, independent of which elements are to be versioned. I will include some structure on this in the next version of the model.

We would furthermore need to define that the one proposing the changes always acts on behalf of the student.

If I understand you correctly. this is a non-technical matter, and therefore outside the scope of the model/format.

For attributes related to the basic information there would probably not be a change needed. As Kai mentions, once we have an updated technical data model, it will be relatively straight forward to mark the elements that would require an approval of sorts.

I assume you all got my message yesterday about new versions of the model and format on GitHub. Model-wise, the urgent part here is whether any of the information in the Mobility entity needs to be versioned or not, or if it's only the Components.

[wrygiel]

once we have an updated technical data model, it will be relatively straight forward to mark the elements that would require an approval of sorts.

Just working on this. But it's definitely not straight-forward to me :)

[wrygiel]

(...) It is a basis of possible legal dispute and therefore we should be sure that our technical solution caters for (...)

Let's come back to this legal matter for a moment. We have discussed this briefly on one of the meetings, but I don't think we have any written record? And this seems important. We need a clear answer on this before we accept the Outgoing Mobilities API XSDs.

If we don't use full-versioning and digital signatures, then the following scenario MAY happen:

Receiving institution R fetches a learning agreement document D1 from sending institution S's servers. It verifies it, and it's okay. So, R tells S that "I, R, hereby sign D1", and S stores this information on its servers. Now a weird thing happens - due to a programming bug (feature?), or due to a hacking incident - the document is modified and now S claims that R signed a different version of the document - D2. However, R is not aware of the programming error, nor any hacking incident. It turns out that neither S nor R can prove which version of the document S has actually signed.

Can we assume, that it is not among the requirements of EWP to design a system which is able to provide such a proof?

(It's worth noting that full-versioning and signatures will also probably be considerably harder to implement. I'm happy to design it either way.)

[kaiqu]

If we don't use full-versioning and digital signatures, then the following scenario MAY happen:

Receiving institution R fetches a learning agreement document D1 from sending institution S's servers. It verifies it, and it's okay. So, R tells S that "I, R, hereby sign D1", and S stores this information on its servers. Now a weird thing happens - due to a programming bug (feature?), or due to a hacking incident - the document is modified and now S claims that R signed a different version of the document - D2. However, R is not aware of the programming error, nor any hacking incident. It turns out that neither S nor R can prove which version of the document S has actually signed.

Can we assume, that it is not among the requirements of EWP to design a system which is able to provide such a proof?

There's not much traffic on this issue - are we any closer to resolving it?

In the model, both Mobility and LA (which simply represents the underlying LA Components) are at the moment versioned - resulting in two Revision labels. This is cumbersome, but doable. It also entails some conceptual logic for the case when only the Mobility is changed, not the LA (Components):

1. The easiest is to associate the (unchanged) LA with the new Mobility revision, but then the change history is harder (maybe impossible?) to reconstruct
2. To ensure a consistent change history, the LA + Components must be copied to the new Mobility revision

It would be so much easier/clearer if we only had to version LA, not the Mobility itself...

[wrygiel]

We discussed this during the meeting yesterday, but partners didn't offer much input.

The only suggestion was to keep a history of who approved each version of the LA.

[kaiqu]

The only suggestion was to keep a history of who approved each version of the LA.

What does "version" mean here - actually versioned data, or just a version label? (The latter doesn't seem very useful to me)

These non-LA fields make sense to change:

• Status
• Planned/Actual Arrival/Departure Date
• Accept/Reject Date
• Language Competence (skill)

And maybe these?

• EQF Level
• ISCED-F/Subject Area

Do we need versioning of any of them, i.e. is it important to be able to reconstruct earlier values?

[wrygiel]

Do we need versioning of any of them, i.e. is it important to be able to reconstruct earlier values?

It seems to be important for some of the partners, but not for others. My impression was that the partners want to stick with their local solutions, and they don't really care if the partner keeps it versioned or not. The only thing everyone agreed that needs to be (somewhat) versioned is the list of LA components - so that we will be able to generate the PDF LA from that.

Some of the partners keep a full history of all changes to LA components (with some approved revisions), while some other partners keep a set of only two states: a snapshot of the state called "before mobility", and "the last approved state". Also, many partners don't keep a flag if the "last approved state" was actually approved by the partner (and I'm not sure if they want to keep it).

I don't think there is a base for any common model here. I have tried to imprint a model in the API, but this attempt has somewhat failed (people still don't understand what timeline was supposed to be for). Fortunately, we only need a common representation, not a common model. So, perhaps, they easiest way would be to design this API to resemble the structure of the EUC PDF LA template.

But still, we would need a third snapshot state - a "latest draft" of sorts. We want a clear distinction between "last approved state" and this "latest draft". If we intend to use EWP to approve things, then the sending HEI will need to expose such drafts, and the "last approved state" would be updated only after the receiving HEI actually approves "latest draft"...

In other words, this still requires much thought and consulting. I wasn't given time to design this new proposal, but I will prepare a new draft version of this API by the end of February.

[wrygiel]

I will prepare a new draft version of this API by the end of February.

This is already done, and new API is being implemented. So I think we may close this issue.