IIA version change impact on cooperation conditions hash

[mkurzydlowski]

As we already know IIA major version change impacts the cooperation conditions hash, as it changes the XML namespace that is part of the cooperation conditions XML element being hashed. So even if a IIA version change doesn't impact the fields in this element, the hash changes.

Let's discuss how this case should be handled and what impact this has.

Should the cooperation conditions hash change after version change or should it be stored and stable until IIA changes? If we choose to recalculate hashes then should we send CNRs? If the partner chooses to recalculate hashes, should we recalculate IIA approvals?

[umesh-qs]

I suggest a solution in which the approving party would have to issue a new approval as soon as it detects that the owning party has switched to a new version of the IIA API. Most of the approvals could be updated automatically without the intervention of the approving IRO if the software of the approving party inspects the IIA GET response of the owning party and finds no business level changes in it.

How many providers would agree to such solution? We all have put a lot of effort looking for the solution which would allow us to avoid re-approval. Do we all agree that re-approval does not hurt?

@janinamincer-daszkiewicz I don't think @jiripetrzelka suggested "re-approval does not hurt". What he is suggesting here is that each provider does the work of extracting only the required fields (defined in spec for each version) for hash calculation rather then using the XSLT. But I would do that only if the XSLT does not support a specific case. Most of the transformation for hash calculation should be handled by standard XLST. I also agree that IIA ID should be inside cooperation condition if we choose to add it in hash calculation

P.S. The truth is that if there is no change, and partners do not send CNRs to each other, and partners do not do automatic IIA gets not triggered by CNRs, then the change in hash will not be discovered and re-approval will not be needed. If partners start modifying mutually approved IIA, the re-approval will be needed anyway or may be done (if they decide to revert back to the previous version).

@janinamincer-daszkiewicz I don't think so. This is an ideal situation that everyone would want to be in, but the current EWP API design cannot achieve this. CNR is not full proof. Even if we trust CNR, there is no reason why a CNR cannot be re-triggered even when there is no change.

[janinamincer-daszkiewicz]

Yeah, I know, this is why we would need re-calculation. There IS a difference between 0 and 1 mobilities.

Exactly, but if I said no mobilities and you approved it, now I don't want to tell you that they are 1.

The main problem is: do we have a set of elements, mandatory or not that made up an IIA? If yes, we can have an XSLT that considers them all and fill with empty value if not previously considered or no more considered. If not, I don't see any easy solution: we can always have something that is not compatible with the previous version.

In that particular case we have no choice. We want mobility per year required and positive, especially when we at the same time make a total number of months or days for student and staff mobility optional. Every change from empty to positive will have to be re-approved.

Every partner who has mutually approved IIA with empty mobility per year element, when making a change in the local system from version 6.0.0 to version 7.0.0 will have to fill that field with some value (probably calculated on the basis of the total number of months or days) and send CNR to the partner. That would be processed as any other modification.

The only possible way to avoid re-approval would be to commonly agree that some numbers of months or days for student and staff mobility are equivalent to some numbers of mobilities per year and if the transformation confirms such change it does not need re-approval.

With academic years the situation is different, the old and new ways of describing the set of academic years are equivalent on a business level.

[janinamincer-daszkiewicz]

As to the XSLT proposition, I do agree that something in this direction would be beneficial and would reduce the probability of unnecessary changes in the conditions-hash in the future (when switching from version 7 to version 8), but a similar effect could possibly be attained if we decided to specify that the namespace should not be part of the hash calculation, and how many decimal digits should be present in number of months, and the number of digits in ISCEDs and possibly something else as well.

May be giving XSLT would be the simplest way to specify all these conditions? May be such XSLT should take the complete IIA, with IIA-ids, and extract from it only these parts which should be preserved for further comparison? @georgschermann do you have the needed XSLT for IIAs in version 6.0.0?

[demilatof]

We want mobility per year required and positive, especially when we at the same time make a total number of months or days for student and staff mobility optional. Every change from empty to positive will have to be re-approved.

Every partner who has mutually approved IIA with empty mobility per year element, when making a change in the local system from version 6.0.0 to version 7.0.0 will have to fill that field with some value (probably calculated on the basis of the total number of months or days) and send CNR to the partner.

Who asked for this? BPOs, DG EAC? They explicitly said that are void all the approval where one or both of the IIAs lacks the mobility-per-year element? If so, no question, but they should clearly state so that the involved HEIs start immediately to re-approve them, even with the current version. If not, please consider that a mutually approved IIA is a valid agreement that cannot become invalid just because a technical specification requires that FROM NOW ON a field is mandatory.

[janinamincer-daszkiewicz]

You have IIA in version 6.0.0, it is mutually approved. What will you send in response to IIA get in version 7.0.0?

[georgschermann]

A few thoughts I want to add (again):

Beeing backward compatible with the hash calculation is imho not neccessary, to have a chain of proof one side needs to have the old IIA GET and the old IIA APPROVAL GET anyway which are not invalidated by a new hash and between v6 and v7 it is not feasible to keep the hash the same via XSLT or another algorithm. When the hash is used to check for any changes, cross version checks could be made internally if needed, otherwise the worst case would be that a user is notified of a change where there is none.

I also think that need for a new approval is in some cases a good thing, especially when relevant data is added/changed like the . The old approved version is still valid, still in-effect, etc., the new version is not valid until it is mutually approved again. This is exactly how I would want it. Also this should be already possible in all systems when changes to mutually approved IIAs are officially allowed.

For the iia-id in hash calculation I am not sure, if the iia had a previous approval this one stays valid anyway, if the CCs don't change... why should anyone care which ids are on it? The partner IIA-ID is not relevant for the content of the IIA. If I change my own IIA-ID the IIA and the approval for this are on new URLs, so new approval is needed anyway. If I change my partners IIA ID, I could send him an approval CNR for the new one, if I un-map my partners IIA-ID there is no need for any approval. So from my understanding a new approval only becomes needed when the content changes, until then I am perfectly fine with the previous approved version.

The manifest is capable to describe endpoints for different versions of the same API. Since we are in production since iia v1, we had quite some version rollovers with up to 3 differenct versions available simultaneously. Not sure if I would recommend it for the v6-v7 rollover but it worked quite well. The effort is unfeasible though.

You have IIA in version 6.0.0, it is mutually approved. What will you send in response to IIA get in version 7.0.0?

the same iia, changed for v7 output, with different hash

@georgschermann do you have the needed XSLT for IIAs in version 6.0.0?

I posted links to snippets for the namespace and start/end year, I didn't have the time to check for all other changes. Quick XSLT example for namespaces and years based on full iia-get-response (copy/past, most likely with room for improvments):

<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:output method="xml" indent="yes" encoding="UTF-8"/>
<xsl:template match="/">
    <xsl:copy>
        <xsl:apply-templates/>
    </xsl:copy>
</xsl:template>
<xsl:template match="@*">
    <xsl:attribute name="{local-name()}">
        <xsl:value-of select="current()"/>
    </xsl:attribute>
</xsl:template>
<xsl:template match="*">
    <xsl:element name="{local-name()}">
        <xsl:apply-templates select="@* | * | text()"/>
    </xsl:element>
</xsl:template>
<xsl:template match="text()">
    <xsl:copy>
        <xsl:value-of select="current()"/>
    </xsl:copy>
</xsl:template>
<xsl:template match="*[local-name() = 'partner']"/>
<xsl:template match="*[local-name() = 'in-effect']"/>
<xsl:template match="*[local-name() = 'conditions-hash']"/>
<xsl:template match="*[local-name() = 'iias-get-response']"><xsl:apply-templates/></xsl:template>
<xsl:template match="*[local-name() = 'iia']"><xsl:apply-templates/></xsl:template>
<xsl:template match="*[local-name() = 'receiving-academic-year-id']">
<xsl:choose>
  <xsl:when test="not(following-sibling::*[local-name()='receiving-academic-year-id'])">
     <end-academic-year> <xsl:value-of select="."/> </end-academic-year>
  </xsl:when>
<xsl:when test="not(preceding-sibling::*[local-name()='receiving-academic-year-id'])">
     <start-academic-year> <xsl:value-of select="."/> </start-academic-year>
  </xsl:when>
</xsl:choose>
</xsl:template>
</xsl:stylesheet>

[demilatof]

You have IIA in version 6.0.0, it is mutually approved. What will you send in response to IIA get in version 7.0.0?

I don't think this is the correct question. The question is what BPOs/DG EAC want to do with mutually approved IIAs.

If they don't want a re-approval, we can:

• Declare mobilities-per-year nonNegativeInteger, so we can accept zero
• If they explicitly forbid the zero value, we can agree to a very large number, that is 9999
• Use XSLT that receive as input an IIA without that field and produce as output an XML with that field set to 9999
• Compute the hash code and compare it with the IIA Get v7 (best if transformed with another XSLT)

Anyway, I strongly suggest to verify if mobilities-per-year greater than zero is a DG EAC requirement or not. Declaring it as nonNegativeInteger and stating that it should be >0 in the description is a solution we have already used in the specifications.

[demilatof]

You have IIA in version 6.0.0, it is mutually approved. What will you send in response to IIA get in version 7.0.0?

the same iia, changed for v7 output, with different hash

I think she refers to elements that could not be present in version 6 but MUST be present in version 7 (such as mobilities-per-year)

[georgschermann]

I think she refers to elements that could not be present in version 6 but MUST be present in version 7 (such as mobilities-per-year)

yes, and we would serve exactly that, one v6-version without it and the same iia v7-version with it, internally we may or may not keep the data on all fields, but we would serve it to all versions as it needs to be. We could re-enable IIA versions 3-5 at an instant and the same IIAs would also be served for these versions again.

We had the same cases with all other versions of IIA-API, with the same IIAs existing across 3+ versions of the API.

[demilatof]

I think she refers to elements that could not be present in version 6 but MUST be present in version 7 (such as mobilities-per-year)

yes, and we would serve exactly that, one v6-version without it and the same iia v7-version with it, internally we may or may not keep the data on all fields, but we would serve it to all versions as it needs to be. We could re-enable IIA versions 3-5 at an instant and the same IIAs would also be served for these versions again.

Just to recap, you suggest to keep on working with version 6 and version 7, with or without an XSLT? How do you suggest to choose between version 6 and 7, in order to call your partner's IIA Get API?

[skishk]

You have IIA in version 6.0.0, it is mutually approved. What will you send in response to IIA get in version 7.0.0?

we need to find a way to stay approved, otherwise every change of version we will have the same situation...

[janinamincer-daszkiewicz]

I assume, that the whole network will switch to 7.0.0 in more or less the same time. I do not suspect that many providers will be willing to support both versions.

When switching from 6.0.0 to 7.0.0 we have to make our IIA object compliant with 7.0.0 which means that fields becoming mandatory will have to get a value.

It would be interesting to know how many IIAs in 6.0.0 do not have value for the number of mobilities per year. If not many may be we can assume that these will have to be re-approved.

May be Jiri is right and we do not to fight so strongly against re-approval?

[skishk]

I assume, that the whole network will switch to 7.0.0 in more or less the same time. I do not suspect that many providers will be willing to support both versions.

When switching from 6.0.0 to 7.0.0 we have to make our IIA object compliant with 7.0.0 which means that fields becoming mandatory will have to get a value.

It would be interesting to know how many IIAs in 6.0.0 do not have value for the number of mobilities per year. If not many may be we can assume that these will have to be re-approved.

May be Jiri is right and we do not to fight so strongly against re-approval?

i think if you have few IIAs is not a problem to re-approve them... if you have hundreds of them... and you know that every change of version you need to do that... it is another work...

I hope no one heve no value for the number of mobilities per year, but if i read well, the actual value of Language-Skills is not required and the proposal is to become mandatory right? that is possible to have no value actually, so yes a lot of IIAs need re-approval.

[janinamincer-daszkiewicz]

but if i read well, the actual value of Language-Skills is not required and the proposal is to become mandatory right? that is possible to have no value actually, so yes a lot of IIAs need re-approval.

That's true :(

[demilatof]

but if i read well, the actual value of Language-Skills is not required and the proposal is to become mandatory right? that is possible to have no value actually, so yes a lot of IIAs need re-approval.

That's true :(

No, this is not really true, unless we receive an explicit disclaimer from "DG EAC" that says "all the mutual approvals that have one or both IIAs without language skill in student mobilities for studies and staff mobilities for teaching are void and need a re-approval". Once approved, an IIA stays approved, and we, as technician, have the duty to remove all technical issues that could impose a re-approval when it is not necessary by the process.

https://github.com/erasmus-without-paper/ewp-specs-api-iias/commit/a4e987b06a44ee71f65c94a338bf2598245c653d

[janinamincer-daszkiewicz]

I have already written to Harpa and Paul with the question.

[janinamincer-daszkiewicz]

Anyway, when moving the objects (IIAs) from version 6.0.0 to 7.0.0 some values will have to be added.

[skishk]

a4e987b

but is written: "At least one element MUST be provided for student-studies-mobility-spec and staff-teacher-mobility-spec elements."

if they remove this comment is solved

[janinamincer-daszkiewicz]

if they remove this comment is solved

This is one of the changes for 7.0.0

[fioravanti-unibo]

Anyway, when moving the objects (IIAs) from version 6.0.0 to 7.0.0 some values will have to be added.

and if we have to add some elements in the cooperation conditions, it seems obvious to me that these coopertation conditions are changed and need to be re-approved

[janinamincer-daszkiewicz]

and if we have to add some elements in the cooperation conditions, it seems obvious to me that these coopertation conditions are changed and need to be re-approved

I agree

[demilatof]

and if we have to add some elements in the cooperation conditions, it seems obvious to me that these coopertation conditions are changed and need to be re-approved

I agree

Not from a legal point of view without a law that declares the previous agreements void

[skishk]

and IROs don't agree to re-approve all IIAs already approved... that is why we are trying to find a way to not re-approve all IIAs

[demilatof]

but is written: "At least one element MUST be provided for student-studies-mobility-spec and staff-teacher-mobility-spec elements."

if they remove this comment is solved

This point is necessary to not split all the codes we have, because:

• CC section is a set of several kinds of mobilities (student-studies-mobility-spec, student-traineeship-mobility-spec, staff-teacher-mobility-spec, staff-training-mobility-spec)
• Only two of them will have the language element mandatory
• BUT all of them are an extension of StudentMobilitySpecification or StaffMobilitySpecification
• THAT in their turn extend MobilitySpecification
• We have the declaration of the language skills at the last level; to make it explicitly mandatory only for two kinds of mobility we have to rewrite all this part (unless XML allows to redefine the value in a subclass)

Nevertheless, if you approved me an IIA when the language skills were not mandatory, until we don't need to change that IIA for other reasons, the language skills don't play any rule (if not explicitly imposed by BPO/DG EAC). As soon as we modify the IIA (e.g., to increase the total-months), the language skills become mandatory.

[demilatof]

I have already written to Harpa and Paul with the question.

I think they are welcomed if they would be so kind to share here their opinions

[janinamincer-daszkiewicz]

I will ask again: what will you send in response to IIA get version 7.0.0 for an approved IIA when there was no value before and there should be some value now? You will send some value. Will the hash change? How should your partner react?

[mkurzydlowski]

I believe that this was the only comment regarding the solution I have propose:

Changing current hash to XSLT means that: everyone has to change their implementation, every hash has to change and all approvals are made invalid?

@demilatof answered:

Not necessarily; anyway what we should try to avoid is to implement something that works only under optimal assumptions.

Why are you saying that it "works only under optimal assumptions"?

Before we change the way hash is calculated (XSLT) could we be more explicit why we are proposing changes?

What is wrong with the current way hash is calculated if after approval we "freeze" the hash?

Being subject to IIA version change is not an issue for unapproved IIAs and it would not be an issue for approved (after we add the "freeze").

Also there is no problem if "business" decides that some IIAs need to be re-approved after IIA version changes and some fields become mandatory. That would mean that HEIs need to treat such changes as modifications to approved IIA - change the hash and send a CNR.

[skishk]

Also there is no problem if "business" decides that some IIAs need to be re-approved after IIA version changes and some fields become mandatory. That would mean that HEIs need to treat such changes as modifications to approved IIA - change the hash and send a CNR.

no problems? do you know what do you mean re-approve hundreds of IIAs for IROs? and repeat it every time we need to change the versione for new fields? did you communicate to IROs that they will have to re-approve all the already approved IIAs? :sweat_smile:

[mkurzydlowski]

I meant no technical problem if this is decided! I don't suggest they need re-approval but I'm only a technical person.

[skishk]

I will ask again: what will you send in response to IIA get version 7.0.0 for an approved IIA when there was no value before and there should be some value now? You will send some value. Will the hash change? How should your partner react?

exactly that is the problem, we probably need some default value for new mandatory fields that means to not consider it in the hash in first days/months , but i don't know if it could be a good idea...

[demilatof]

Why are you saying that it "works only under optimal assumptions"?

Because you are mixing the meaning and the use of the hash code; sometimes it detects and reports changing in the IIA, sometimes not, depending on what the IRO is doing:

• As soon as mutual approval is reached both parties store their hashes (the ones that were approved) and serve them in the IIA GET response instead of calculating them dynamically. This is the last moment the hash needs to be verified by the partner.

• Now if IIA API version changes the hash for all approved IIAs stays the same.

• Only if a partner decides to modify an approved IIA, then the hash is appropriately recalculated and the normal change-approve workflow starts again

This can do the job only if every system understands and implements correctly the same strategy; at that time, if we receive the same hash code, but the IIA is indeed modified, we have no way to check it, if not implementing our internal controls. No way to use the hash code.

What is wrong with the current way hash is calculated if after approval we "freeze" the hash?

Freezing hash is like saying that while we are switching from version 6 to version 7 we relax controls.

Being subject to IIA version change is not an issue for unapproved IIAs and it would not be an issue for approved (after we add the "freeze").

Also there is no problem if "business" decides that some IIAs need to be re-approved after IIA version changes and some fields become mandatory. That would mean that HEIs need to treat such changes as modifications to approved IIA - change the hash and send a CNR.

It's all based on several exceptions depending of what someone decides to do, an you are asking me why I said that it "works only under optimal assumptions"? A good solution should be always valid and should be implemented applying a set of rules that all of us can follow without leaving space for interpretation

[janinamincer-daszkiewicz]

Great, so where is counter-proposal?

[demilatof]

I will ask again: what will you send in response to IIA get version 7.0.0 for an approved IIA when there was no value before and there should be some value now? You will send some value. Will the hash change? How should your partner react?

And I ask you again: what I have to return in response to IIA get version 7.0.0 if I haven't populated my database with required values? Do you impose a version that will break the XML? Are you and DG EAC/BPOs that have to say if we need to set a default value and what.

In my opinion, you have to resolve the compatibility problems in the new version. How? We have several options:

• By a DG EAC's authoritative interpretation: if not else defined, all the mobilities per year must be assumed as one
• Or modifying the version 7 to accept nonNegativeInteger
• Or specifying that for compatibility issue, every mobilities-per-year not specified, must be set to 99999 so that the XSLT can then remove it

Without fixing the above points, we are discussing about all and nothing

[demilatof]

Great, so where is counter-proposal?

It's here https://github.com/erasmus-without-paper/ewp-specs-api-iias/issues/109#issuecomment-1574988634

Who knows a bit the XSLT should know that it is the best way to detach data from a particular version. Personally, I don't like to jump into an XML solution, because I know it hides the problem we are facing. But if I decide to use XML, happy or unhappy, I have to deal with XSLT to solve this kind of problems.

[janinamincer-daszkiewicz]

Are you and DG EAC/BPOs that have to say if we need to set a default value and what.

Not me, definitely :)

May be partners should agree on that, which means that re-approval makes sense.

It's here https://github.com/erasmus-without-paper/ewp-specs-api-iias/issues/109#issuecomment-1574988634

Too many open issues to consider it for voting. Also I do not see the common agreement that this is a way to go.

[mkurzydlowski]

This can do the job only if every system understands and implements correctly the same strategy; at that time, if we receive the same hash code, but the IIA is indeed modified, we have no way to check it, if not implementing our internal controls. No way to use the hash code.

If the hash is unchanged then there is no point in checking if the IIA is unmodified. If the partner wants to modify an approved IIA then he has to change the hash. There is nothing wrong about it.

Freezing hash is like saying that while we are switching from version 6 to version 7 we relax controls.

We don't need to "control" an IIA after it has been approved but not modified (hash not changed). Also nothing wrong about it.

It's all based on several exceptions depending of what someone decides to do, an you are asking me why I said that it "works only under optimal assumptions"? A good solution should be always valid and should be implemented applying a set of rules that all of us can follow without leaving space for interpretation

This solution doesn't depend on any exceptions. I only said that if the business people decide that for some situations the IIA hash to be re-approved after IIA version change then the HEI has to proceed with a standard "modification after approval" workflow.

There isn't really any space for misinterpretation.

[mkurzydlowski]

PS. Aren't we still susceptible to XML format changes (whitespaces) even if we use XSLT?

[demilatof]

Not me, definitely :)

You or who writes the specifications; we depend from that, we could even not know if the specifications are written with or without DG EAC/BPO's opinions on them

It's here #109 (comment)

Too many open issues to consider it for voting. Also I do not see the common agreement that this is a way to go.

We cannot have an easy solution for a complex problem; could you list the open issues you see? I'm trying to recap:

• Accept zero/empty values for mandatory fields
• Define two different XSLT, one for version 6 and one for version 7 (XSLTv6 and XSLTv7)
• If you want the hash code for your IIA version 6 use XSLTv6, if you want the hash code for your IIA XSLTv7 use XSLTv7
• Insert in your IIAs the above hash code
• Compare them as usual, now they are compatible

We can have a clean solution to the mandatory fields and the hash code introducing an optional attribute "not-yet-defined" to the new mandatory elements, so that you can put there whatever you want and be sure that it will be ignored.

• If it is "true", it is set to zero by XSLT and the IIA cannot be re-approved until it is present.
• If it is "false" or not present, the XSLT keeps the element with the original value and the IIA can be approved.

This new attribute is not set by a human operator, it's your code that triggers it to true whenever it is asked for an element that until now it cannot find in your DB (a null value, I suppose)

[demilatof]

If the hash is unchanged then there is no point in checking if the IIA is unmodified. If the partner wants to modify an approved IIA then he has to change the hash. There is nothing wrong about it.

Really? And how you manage a contact change? Currently, if I receive an IIA with the same hash code, I'm free to replace my old one IIA to update the fields out of approval. If you keep the hash code and you send me a changed IIA, because you don't want to change anything, but you have to consider new required element, how can I verify this? By definition, the hash code triggers a particular behavior when something changes

We don't need to "control" an IIA after it has been approved but not modified (hash not changed)

You are saying that it is not changed, but I don't trust you and I want to verify your hash. How can I do? It's exactly what I said before, you look always at the best case.

This solution doesn't depend on any exceptions. I only said that if the business people decide that for some situations the IIA hash to be re-approved after IIA version change

The exceptions are inside everything the business people don't list in "some situations". The exceptions are in the nested IFs that we need to model this kind of approach (If I have not changed this, and this one, but this is so and not so... then the hash code is the same).

Anyway, how do you answer to @janinamincer-daszkiewicz question?

I will ask again: what will you send in response to IIA get version 7.0.0 for an approved IIA when there was no value before and there should be some value now? You will send some value. Will the hash change? How should your partner react?

[mkurzydlowski]

Currently, if I receive an IIA with the same hash code, I'm free to replace my old one IIA to update the fields out of approval.

I don't understand that sentence.

If you keep the hash code and you send me a changed IIA, because you don't want to change anything, but you have to consider new required element, how can I verify this?

Why do you need to verify this? If the hash in unchanged then the partner does not want to modify the approved IIA. No point in verifying anything.

You are saying that it is not changed, but I don't trust you and I want to verify your hash. How can I do?

There is no point in verifying anything if the hash is unchanged because the partner is not modifying the approved IIA according to spec. The use case you are describing is nonexistent.

The exceptions are in the nested IFs that we need to model this kind of approach (If I have not changed this, and this one, but this is so and not so... then the hash code is the same).

I don't see this decision as an IF in the code. If it is decided that some IIAs need re-approval then this is something the IROs will need to issue by modifying the IIAs appropriately but we are discussing points that haven't been decided yet.

[mkurzydlowski]

Concerning the open issues of XSLT I see:

• Do we assume that we are able to always (now and in the foreseeable future) provide an XSLT that make hash indepent of every IIA version change? I don't believe that we can state that.
• Is the hash stable even if the XML format changes due to white spaces?
• Are we sure that every provider is willing to make changes in their software and incorporate XSLT transformations?
• Are the results of those transformations the same for every language or library - for example white space or special character treatment?

And finally is it worth it comparing to a simpler solution?

[mkurzydlowski]

I'm not trying to say that there is something fundamentally wrong with the XSLT solution but I'm not convinced that we really need to go that way and that we are aware of the possible drawbacks. We probably weren't aware of some when we introduced the current hash - and that started this issue years after...

[georgschermann]

Since modifying an approved IIA becomes a regular use case, is it necessary to prevent hash changes due to new versions? Providers can internaly check if the hashes were for different IIA versions and could adapt the GUI accordingly. Universities are not forced to re-approve anything they don't want to, if they have 2000 iia-version-changed-please-check notifications in theiry system - they can ignore - is this a problem?

[skishk]

Since modifying an approved IIA becomes a regular use case, is it necessary to prevent hash changes due to new versions? Providers can internaly check if the hashes were for different IIA versions and could adapt the GUI accordingly. Universities are not forced to re-approve anything they don't want to, if they have 2000 iia-version-changed-please-check notifications in theiry system - they can ignore - is this a problem?

so we are introducing the history of every IIA? we are not working on the last version of the object?! because the first specification about IIA say no history of IIA and working on last version of it. and there is no way to say to the partner I refuse your change so go back to previews version in EWP.

[demilatof]

Currently, if I receive an IIA with the same hash code, I'm free to replace my old one IIA to update the fields out of approval.

I don't understand that sentence.

If you keep the hash code and you send me a changed IIA, because you don't want to change anything, but you have to consider new required element, how can I verify this?

Why do you need to verify this? If the hash in unchanged then the partner does not want to modify the approved IIA. No point in verifying anything.

Maybe you miss a point: if an IIA doesn't need a new approval, I store it anyway to update the elements that could be updated, such as sending/receiving contacts. Instead, with your approach, I will have to update my DB in a blind fashion just because you said that the hash code is not changed and that I have to trust you.

Otherwise, you're saying that when there is a version change we cannot anymore update data that are outside the hash code, such as contacts or signers.

I don't see this decision as an IF in the code. If it is decided that some IIAs need re-approval then this is something the IROs will need to issue by modifying the IIAs appropriately but we are discussing points that haven't been decided yet.

Then I think we have to suspend any discussion, because before we have to know the business requirement and only after we can discuss the best technical approach. Anyway, this doesn't solve the problem of updating fields that are outside the approval

[mkurzydlowski]

Why is updating fields outside CC a problem in the solution I proposed? How that is concern hash?

[demilatof]

Why is updating fields outside CC a problem in the solution I proposed? How that is concern hash?

Because I've no way to verify that the CCs are not really changed; in a world where the hash code changes with CC, once I've checked that the hash is valid and not changed, I can store the whole data, without looking for only the changed data. If the hash code represents only your will, I've to add several checks to update not relevant data.

I don't know how many others could have this problem, but I think it's real

[demilatof]

Concerning the open issues of XSLT I see:

* Do we assume that we are able to always (now and in the foreseeable future) provide an XSLT that make hash indepent of every IIA version change? I don't believe that we can state that.

I think so; not only one XSLT, but an XSLT per API version

* Is the hash stable even if the XML format changes due to white spaces?

The XSLT can manage this issue (e.g. removing all the spaces between the elements). Nothing can be done for spaces inside an element, but this outside the scope

* Are we sure that every provider is willing to make changes in their software and incorporate XSLT transformations?

Are you sure that every provider is willing to make changes as you described?

I'm afraid you don't fully know the power of the XSLT and how simple it could be to use it (not to design it). The XSLT must be the same for all, a provider must receive it, not write it; a provider is required to add just ten lines of code to transform an XML into another, then everything is the same. Do you really think that ten lines of code are too much?

* Are the results of those transformations the same for every language or library - for example white space or special character treatment?

Did you ask this question when you write about the canonicalization of the XML before computing the hash code? Anyway, the XSLT explicitly says how an XML must be produced, therefore a library or a language that doesn't fulfill its instruction has something wrong. XLST could have been used even in the initial hash code.

And finally is it worth it comparing to a simpler solution?

Are you sure you find this a complex solution just because you're not comfortable with XSLT?

[mkurzydlowski]

Updating fields outside CC by copying all of the IIA fields would not work, as you pointed out. Implementers who do it that way would need to change to copying only the updated section.

Maybe you are right that it's me that finds it complex. Everyone needs to judge it themself. Still the complexity must be evaluated by comparing it to the current solution.

[mkurzydlowski]

Getting back to: https://github.com/erasmus-without-paper/ewp-specs-api-iias/issues/109#issuecomment-1574988634

You assume that UV should look like v7 but that would only work this time, not after we introduce v8.

If more fields will be introduced, changed, removed in v8 then UV will not be able to accommodate that and the hash will not be stable.

I understand that for the hash to be stable we would always need a vX -> vX-1 transformation (as UV must be based on v6 or v7) but that would not be possible for all the possible changes we might introduce in XSD schema in future IIA API versions.

Please correct me if I'm looking at it from the wrong perspective.