search

erasmus-without-paper / ewp-specs-api-iias

Specifications of EWP's Interinstitutional Agreements API.
MIT License
4 stars 13 forks source link

IIA version change impact on cooperation conditions hash #109

Closed mkurzydlowski closed 1 year ago

mkurzydlowski commented 1 year ago

As we already know IIA major version change impacts the cooperation conditions hash, as it changes the XML namespace that is part of the cooperation conditions XML element being hashed. So even if a IIA version change doesn't impact the fields in this element, the hash changes.

Let's discuss how this case should be handled and what impact this has.

Should the cooperation conditions hash change after version change or should it be stored and stable until IIA changes? If we choose to recalculate hashes then should we send CNRs? If the partner chooses to recalculate hashes, should we recalculate IIA approvals?

mkurzydlowski commented 1 year ago

@BavoNootaert, after IIA version changes an approved IIA GET response changes and so should the CC hash change (currently). If it changes then from the partner's point of view there is a mismatch between what has been approved and what is currently being served. That's why we propose not changing the CC hash in such case by storing it rather then calculating on demand.

demilatof commented 1 year ago

The application should respond to IIA Get any time, also after approval. even in case when CNR has not been sent by the application before. What namespace should be used in such response?

With a bit of logic in both of the internal systems, if we want, we can manage this problem. The trick is to ignore the API version in IIA Get; we should be able to recognize it from the fetched XML.

When we receive an IIA Get, we know If we have previously sent an IIA CNR to the partner (that is, we know if we have a new version or not of the IIA we need to expose) Flowchart scelta namespace

When the partner fetches our XML, he computes the hash code in blind fashion; indeed, if he use something similar to my code, it is DOM that read the namespace from the XML code. Once he realized if the IIA is new or not, he can choose if he need to parse it with API v6 or v7 Flowchart scelta namespace ricevuto

janinamincer-daszkiewicz commented 1 year ago

With a bit of logic in both of the internal systems, if we want, we can manage this problem.

Do you suggest that instead of what was proposed at IF, we should recommend such implementation to every provider?

demilatof commented 1 year ago

Do you suggest that instead of what was proposed at IF, we should recommend such implementation to every provider?

I'm in favor of the new hash code as version number, but there are anyway some problems to manage when we change the namespace version (e.g. new fields to populate). The above solution could be more conservative; I don't know if it could compatible with the most of the implementations and if it's easy to understand or not. If you think that it could be an alternative, we can discuss here on GitHub to understand if it has hidden problems and if other providers are interested in, before proposing it at IF

janinamincer-daszkiewicz commented 1 year ago

I am in favour of my proposal, which seems to me more straightforward and also as a side affect solving the hash problem once and for all. I do not see a strong risk of problems with populating new fields. I do not see 'strong objection' against it so for the time being plan to continue with this proposal. Your proposal may work but I suspect that may be a source of many wrong implementations, if not handled with due care and attention. I personally would prefer not to have to parse the old XMLs or keep track of get requests.

demilatof commented 1 year ago

I am in favour of my proposal, which seems to me more straightforward and also as a side affect solving the hash problem once and for all. ... Your proposal may work but I suspect that may be a source of many wrong implementations,

Maybe, but even your proposal is good only if there are no wrong implementations. If you change the version number when there are "important" changes, I can realize that there is a new IIA to evaluate. But what happens if you don't change your version number when instead you change the cooperation conditions in your IIA? I see two possibilities:

  1. I don't notice any change and I ignore the IIA
  2. I don't notice any change and I simply update the IIA thinking to update only secondary fields (e.g. sending/receiving contact)

The first point has few problems, whilst the second one can even bring myself to give a new approval without I can realize it. Therefore, in your proposal you should point out that every provider must implement its own algorithm to detect if the received version is changed or not, reminding that the current hash calculations is not useful if we have a namespace upgrade from v6 to v7.

Without this algorithm, we have no mode to understand if the version number is correctly used whilst, using the hash code, with all its limits, we can detect changes both on the sending side and on receiving one

demilatof commented 1 year ago

I'm reading the slides for the next IF. I see: approvalFullXmlProve

I'm quite sure that the word "prove" is not the best one in the sentence "After that happens A is going to be able to prove the exact IIA data being approved by B". A proof is something more robust; I think that it would be more correct using the term "recognize", because the presence of the whole XML in the approval let A recognize immediately what B claims to have approved. Indeed to prove something, A should be able to demonstrate that this information has been produced by B and not tampered by itself. Presently EWP doesn't provide any way to demonstrate that the stored approval has been produced by B.

mkurzydlowski commented 1 year ago

Please take a look at the suggested list of changes: https://github.com/erasmus-without-paper/ewp-specs-api-iias/compare/stable-v7...hash-as-version https://github.com/erasmus-without-paper/ewp-specs-api-iias-approval/compare/stable-v2...hash-as-version

demilatof commented 1 year ago

Please take a look at the suggested list of changes: I took a look stable-v7...hash-as-version

I read "It MUST change when a cooperation-conditions element changes". Haven't we said that it should change even when we have a change in the mapping?

E.g.: I have an IIA with my IIA ID A1 and partner's IIA ID B1. Even if I change the mapping from B1 to B2 in my A1 IIA, my cooperation conditions are still the same, but there is a relevant change. Therefore, in my opinion, the version should change to trigger partner's controls.

mkurzydlowski commented 1 year ago

This change wasn't about the mapping issue, that's why this fragment doesn't mention mapping.

demilatof commented 1 year ago

This change wasn't about the mapping issue, that's why this fragment doesn't mention mapping.

Anyway, it describes what is a hash as a version number, and I think that if we plan to use it for mapping (or simply, remove mapping) it could be better to add it now, and not later. We shouldn't add and remove a specification from time to time if we already know what we'll need in a short time.

janinamincer-daszkiewicz commented 1 year ago

This is not yet official specification, just a branch and request for comments :)

demilatof commented 1 year ago

This is not yet official specification, just a branch and request for comments :)

Of course, and my comment is that we should keep into account the mapping in the version number :wink:

mkurzydlowski commented 1 year ago

I was just suggesting using different commits for different changes. I wasn't suggesting that the final release would not contain a relation between hash and mapping.

demilatof commented 1 year ago

I was just suggesting using different commits for different changes.

I understand, even if I think that it could better discuss about an issue (version ID) as a whole, so that once accepted we have not the risk to reopen a closed discussion because someone agreed to the new hash code for cooperation conditions but not for mapping. We have already seen this problem for termination (as a whole was introduced later and could have changed the approach to termination if discussed previously) and for deletion (only for not mutual approved IIAs, then a little try even for mutual approved IIAs, then back to the first solution). Too much time that can be saved.

janinamincer-daszkiewicz commented 1 year ago

The whole set of changes (after decision in https://github.com/erasmus-without-paper/ewp-specs-api-iias/issues/101):

demilatof commented 1 year ago

The whole set of changes (after decision in #101):

* [erasmus-without-paper/ewp-specs-api-iias-approval@9bfb404](https://github.com/erasmus-without-paper/ewp-specs-api-iias-approval/commit/9bfb4046ba9851c636a4be3b3b4eb408304a4f66)

* [erasmus-without-paper/ewp-specs-api-iias-approval@118b162](https://github.com/erasmus-without-paper/ewp-specs-api-iias-approval/commit/118b1621d71ff2de5f62c8d84973174e19fe62a0)

* [erasmus-without-paper/ewp-specs-api-iias-approval@25f1996](https://github.com/erasmus-without-paper/ewp-specs-api-iias-approval/commit/25f19966c82076b2008499fb8bcd6de1e8ffe133)

* [erasmus-without-paper/ewp-specs-api-iias-approval@50ee4bb](https://github.com/erasmus-without-paper/ewp-specs-api-iias-approval/commit/50ee4bb974aff8759b2488e7c1c18c5e1fb31fea)

* [9ff2868](https://github.com/erasmus-without-paper/ewp-specs-api-iias/commit/9ff2868220365c5e370d5485b5f1ca01e9ffd004)

I don't remember we have voted to accept:

  1. Hash as version ID ONLY for cooperation conditions (we may need it after we change the mapping, but if you say uniquely identifying a cooperation condition version yo exclude that we can use it for other controls)
  2. "HTTP signature server authorization to ensure non-repudiation"

We haven't either discussed the second point on GitHub to understand if such a solution is commonly accepted and recognized as a proof. What you stated to be changed, for me might be only proposals of changes.

mkurzydlowski commented 1 year ago

@demilatof, so you suggest that this fragment:

It MUST also change when partner IIA id (partner/iia-id element) changes.

That will be added in this change is not enough and we should also modify the first sentence of the description?

demilatof commented 1 year ago

@demilatof, so you suggest that this fragment:

It MUST also change when partner IIA id (partner/iia-id element) changes.

That will be added in this change is not enough and we should also modify the first sentence of the description?

I think that specifications must be coherent; it makes confusion if in a sentence we have that an ID is mandatory for a task and in another part we see that the same ID may be useful for something else. So that, we should have something similar:

The Version ID changes when:

  1. ....
  2. ....
  3. and so on

The specification on the version ID must be on version ID, not collected from its uses

mkurzydlowski commented 1 year ago

The specification on the version ID must be on version ID, not collected from its uses The sentence I was quoting is added to the cooperation hash description.

I think that the first sentence is not misleading as it refers to the element name and its location in the XML.

Both cases when hash must change are already listed one after the other. We might discuss changing two sentences into an enumeration but I don't see any real problem with the current proposal.

demilatof commented 1 year ago

Both cases when hash must change are already listed one after the other. We might discuss changing two sentences into an enumeration but I don't see any real problem with the current proposal.

As you like; during the infrastructure forum I realized that some participants were considering the version ID only for cooperation condition, and this is the consequence, in my opinion, of the fact that it has been represented only as a substitute for the hash code. So we have to make clear to everyone what is a version ID:

  1. Something that changes only with cooperation-condition?
  2. Something that changes whenever we need a new approval? We should list all the changes that trigger a new approval.

Currently the proposal is https://github.com/erasmus-without-paper/ewp-specs-api-iias-approval/commit/9bfb4046ba9851c636a4be3b3b4eb408304a4f66 :

A hash or other ID uniquely identifying a cooperation conditions version that is being approved.

So we are interested only in the "cooperation conditions version". But I recall that until now the "cooperation conditions" were the only that could trigger a new approval, whilst recently we introduced new conditions, such as changes in mapping.

umesh-qs commented 1 year ago

Both cases when hash must change are already listed one after the other. We might discuss changing two sentences into an enumeration but I don't see any real problem with the current proposal.

As you like; during the infrastructure forum I realized that some participants were considering the version ID only for cooperation condition, and this is the consequence, in my opinion, of the fact that it has been represented only as a substitute for the hash code. So we have to make clear to everyone what is a version ID:

  1. Something that changes only with cooperation-condition?
  2. Something that changes whenever we need a new approval? We should list all the changes that trigger a new approval.

Currently the proposal is erasmus-without-paper/ewp-specs-api-iias-approval@9bfb404 :

A hash or other ID uniquely identifying a cooperation conditions version that is being approved.

So we are interested only in the "cooperation conditions version". But I recall that until now the "cooperation conditions" were the only that could trigger a new approval, whilst recently we introduced new conditions, such as changes in mapping.

@demilatof purpose of calling hash as version ID was only to make sure that the re-approval is not triggered because of cosmetic changes. Hash/Version ID should change only when re-approval is needed. And list of the fields that need re-approval should be mentioned

demilatof commented 1 year ago

@demilatof purpose of calling hash as version ID was only to make sure that the re-approval is not triggered because of cosmetic changes. Hash/Version ID should change only when re-approval is needed. And list of the fields that need re-approval should be mentioned

I agree, but the change in mapping requires a new approval, as we saw during the last Infrastructure Forum. This is the reason why I'm asking for a list of events that change the version ID (or, as you say, that need re-approval). In my opinion is chaotic a specification where we have to build that list from several points (we can miss the warning)

ChangeInMapping

umesh-qs commented 1 year ago

@demilatof purpose of calling hash as version ID was only to make sure that the re-approval is not triggered because of cosmetic changes. Hash/Version ID should change only when re-approval is needed. And list of the fields that need re-approval should be mentioned

I agree, but the change in mapping requires a new approval, as we saw during the last Infrastructure Forum. This is the reason why I'm asking for a list of events that change the version ID (or, as you say, that need re-approval). In my opinion is chaotic a specification where we have to build that list from several points (we can miss the warning)

ChangeInMapping

Agreed. Specification description should mention the fields to include in version/hash calculation. Proposed description "A hash or other ID uniquely identifying a cooperation conditions version that is being approved" is too generic.

Also the proposed change say "A hash or other ID uniquely identifying", but forces "ewp:Sha256Hex" which seems contradictory.

mkurzydlowski commented 1 year ago

I will try to find a less generic description that still fits the element name but it just occurred to me that we should probably change the element name too. Do you agree?

The type has changed to xs:string in the proposal. Maybe you missed that?

demilatof commented 1 year ago

it just occurred to me that we should probably change the element name too. Do you agree?

I agree; it could be simply "VERSION-ID"

mkurzydlowski commented 1 year ago

Please review this change proposal: https://github.com/erasmus-without-paper/ewp-specs-api-iias/commit/a67e2a9bfb6bb1563d86e23062c6ea95233a7726 https://github.com/erasmus-without-paper/ewp-specs-api-iias-approval/commit/b8d0e7273e96a632eb33d57d35866c4fe0dff03f

mkurzydlowski commented 1 year ago

Please review this set of changes: https://github.com/erasmus-without-paper/ewp-specs-api-iias/compare/stable-v7...hash-as-version https://github.com/erasmus-without-paper/ewp-specs-api-iias-approval/compare/stable-v2...hash-as-version

mkurzydlowski commented 1 year ago

How should we use it as a proof? Could you make an example how could it be used as a proof?

@demilatof, I can only answer it from the technical perspective:

Storing both API responses with the HTTP signatures present in the response headers gives a proof that the given version of IIA has been approved by the partner.

So it is possible to undeniably show what exact data (IIA) has been approved by the partner.

Depending on the task we have to perform to check that proof, we may need further specifications/rules

The same library used to produce HTTP signature might be used to verify the stored responses (Approval, IIA file).

If the signer changed keys in the catalog after he used them for signing then the old keys might be found in the catalog log: https://github.com/erasmus-without-paper/ewp-registry-log-prod

mkurzydlowski commented 1 year ago

approved-iia-file ?!?! are we leaving the way of exchange data to exchange files?

@skishk, we need to exchange an XML file with the IIA GET response and decided to use the new File API for every file exchange in the EWP network. Do you see a problem here?

demilatof commented 1 year ago

So it is possible to undeniably show what exact data (IIA) has been approved by the partner.

Are we sure that the exact data approved by the partner is the same we have sent? Are we talking about data or file?

skishk commented 1 year ago

Do you see a problem here?

sincerely? :sweat_smile: Yes

I can't understand why we are going to use an XML file containing something that is specified with the iia-id and version.

what this XML file is going to solve? i can put inside it everything i want (i can fill it with a wrong iia, i'm not the owner of te information) so it has no value for the partner that receive it.

Probably you can explain me something i can't understand in this new implementation.

new File API is needed for?

what is the main content of the information that we are exchanging the content of the IIA-GET-RESPONSE or Files?

fioravanti-unibo commented 1 year ago

approved-iia-file ?!?! are we leaving the way of exchange data to exchange files?

@skishk, we need to exchange an XML file with the IIA GET response and decided to use the new File API for every file exchange in the EWP network. Do you see a problem here?

Wrong. The IIA Get Response is not a file, is only an XML response. A file MUST be served as an attachment and qualified by a file name, as in:

Content-Disposition: attachment
Content-Disposition: attachment; filename="filename.jpg"
mkurzydlowski commented 1 year ago

Are we sure that the exact data approved by the partner is the same we have sent?

We need to compare to be sure.

Are we talking about data or file?

As pointed out by @fioravanti-unibo we should be naming it "response" as the word "file" might suggest that it has a name.

I can't understand why we are going to use an XML file containing something that is specified with the iia-id and version.

Because there is no verifiable link between those ID's and the data that is being approved.

what this XML file is going to solve? i can put inside it everything i want (i can fill it with a wrong iia, i'm not the owner of te information) so it has no value for the partner that receive it.

Responding with fake data would make the approval invalid, of course. As would any other fake data in the response (bad version ID, etc.). I don't see how this can be used as an argument.

new File API is needed for?

It is used to serve unstructured data. We use it here to send an IIA GET response XML - no restrictions on the namespace version used. This response is stored by the partner as text.

what is the main content of the information that we are exchanging the content of the IIA-GET-RESPONSE or Files?

We are serving a stored content of the IIA GET response.

skishk commented 1 year ago

Because there is no verifiable link between those ID's and the data that is being approved.

so the partner receive the text of the content in any version and he will compare it with his own IIA that could be the old version and not the actual? and now probably have a new XSD template for version 7 and the approval was made in with the version 6? so every one need to store all history of IIA?

demilatof commented 1 year ago

Are we sure that the exact data approved by the partner is the same we have sent?

We need to compare to be sure.

Are we talking about data or file?

As pointed out by @fioravanti-unibo we should be naming it "response" as the word "file" might suggest that it has a name.

I'll try to be more explicit, because I don't find your answer doesn't help me yet to understand what I have to do. Should I compare data inside the "file" or the file by itself, as an exact sequence of bytes?

In the first case I need to know how to read the response (API version), and therefore I have to manage all the versions from now on. In the latter case I have to be sure that after I send my response to you, you stored it exactly as I sent, without filter, gateway, or whatever we can have in the middle.

Responding with fake data would make the approval invalid, of course

And once we said that the approval is invalid, what can we do? The approval happens when I say that I approve your IIA sending you an Approval CNR; the Approval API is to confirm you later what I have approved. But if you don't recognize what I have approved, what can we do by specifications? They never talk about an invalid approval and how to manage it. Have I to remove the approval flag from my implementation? Just because you are sending me a "file" that may be different due to your management?

new File API is needed for?

It is used to serve unstructured data. We use it here to send an IIA GET response XML - no restrictions on the namespace version used. This response is stored by the partner as text.

what is the main content of the information that we are exchanging the content of the IIA-GET-RESPONSE or Files?

We are serving a stored content of the IIA GET response.

I'm losing myself: we use it to serve unstructured data and to serve the content of an IIA GET response, that is a structured XML file. It sounds a bit strange; again, my question is if are we interested in the data as a whole or in the content, that we have to know how to read?

demilatof commented 1 year ago

As pointed out by @fioravanti-unibo we should be naming it "response" as the word "file" might suggest that it has a name.

I'm not sure that the problem is with the missing file name, but with the fact that the response is never served neither managed as a file, that is as a document. Whilst if it is now returned as a file, we give no more importance to its content but to it as a sequence of bytes, that is a file or document.

mkurzydlowski commented 1 year ago

so the partner receive the text of the content in any version and he will compare it with his own IIA that could be the old version and not the actual? and now probably have a new XSD template for version 7 and the approval was made in with the version 6?

You probably misunderstood me here:

Are we sure that the exact data approved by the partner is the same we have sent?

We need to compare to be sure.

What I meant is that if you want to check you might compare the approved XML with the IIA GET response but such comparison is done the moment you receive an approval. Doing it after some time would probably be impossible due to IIA data or version change.

so every one need to store all history of IIA?

There is no need to.

Should I compare data inside the "file" or the file by itself, as an exact sequence of bytes? In the first case I need to know how to read the response (API version), and therefore I have to manage all the versions from now on. In the latter case I have to be sure that after I send my response to you, you stored it exactly as I sent, without filter, gateway, or whatever we can have in the middle.

You should compare it as a text - without parsing it as an XML. The response content should not be modified by the middleware as it would invalidate the HTTP signature.

And once we said that the approval is invalid, what can we do?

I'll try to answer but this is a different topic. Adding XML to the approval doesn't change things in this regard.

The approval happens when I say that I approve your IIA sending you an Approval CNR; the Approval API is to confirm you later what I have approved. But if you don't recognize what I have approved, what can we do by specifications? They never talk about an invalid approval and how to manage it.

As with every GET API in the EWP network, there is no other way to say you suspect an error on the partner side than report it to the EWP monitoring portal and/or make a call.

Have I to remove the approval flag from my implementation?

What do you mean?

Just because you are sending me a "file" that may be different due to your management?

What do you mean "different due to your management"?

I'm losing myself: we use it to serve unstructured data and to serve the content of an IIA GET response, that is a structured XML file. It sounds a bit strange; again, my question is if are we interested in the data as a whole or in the content, that we have to know how to read?

We don't parse it or treat it as a structured data - just a text. We probably won't be able to parse it in the future when IIA version changes.

I'm not sure that the problem is with the missing file name, but with the fact that the response is never served neither managed as a file, that is as a document. Whilst if it is now returned as a file, we give no more importance to its content but to it as a sequence of bytes, that is a file or document.

That's right, we will treat it as a text.

demilatof commented 1 year ago

Now that I better understood what should I do, I think that adding the XML approved as a file only complicates the model, adds overload and solves nothing.

What I meant is that if you want to check you might compare the approved XML with the IIA GET response but such comparison is done the moment you receive an approval. Doing it after some time would probably be impossible due to IIA data or version change.

I think this is not what we aimed when we moved from the hash code to the version ID. Even the hash code is good in a short time

You should compare it as a text - without parsing it as an XML.

If we have to compare it as a text we don't need the full XML, we can simply compute the hash code of the whole response and send/store that hash code.

The response content should not be modified by the middleware as it would invalidate the HTTP signature.

May be I'm wrong, but the HTTP signature can ensure that a response arrives from a given server and that there is no man in the middle. But in a more complex data center, are we sure that behind the server that signs the HTTP response, the response arrives unmodified from the producer (e.g. internal application server) to the web server that implements the security and sign the response that goes to the partner's server?

Anyway, the proposal is to approve an IIA considering its data, but check it later as if we gave the approval to the IIA document. The comparison of the two responses (produced and received) cannot give us any warranty if we mixed data management with document management; we have a distributed and asynchronous system that complicates all. For example:

Have I to remove the approval flag from my implementation?

What do you mean?

If I call your Approval API and I see that your XML file is different from mine, have I to consider your approval invalid? What is the real effect of this? I filed my IIA as approved by you a week ago (for example), and now is it no more approved? Have I to reset the approval in my system?

Just because you are sending me a "file" that may be different due to your management?

What do you mean "different due to your management"?

You may have added, by mistake, an empty space to the file, or as I've just said above, you might have saved my IIA before I changed an element not relevant to the approval.

We don't parse it or treat it as a structured data - just a text. We probably won't be able to parse it in the future when IIA version changes.

We approved the IIA structured data, not the IIA document. That's all.

janinamincer-daszkiewicz commented 1 year ago

I send you an approval CNR after I've fetched your IIA via IIA Get and then I store it You miss my Approval CNR and so you don't know I've already approved your IIA Then you modify an element not essential to the approval (e.g. you add a space in a signer's name)

After that you should call IIA CNR and avoid misunderstanding in the further steps.

janinamincer-daszkiewicz commented 1 year ago

The two responses, compared as text file, are different, you consider invalid my approval because I gave it to you evaluating IIA data, whilst you have checked it as a document.

You can compare it any way you want. What is important is that in the Approval we get the whole set of information concerning approval - what exactly has been approved (on both sides). We know that qualified signatures would be the best but we do not want to introduce them. Sending just a version-id or hash is not enough. What we approve is the content of the partner's copy of IIA and this is what we want to say in the Approval get.

If you prefer to send in the Approval get partner's IIA exactly the samy way we get it from IIA get (text in base64, because we cannot embed XML in XML) - fine, let's do it that way.

demilatof commented 1 year ago

I send you an approval CNR after I've fetched your IIA via IIA Get and then I store it You miss my Approval CNR and so you don't know I've already approved your IIA Then you modify an element not essential to the approval (e.g. you add a space in a signer's name)

After that you should call IIA CNR and avoid misunderstanding in the further steps.

If you consider only the best scenario, ok. If you wish to create a system always valid for 5.000 HEIs not.

  1. The IIA CNR could be lost, and then?
  2. Who receive the IIA CNR see that the version-ID is the same, the cooperation conditions are the same and therefore discard the response without saving it. And then?

For the data approval we need only a set of element, in whatever order now, after we moved from the hash code to the version id. For the check of the approval you want that we use the whole response, as a whole.

You can compare it any way you want.

You could compare the content only if we accept to keep on using all versions for ever.

What is important is that in the Approval we get the whole set of information concerning approval - what exactly has been approved (on both sides).

No, what is important is that we can read that set of information, that is tied to a given version.

Sending just a version-id or hash is not enough.

If, as @mkurzydlowski said, we have to compare the two responses as a file, we never inspect the inside. We compute the two hashes and then we compare them. How can you say that comparing the two hash codes is not enough, if at the end is what we have to do? Do you hope that someone, a human I mean, reads and compares them?

What we approve is the content of the partner's copy of IIA and this is what we want to say in the Approval get.

As you say, we approve the content of the partner's copy of the IIA whilst in the Approval get we say that we approved the IIA file. It can still contain the right content, but it can be different from what have stored the sender. So, basing on it, the approval should be considered invalid whilst it is still perfectly valid.

mkurzydlowski commented 1 year ago

We could exchange the Digest header of the IIA GET response instead of the whole response but still every implementation also needs to store the corresponding XML to be able to prove what the data was.

janinamincer-daszkiewicz commented 1 year ago

There are pros and cons of every approach. Let's put some order in this discussion. Lets's start from the beginning. What should be sent in the IIA Approval get to let the partner know what exactly do we approve:

  1. metadata of our copy (iia-id, version-id)
  2. metadata of partner's copy being approved (iia-id, version-id)
  3. partner's copy of IIA

In our opinion, as explained earlier, to get some kind of repudiation we need (3). We can send this copy in various formats: file, text, PDF. So the options are: resign from (3) and loose repudiation, choose the most convenient format. Do you see other options?

skishk commented 1 year ago

3. partner's copy of IIA

personally i don't see it as an option.

and in all these changes for new IIA version API 7 I can't evaluate a solution for that

georgschermann commented 1 year ago

Since the IF today showed that switching away from the hash and trying to keep the provability introduces a cascade of new issues for some providers we propose the following:

Stay with the hash, don't change the XSD Define a namespace un-aware hash calculation (e.g. using a XSLT template), stripping out namespaces, white spaces, irrelevant elements, probably non ascii characters to prevent encoding issues, etc. Probably with a shared reference implementation in Java or other languages.

Alternatively remove hash an use the http sig digest instead in the approval, but I think this would be inferior to the above.

Adding the full xml to approval is not a feasible solution I think.

skishk commented 1 year ago

using a XSLT template

we agree with that!

fioravanti-unibo commented 1 year ago

Define a namespace un-aware hash calculation (e.g. using a XSLT template), stripping out namespaces, white spaces, irrelevant elements, probably non ascii characters to prevent encoding issues, etc. Probably with a shared reference implementation in Java or other languages. [...] Adding the full xml to approval is not a feasible solution I think.

Agree, adding the XML in the approval adds only complexity and resolves only a very small set of cases; using an XSLT to generate version-agnostic text on which calculate an hash seems a good idea to us: we were talking about this option last week.

BTW I would keep the version number, because is clear to operators of the system; to me the correct implementation may be based on a version number AND a version-agnostic hash.

demilatof commented 1 year ago

Define a namespace un-aware hash calculation (e.g. using a XSLT template), stripping out namespaces, white spaces, irrelevant elements, probably non ascii characters to prevent encoding issues, etc.

I agree, I was thinking about this solution. Should we share the result of the transformation or not?

It could even be used to solve (partially) this issue. If we can compute the hash code of every cooperation condition, from one time to another we can easily identify the added, removed or changed cooperation conditions (of course, we can do this as an internal solution too)

mkurzydlowski commented 1 year ago

What about changes to the IIA structure (new API version)? That would change the hash even when using custom XSLT transformation.