Closed janinamincer-daszkiewicz closed 9 months ago
Assumptions
Deletion of IIA – impact on the deleting party A
Deletion of IIA – impact on the partner B
Deletion of IIA – impact on IIA Approval We have not yet considered deletion of IIA Approval, whether in reaction to IIA deletion or without IIA deletion. Because we cannot avoid decision on the impact of IIA deletion on IIA Approval, it is necessary to consider both situations.
Impact on IIA Approval at the deleting party A
Impact on IIA Approval at the partner B
Deletion of IIA Approval We already know that deletion of IIA may trigger deletion of IIA Approval. We should also decide if deletion of IIA Approval, not triggered by deletion of IIA, is allowed.
Other issues It is natural in a distributed network that the partner does not react immediately to our requests. A deletes IIA and notifies B. It may take some time for B to react, regardless of whether B reacts automatically or as a result of a human intervention. Therefore, system A should be able to deal with the situation where it receives a CNR for an object that does not exist (it is OK to ignore such CNR). In turn, B must be aware that delaying the decision may be troublesome for A. If A receives a proposal from an index endpoint, it will present it to its user, and the user has every right to treat it as full-fledged and ready for approval.
My 2 cents
Deletion of IIA – impact on the deleting party A
- A MAY delete IIA with A.iia_id. From now on, A MUST not ever have an IIA exposed on the EWP network with such identifier, so A MUST never expose IIA with A.iia_id on any endpoint (as a server).
I suggest to add the bold words, to conform to what have been said previously.
Deletion of IIA – impact on the partner B
- B MUST remove A.iia_id from his copy of IIA, because B already knows that A.iia_id does not exist.This is compliant with a general referential integrity rule.
I don't think that this is due to the general referential integrity rule. We can agree that we have to remove A.iia_id from B's copy, but not to comply the above mentioned rule, that says:
You, as a server developer, are allowed to hide some of your entities from some (or from all) of the other partners, but if you do, then you MUST hide them consistently, so that referential integrity is not broken.
B is hiding nothing, A.iia_id is a piece of information that is not tied to anything in the B's availability. Even if we decide to leave it in B's copy of the IIA, A cannot use it to access anything in the B' side.
- If this IIA contains mapping A.iia_id which does not exist (this should not happen, but A should properly respond to such request), A MAY ignore such IIA (or the mapping).
Good to know, therefore it's better if B removes A.iia_id, but if B doesn't react immediately B is not guilty of crashing A's system
- If this IIA does not contain any mapping, A MUST treat it like a new proposal.
If all providers acted as me, I would not see any problem in this. But since I'm afraid that someone creates its copy of the IIA as soon as its system receives a CNR for an IIA that is considered "a new proposal", this could mean that the previous deletion is useless, because an identical IIA is recreated, only with a different IIA_id. Anyway, I don't have a solution to suggest, except:
Since it's not my problem, I can simply ignore it, hoping that if someone is really interested in, he will give its contribution.
- B has to be aware that from the business perspective an IIA exposed in the network is a valid IIA, ready to be approved and put in effect. The fact that A decided to delete it may be (but does not have to) a strong indication that A does not intend to sign an agreement with B in such form. Modifications will be needed, anyway. B’s user needs time to prepare these modifications, so keeping the IIA in the network in the old form does not help in smooth cooperation. It may be better to hide the IIA from EWP (i.e. delete it) and when the user decides about modifications, post it again (with a new id).
This is your view, and it's not written as a rule, so ok. But from the business perspective the B's IIA exposed on the network can always remain a valid IIA, ready to be approved and put into effect, as it is. As if B tells A: "this is my mind for an IIA with you; if you approve it we put it effect, otherwise I will never offer you anything different. Our basis to discuss for an IIA will be ever the IIA I have exposed". B has the full right to assume this position; maybe it's not friendly, but even not forbidden.
As concerns the impact on the approval, I'll give my opinion in a separate post, to keep the two activities separated
Anyway, I don't have a solution to suggest, except: Not un-mapping on B side until the IIA will b really associated to a new A.IIA_id (so that B has no IIA CNR to send) Not sending the IIA CNR after the un-mapping, even if required by general rules
I have a solution to suggest. In fact, as I remember, in the past it has been suggested to other providers. Do not create IIA automatically in your system after getting IIA CNR from the partner. This IIA will be visible to the partner from the moment it is created and the partner has a full right to assume that you like it and send the approval. But your customers have not yet seen it, may be they would never agree to it (in the situation considered this is exactly it, they have already deleted their copy).
As I remember from the chat this problem was raised by IS4UE. Let's discuss together how to approach it. Quite possibly I do not properly assume how the local solution works.
I think we should face the approval in its own repository, and only once we all have agreed on its features we can apply it to this particular case.
Anyway:
Impact on IIA Approval at the deleting party A
- A deletes IIA with A.iia_id. If A keeps approval of B’s copy (A has IIA Approval for A.iia_id and B.iia_id with B0.hash), then according to the referential integrity rule, A MUST also delete IIA Approval. A MUST not use A.iia_id in his system which means that it MUST not keep IIA Approval related to A.iia_id. Deleting IIA Approval, A has to send IIA Approval CNR with B.iia_id and respond to IIA Approval request with B.iia_id with an empty response.
I think this is not correct; you said that A cannot expose anymore on the network but now you're saying something more: _A MUST not use A.iiaid in his system This is not necessarily true. Moreover, in this case there is no referential integrity to preserve. How A has arrived to approve B copy?
Could you explain me where is involved A.iia_id in this task and what referential integrity rule has been infringed?
What could happen if A does nothing? B calls A's approval and sees that A has previously approved it's IIA identified by B.iia_id and B.hash_code That's true; how can B use this information to take and advantage? No way for B, in my opinion; B should already have un-mapped its IIA, therefore the hash code now is different. But even if B has not un-mapped yet, it cannot send a CNR Approval to A, because it has to call previously A's IIA Get API and this should prevent B to approve anything.
To conclude: in my opinion, A can do whatever it prefers with the previous approval.
Impact on IIA Approval at the partner B
- A deletes IIA with A.iia_id, and informs B, B keeps not only IIA, but also IIA Approval (for A.iia_id with A0.hash and B.iia_id).
As I said, this is not necessarily true; I add that I wish that an approval keeps always both of the IIA IDs and hash codes, but currently it's not so; I will come back to this on the approval repository.
- If B decides to delete his copy B0, then according to the referential integrity rule, B
Even this is not true in my opinion, it's equal to the first case (Impact on IIA Approval at the deleting party A) so I don't repeat the above consideration.
- B has to be aware that from the business perspective an IIA Approval exposed in the network is a valid IIA Approval, meaning that for B this IIA, mapped with a non-existing A.iia_id copy and A0.hash, is ready to be put in effect.
This is not absolutely true, this is your interpretation useful to suggest wrong readings. You should use the past, that is:
from the business perspective an IIA Approval exposed in the network is a valid IIA Approval, meaning that for B this IIA, mapped with a non-existing A.iia_id copy and A0.hash, WAS ready to be put in effect
The hash code has the fundamental role to confirm that an approval is related only to a particular IIA version. If the current IIA is changed, or not exists anymore, it cannot be put into effect, it would require a new update approval (that in this case we cannot give anymore)
Deletion of IIA Approval We already know that deletion of IIA may trigger deletion of IIA Approval.
Really? Until now, we didn't know that an Approval can be deleted and suddenly we know that the deletion of IIA may trigger deletion of IIA Approval?
In my opinion, since an IIA Approval is valid only for a given hash code, we don't need to delete anything, it becomes invalid by itself when the hash code changes. And this is not only my opinion, it is what have been suggested by BPOs and commonly accepted:
The above "Mandatory Business requirements Erasmus+ Inter-institutional agreement" never consider the possibility of withdrawing an approval, therefore we are walking in a dangerous way. For them, the user requirements are only: "Users must be able to consult/import/approve an agreement from the partner via EWPs"
On the contrary they recommend:
The local implementation should keep the approval response from the partner in its system to demonstrate at any time the agreement was approved;
Talking about the deletion of an "approval object" you ask us to do something that BPOs recommended us to non to do. They don't talk about mutual approval and not they don't consider the deletion of an IIA; they simply know that we may need "to demonstrate at any time the agreement was approved"
On the contrary, deleting the approval could mean attesting the false, that is a partner has never approved a copy. This is not true: it has approved the IIA identified by IIA_id.a0 an IIA_hashcode.a0
If the BPOs will ask us to deal with the possibility to withdraw an approval, we can do, but we have to represent them all the possible consequences and to consider the necessity to change something in the Approval CNR API and/or Approval API
Statement After intensive consultations, the ESCI consortium and DG EAC decided to approve selection of option B in the scenario presented in the file 2023-10-06-DeleteWorkshopChat.xlsx. It means that partner B MAY keep his copy of the IIA deleted by partner A, but MUST remove the mapping, i.e. A's identifier of this IIA. Nevertheless, the ESCI consortium and DG EAC would like to emphasize that the recommended option for B is to delete his copy from the network, but – if needed – to keep it locally and restore it on the network after modifications made by the end user.
Assumptions
Deletion of IIA – impact on the deleting party A
Currently in README.md for IIAs we have a text:
Such IIA MUST not be present in any of the IIA endpoints.
To avoid any misinterpretation we will change it to:
Such IIA MUST NOT be exposed by the server in any of the IIA endpoints.
Deletion of IIA – impact on the partner B
Such text will be added to get-response.xsd to enforce this rule (new part in italic):
ID used by the partner MUST be provided as soon as it is mapped in the local system. Otherwise, an agreement might be interpreted as not mapped, which may lead to duplicates. Also, it is REQUIRED to provide partner agreement ID to be able to approve it. Yet, this ID MUST NOT be present if the partner has deleted the IIA.
Statement After intensive consultations, the ESCI consortium and DG EAC decided that in order to avoid imposing additional requirements on developers to handle the impact of IIA deletion on IIA Approval, no changes will be made to the specification. In practical terms, this means that implementers have a free hand in this respect. Nevertheless, the ESCI consortium and DG EAC would like to emphasize that the strongly recommended option is to delete the IIA Approval together with the deletion of the IIA to which it applies. The description provided below should be treated as the recommended implementation.
Deletion of IIA – impact on IIA Approval at the deleting party
Deletion of IIA – impact on IIA Approval at partner B
It is natural in a distributed network that a partner does not react immediately to our requests. Partner A deletes IIA and notifies partner B. It may take some time for B to react, regardless of whether B reacts automatically or as a result of a human intervention. Therefore, system A should be able to deal with the situation where it receives a CNR for an object that does not exist (it is OK to ignore such CNR). In turn, B must be aware that delaying the decision may be troublesome for A. If A receives a proposal from an index endpoint, it will present it to its user, and the user has every right to treat it as full-fledged and ready for approval.
The content of this post, in PDF format, has been uploaded to https://esci-sd.atlassian.net/wiki/spaces/ITSC/pages/113999873/Infrastructure+Forum+and+technical+workshops (resources for Infrastructure Forum meeting 15/11/2023).
The file 2023-10-28-Delete ModifyScenarios.xlsx with updated scenarios, compliant with the above description, has been uploaded to https://esci-sd.atlassian.net/wiki/spaces/ITSC/pages/113999873/Infrastructure+Forum+and+technical+workshops (resources for Infrastructure Forum meeting 15/11/2023).
Nevertheless , we would like to emphasize that the recommended option for B is to delete his copy from the network, but – if needed – to keep it locally and restore it on the network after modifications made by the end user.
"We" who should represent? I think that such a recommendation is an attempt to force a behavior that only you (the same of your "we") are interested in and therefore should not be present in any official document after the decision of the ESCI consortium and DG EAC.
Anyway, I hope that all the described behaviors will be added to the IIA API page.
- Comment: B has to be aware that from the business perspective an IIA exposed in the network is a valid IIA, ready to be approved and put in effect. The fact that A decided to delete it may be (but does not have to) a strong indication that A does not intend to sign an agreement with B in such form. Modifications will be needed, anyway. B’s user needs time to decide about withdrawal of this IIA or possible modifications, so keeping the IIA in the network in the old form for a longer time may be misleading. It may be better to hide the IIA from EWP (i.e. delete it from the network, but keep local copy), and when the user decides about modifications, post it again (with a new identifier).
I don't like at all this way to do a moral suasion with a misrepresentation of the reality. First of all: you invite B " to be aware that from the business perspective an IIA exposed in the network is a valid IIA, ready to be approved and put in effect." Have you ever reminded A the same thing? A provider has exposed on the network a valid IIA, ready to be approved and put into effect, but he/she is free to delete it, without a warning from your. Why? Isn't his/her behavior the same of B, if not worse? But you seem interested only in making the B partner to delete its copy, not in the business perspective.
Moreover, the expression "a strong indication that A does not intend to sign an agreement with B in such form" proceeded by "(but does not have to)" is only a trick to suggest an interpretation, ignoring that A, or B, could simply have made a mistake in mapping and the system one of them finds more useful a deletion than a un-mapping.
Your expression "A does not intend to sign an agreement with B in such form" instead reveals a different thing: A's system implements a "parasite" behavior so that, for it, it is easier to delete and force B to a lot of duties than amending the IIA by itself (as it is clarified by the next sentence "Modifications will be needed, anyway"). So that the one who will change the IIA is B and A can simply check and import it. Good to know; but this is not the master-master model.
Obviously, the sentence "Modifications will be needed, anyway" is not true, not at all at least. It may be needed or not, you only represent a scenario that can justify your vision.
Who can be mislead by "keeping the IIA in the network in the old form for a longer time"? Once un-mapped, we have the partner A without a corresponding IIA and B with an IIA, ready to be approved and that B is interested in. This is the most common situation in the EWP Network, when each of us exposes his set of IIAs for the first time. What is misleading in this basic scenario?
The description provided below should be treated as the recommended implementation.
It has been said several times: "should" and "recommended" are not valid terms to be used in specifications. You must provide all the valid solutions without recommending anything. And if all the solutions are valid, you can stress this freedom without adding any further recommendation.
- If A keeps approval of B’s copy (A has IIA Approval with A.iia_id, B.iia_id, B0.hash), then A SHOULD also delete the related IIA Approval. (...) which means that it SHOULD not expose in the network IIA Approval related to the deleted IIA ... If B decides to delete his copy B0 of IIA, then B SHOULD also delete the related IIA Approval.
Not SHOULD, MUST !!
- Even if B decides to keep his copy B0 of IIA, B SHOULD delete IIA Approval with A.iia_id because
Absolutely not, you cannot use the term SHOULD. You're against the vote of all the providers during the last Infrastructure Forum and against the decision of "the ESCI consortium and DG EAC ... to avoid imposing additional requirements on developers"
- Comment: B has to be aware that from the business perspective an IIA Approval exposed in the network is a valid IIA Approval, meaning that for B this IIA, mapped with a non-existing A.iia_id copy and A0.hash, is ready to be put in effect.
This is a useless comment and also a misrepresented situation. What has really happened is that, before the A's deletion, B has approved A's IIA in that version, as specified by the hash code; from a business perspective it would be wrong to compel B to assert that it hasn't approved that version, or the IIA at all.
But here we have the real problem: you're describing the use of the approval without having ever specified what is an approval, what it means from time to time, if it can be withdrawn and so on.
In turn, B must be aware that delaying the decision may be troublesome for A. If A receives a proposal from an index endpoint, it will present it to its user, and the user has every right to treat it as full-fledged and ready for approval.
And what should be the problem? The IROs of A "has every right to treat it as full-fledged and ready for approval". Correct, what would be the difference from the ordinary? B's user is fully entitled to delete his IIA later and then notify to A, isn't it? It's the same that A has just done (B has approved and A has deleted). Therefore, why do we need to stress this possibility, as if B is an unfair partner?
Do you find my answer too long? I agree, please try to write what will be exactly included in the specifications, separating it from the comments and correctly using the expressions "must" and "must not", and by sure I will make shorter and more precise observations, it needed
"We" who should represent?
ESCI consortium and DG EAC.
Anyway, I hope that all the described behaviors will be added to the IIA API page.
The envisioned changes are described in https://github.com/erasmus-without-paper/ewp-specs-api-iias/issues/139#issuecomment-1783760063.
"We" who should represent?
ESCI consortium and DG EAC.
Interesting, therefore they refer to themselves both as "ESCI consortium and DG EAC" and "we", mixing third and first person? And, moreover, they decide "to avoid imposing additional requirements on developers" but at the same time they strongly recommend to do the opposite? And maybe they don't understand yet why the developers are so perplexed...
The envisioned changes are described in #139 (comment).
If the entire comment #139 will become specification, I can already say that it's completely wrong and a carrier of problems.
Interesting, therefore they refer to themselves both as "ESCI consortium and DG EAC" and "we", mixing third and first person?
My bad English again. I will edit this in the original post to avoid confusion.
If the entire comment https://github.com/erasmus-without-paper/ewp-specs-api-iias/issues/139 will become specification, I can already say that it's completely wrong and a carrier of problems.
Of course not, changes to specifications are clearly announced:
If the entire comment #139 will become specification, I can already say that it's completely wrong and a carrier of problems.
Of course not, changes to specifications are clearly announced:
* Such text will be added to get-response.xsd to enforce this rule (new part in italic): * Such text will be added to get-response.xsd to enforce this rule (new part in italic):
We cannot limit the specifications to XSD. It's useful, but not enough. We have the descriptive page, that indeed is, or should be, a specification of how to use the communication (XSD is the envelope, it can describe the format of the data to be exchanged, not how and when)
It has been said several times: "should" and "recommended" are not valid terms to be used in specifications. You must provide all the valid solutions without recommending anything.
They are, see: https://datatracker.ietf.org/doc/html/rfc2119 Also these are not specifications, only supporting documents.
If B decides to delete his copy B0 of IIA, then B SHOULD also delete the related IIA Approval. Not SHOULD, MUST !!
No, it is up to B.
The following clarifications have been added to the specifications to cater for the decisions made:
They are, see: https://datatracker.ietf.org/doc/html/rfc2119 Also these are not specifications, only supporting documents.
Yes, they are, but not as you're using them. You cannot hide a requirement behind the term SHOULD; "the ESCI consortium and DG EAC decided that in order to avoid imposing additional requirements on developers", so you use SHOULD instead MUST but you mean "must"?
MUST is needed to ensure the interoperability, but SHOULD?
If B decides to delete his copy B0 of IIA, then B SHOULD also delete the related IIA Approval.
So we have no more the problem of the referential integrity rule? Using SHOULD, who decides the good reason to not delete? And if B doesn't delete, what happens to A?
SHOULD has to be used to stress a good practice, e.g.: "you SHOULD capitalize the name of constants when you write the code." If you don't, nothing hangs. I'm fully legitimate to not obey to a "SHOULD" instruction
May I remember that if we are still at this point, it is because the EWP specifications are stuffed with too much "SHOULD" and too little "MUST" making a lot of ambiguities?
You cannot hide a requirement behind the term SHOULD; "the ESCI consortium and DG EAC decided that in order to avoid imposing additional requirements on developers", so you use SHOULD instead MUST but you mean "must"?
No, by writing SHOULD I mean that this is recommendation, and I do not mean MUST.
I'm fully legitimate to not obey to a "SHOULD" instruction
Of course, because this is only recommendation.
May I remember that if we are still at this point, it is because the EWP specifications are stuffed with too much "SHOULD" and too little "MUST" making a lot of ambiguities?
The ESCI consortium and DG EAC would definitely prefer to put MUST to the specification. As I understand you support. If the other providers also support, the obligations to delete IIA Approvals can be introduced.
I'm fully legitimate to not obey to a "SHOULD" instruction
Of course, because this is only recommendation.
My observation is mainly about "Part II Impact of deletion of IIA on IIA Approval". I think that it can be summarized with "as concern the IIA approval every partner can implement the most appropriate solution for his system". Instead, there is a lot to read to come to the same conclusion.
May I remember that if we are still at this point, it is because the EWP specifications are stuffed with too much "SHOULD" and too little "MUST" making a lot of ambiguities?
The ESCI consortium and DG EAC would definitely prefer to put MUST to the specification. As I understand you support. If the other providers also support, the obligations to delete IIA Approvals can be introduced.
If they really would have preferred to put MUST whilst the whole community has voted the opposite, I think that they have not the best approach to the EWP development and their vision may complicate the project. In my opinion, this involves that EWP developers have to be even more convinced in rising objection to any decisions they will try to impose, because it could make serious problems into the interoperability.
To be clear, as I said above, I'm not for the obligations to delete IIA Approval, but I remember that this is what you have represented to the last IF for the deleting part. Therefore, I simply stressed that if this is still your position, the IIA Approval deletion must be mandatory for the deleting part (and not for the partner, as voted during the IF). But if you mean that now both of the partners are free to delete or not the IIA approval, I'm glad to know, but at this point the whole section "Part II Impact of deletion of IIA on IIA Approval" is very little relevant.
I think that it can be summarized with "as concern the IIA approval every partner can implement the most appropriate solution for his system".
Statement says exactly that: no changes will be made to the specification. In practical terms, this means that implementers have a free hand in this respect.
If they really would have preferred to put MUST whilst the whole community has voted the opposite,
There was no voting on the impact of IIA deletion on IIA Approval.
Could you kindly share the chat transcription, as in the past? The chat transcriptions are no more present here https://esci-sd.atlassian.net/wiki/spaces/ITSC/pages/113999873/Infrastructure+Forum+and+technical+workshops
I was asked not to, because of GDPR. This is why I alway ask participants of the Infrastructure Forum meetings to save the chat at the end of the meeting.
I was asked not to, because of GDPR. This is why I alway ask participants of the Infrastructure Forum meetings to save the chat at the end of the meeting.
How? I don't remember this opportunity in Zoom. Anyway, I think it should be shared in some way, e.g. within the Registration Portal? Or, better, sending it to the mailing list for ewp-providers, the same who were entitled to participate in the IF and see the chat?
How? I don't remember this opportunity in Zoom.
I confirmed it with the Relationship Managers before I offered it at IF.
Anyway, I think it should be shared in some way, e.g. within the Registration Portal?
Anybody who is present has access and can download. RP is not for sharing content. If anybody really missed this opportunity, can send request to the contact email.
Anybody who is present has access and can download.
Where should I have to go to access? I even registered myself in Zoom, but in my space I cannot see the old meetings. I always entered, as most of us, I think, using the straight link you provide by mail. We don't need to be registered
Where should I have to go to access?
This option is available during the meeting.This is why I remind about it during the meeting.
In this issue I will try to summarise what we already know about IIA deletion and what we still need to clarify to make specification straightforward. I will close other issues on deletion, they are already too long to follow, anyway.
Let's remember that this is not yet specification. Whatever will be decided will be used to augment specs (after approval of DG EAC and the consortium).