search

erasmus-without-paper / ewp-specs-api-iias

Specifications of EWP's Interinstitutional Agreements API.
MIT License
4 stars 13 forks source link

Sending comments in context of IIAs and their approval #97

Closed janinamincer-daszkiewicz closed 1 year ago

janinamincer-daszkiewicz commented 1 year ago

Infrastructure Forum meeting 2022-12-14.

New Comment API proposal has been rejected (see https://github.com/erasmus-without-paper/general-issues/issues/38#issuecomment-1356184410).

Comment field in IIA has been voted with the following results:

(the results of voting are summarised in an Excel file uploaded to the IF shared drive).

We will continue the discussion in this issue to reach a final solution no later than January 18, 2023 (the next IF meeting).

janinamincer-daszkiewicz commented 1 year ago

Most of the providers do not want new API. Having comment in IIA does not allow us to send a comment to a partner when we do not have a local copy of IIA and do not want to create one. Some providers want to share comments in a way similar to how reject decision (with a comment) is delivered for Incoming LAs.

The compromise would be to come back to the proposal from IF meeting on 2022-05-14, see

We proposed to add new update endpoint to https://github.com/erasmus-without-paper/ewp-specs-api-iias/tree/stable-v6/endpoints:

On 2022-05-25 IF we decided to postpone issuing update endpoint for IIA until early 2023. We are almost here :)

Lets summarize. We propose the following comment functionality for IIAs and IIA Approvals:

We may try to design possible implementation for the local systems (in particular if both way of sharing comments are to stay).

Please discuss the details of this functionality here but do not come back to options which have already been rejected.

umesh-qs commented 1 year ago

Having a new end point for comments is same as having new comment API, which is already rejected. So the only option I see is "add comment field to IIA to keep comment on current state of negotiations".

demilatof commented 1 year ago

Having a new end point for comments is same as having new comment API, which is already rejected. So the only option I see is "add comment field to IIA to keep comment on current state of negotiations".

I agree, and in my opinion the comment field should have a passive function: we use it in case of the partner decides to download our IIA copy (such as a post-it on our IIA). If we think of sending an IIA CNR to notify only the presence of a comment and nothing else, the field is no more a comment, but a message. And for the messages we have other and better tools (email, phone, WhatsApp).

JoepDemey commented 1 year ago

For the OLA's, there is 1 endpoint to approve a proposal or decline it with a comment. To be consequent in the way flows in different API's work, I would propose to change the Interinstitutional Agreements Approval API response so you can respond with an approval, or a comment:

<xs:element name="iias-approval-response"> <xs:complexType> <xs:sequence> <xs:element name="iia-id" type="ewp:AsciiPrintableIdentifier" /> <xs:element name="conditions-hash" type="ewp:Sha256Hex" /> <xs:choice minOccurs="1" maxOccurs="1"> <xs:element ref="iia-approval" /> <xs:element ref="iia-comment" /> </xs:choice> </xs:sequence> </xs:complexType> </xs:element>

janinamincer-daszkiewicz commented 1 year ago

@umesh-qs

Having a new end point for comments is same as having new comment API, which is already rejected. So the only option I see is "add comment field to IIA to keep comment on current state of negotiations".

New comment API means two APIs: Comment CNR and Comment (with get endpoint). Update would be more similar to SMS. Some providers seem to prefer such approach.

If no new API and no update endpoint in IIA API, then what remains is a comment field in IIA, sending IIA CNR to inform about a new comment, getting it with IIA get endpoint. This however does not allow to solve issues raised by Masaryk University, when we want to inform the partner about something not having the local copy of IIA. Which means that the problem would not be solved fully.

There is also an option to not add anything. But again, my impression was that many providers were interested in having the possibility to add comments in the context of IIA (not outside it, in email, phone, WhatsApp).

@JoepDemey Your proposal also does not allow to solve issues raised by Masaryk University.

demilatof commented 1 year ago

There is also an option to not add anything. But again, my impression was that many providers were interested in having the possibility to add comments in the context of IIA (not outside it, in email, phone, WhatsApp).

This is my most liked option; if providers are interested in comments they can do as I did in my internal system: two fields, one for internal note (managed by officer, as reminder) and one for system note (managed by system, to log important event). These fields are not shared by means of EWP, and most probably they should not be shared with partners. If we have to communicate with our partners we use communication system that are really reliable and that we can trust in: email, phone, SMS, WhatsApp and so on. If someone wants to facilitate the officers, it's enough to add a "mailto" hyperlink in the GUI next to each IIA item.

I'm not so confident that we can make a reliable communication system with the little game of IIA-CNRs. Mainly I'm afraid of two issues:

  1. The comments may be added to the same IIA several times, we therefore send multiple IIA-CNRs but finally the partner performs only one IIA-Get. It will lost the most part of the communication.
  2. Abusing of the comment system: we will break the EWP model because the officers will start to put aside the EWP functionalities and they will start modifying everything by means of comments. In half a year the EWP system will be out of control.

Therefore, I'm against the introduction of any type of comments in IIAs.

JoepDemey commented 1 year ago

@janinamincer-daszkiewicz what are the issues raised by Masaryk University?

I could only find this in your post: "This however does not allow to solve issues raised by Masaryk University, when we want to inform the partner about something not having the local copy of IIA." With my suggestion to change the IIA approval API, you don't need a local copy of an IIA to send a comment, you just respond to the partner IIA with either an approval (as it works now) or a comment (my suggested change).

umesh-qs commented 1 year ago

@janinamincer-daszkiewicz what are the issues raised by Masaryk University?

I could only find this in your post: "This however does not allow to solve issues raised by Masaryk University, when we want to inform the partner about something not having the local copy of IIA." With my suggestion to change the IIA approval API, you don't need a local copy of an IIA to send a comment, you just respond to the partner IIA with either an approval (as it works now) or a comment (my suggested change).

@JoepDemey my initial proposal was inline with this, expect I wanted to change the name of the API to IIA status instead of IIA approval and additionally accommodate the IIA status field as well

demilatof commented 1 year ago

@umesh-qs

@JoepDemey my initial proposal was inline with this, expect I wanted to change the name of the API to IIA status instead of IIA approval and additionally accommodate the IIA status field as well

I agree with you both, in my opinion, there is even no need to change the API name. We can keep the current approval if we can approve the IIA and add a message to the Approval API answer in case of Incorrect Parameters. We may have an unconstrained message or a code that identify the problem when we cannot give the approval, such as: 01 (wrong parameters) 02 (hash code changed) 03 (missing both IIA IDs) ... 09 (others)

I don't understand the technical reason that arises so many difficulties to add a message element to an answer.

mkurzydlowski commented 1 year ago

In general I would be very cautious about adding comments to EWP objects as they might introduce inconsistencies between the object data and the comment. The party sharing a comment might misbehave by hiding its intent in the comment and wrongly assuming that the partner would respect this comment irrelevant to other data being present in the object or the communication as whole.

skishk commented 1 year ago

i think comments in IIA means the comment must be on own IIA and not to comunicate something to the other it could be just a note or detail to explain something more clearly, and not to change condition or other important information. so it could be just a not required field.

demilatof commented 1 year ago

so it could be just a not required field

@skishk I've already implemented comments, or better notes, in my system without needing to wait for EWP changes. I've 2 note levels: for system (added by the system) and for user (added by officers)

umesh-qs commented 1 year ago

i think comments in IIA means the comment must be on own IIA and not to comunicate something to the other it could be just a note or detail to explain something more clearly, and not to change condition or other important information. so it could be just a not required field.

Yes for own IIA we can add comments internally. But purpose of comments here is to communicate any problems with partner IIA.

umesh-qs commented 1 year ago

so it could be just a not required field

@skishk I've already implemented comments, or better notes, in my system without needing to wait for EWP changes. I've 2 note levels: for system (added by the system) and for user (added by officers)

@demilatof how are you communicating any issues that you have with partner IIA?

demilatof commented 1 year ago

@demilatof how are you communicating any issues that you have with partner IIA?

@umesh-qs do you know email? Phone? WhatsApp? ;-) They are the only reliable systems; comments in the IIA (IIA-CNR then IIA-GET) are a really poor communication system, prone to errors and message lost.

For issues at the system level, we cannot think to use comments in IIA without informing our operator. For this kind of issues I would like a message in the response, but it seems that they prefer building a huge infrastructure instead of add a field where I can write: "approval denied: wrong hash code"

umesh-qs commented 1 year ago

@demilatof how are you communicating any issues that you have with partner IIA?

@umesh-qs do you know email? Phone? WhatsApp? ;-) They are the only reliable systems; comments in the IIA (IIA-CNR then IIA-GET) are a really poor communication system, prone to errors and message lost.

@demilatof of course I do :). But the point was on integration within EWP network.

For issues at the system level, we cannot think to use comments in IIA without informing our operator. For this kind of issues I would like a message in the response, but it seems that they prefer building a huge infrastructure instead of add a field where I can write: "approval denied: wrong hash code"

@demilatof purpose of comments is not to solve system level issues. It is more to do with making approval process faster and easier. Let us not negate the use-case of comments, for want of better solution. We can continue to debate on the best possible solution.

demilatof commented 1 year ago

@umesh-qs we don't need to integrate them within EWP network in the same way we don't need them or other solutions to agree on the content of an IIA before inserting it in the system. We all have IIAs in our system; nobody has invented them, they arise from communication between the parties (by phone or email). The approval process is a really small part compared to the initial step of making an IIA

A good communication system must be a push one, not a pull one as IIA-GET (with a fake push such as IIA-CNR). To communicate something to you I need to call your API and post the IIA-ID, a message, a progressive ID, a timestamp and eventually your progressive ID if I'm answering to a particular comment from you. You have to store immediately what I post to you and give me a status code 200. After that is up to you to represent comments in your system. I didn't see nothing that could be similar to the above process, but only a complicated sequence of IIA-Gets and IIA-CNRs.

I've already listed some of the problems:

  1. Abuse the comments to change the cooperation-conditions
  2. Lost messages if we don't transmit all of them from the beginning each time
  3. Storing problem if we transmit all of them (the xml becomes bigger and bigger)

How can we lose messages? We add a comment in IIA and then we send an IIA-CNR. We can even imagine that none of the IIA-CNRs will be lost; but every partner has its own implementation on fetching the IIA.

A partner can download the IIA immediately, another one only on officer's request, another one every 5 minutes. What happens if in the meantime someone adds a new comment that does not substitute but integrate the previous one? The partner reads only the last part, the officer would not understand and finally he/she would contact the other party by email/phone. The same that he/she could have done immediately.

If we want comments we have to design a good and reliable system; I don't see this competence in EWP at the moment. And personally I have other priorities.

JoepDemey commented 1 year ago

In general I would be very cautious about adding comments to EWP objects as they might introduce inconsistencies between the object data and the comment. The party sharing a comment might misbehave by hiding its intent in the comment and wrongly assuming that the partner would respect this comment irrelevant to other data being present in the object or the communication as whole.

I think that's exactly why comments should not be on the IIA, but on the reaction to that IIA (approval/rejection). The owner of the IIA should not add any comment to it's own IIA, but the partner should be able to reject that IIA and comment why.

umesh-qs commented 1 year ago

In general I would be very cautious about adding comments to EWP objects as they might introduce inconsistencies between the object data and the comment. The party sharing a comment might misbehave by hiding its intent in the comment and wrongly assuming that the partner would respect this comment irrelevant to other data being present in the object or the communication as whole.

I think that's exactly why comments should not be on the IIA, but on the reaction to that IIA (approval/rejection). The owner of the IIA should not add any comment to it's own IIA, but the partner should be able to reject that IIA and comment why.

IIA approval API needs some changes to accommodate this to tell the calling partner to differentiate between approval and rejection. Also both IIA ids in response need not be mandatory if it is a rejection.

demilatof commented 1 year ago

IIA approval API needs some changes to accommodate this to tell the calling partner to differentiate between approval and rejection.

I agree with @JoepDemey, it's a long time I'm asking for. I think we need small changes to accommodate this, for example we can add:

    <not_approved>
        <iia-id>0f7a5682-faf7-49a7-9cc7-ec486c49a281</iia-id>
        <message>We sent Approval-CNR for hash code XX now you hashcode is YY</conditions-hash>
    </not_approved>

Also both IIA ids in response need not be mandatory if it is a rejection.

I agree (as concern them, I accept to add both of them if they want, but I don't realize what they could helpful for).

umesh-qs commented 1 year ago

IIA approval API needs some changes to accommodate this to tell the calling partner to differentiate between approval and rejection.

I agree with @JoepDemey, it's a long time I'm asking for. I think we need small changes to accommodate this, for example we can add:

    <not_approved>
        <iia-id>0f7a5682-faf7-49a7-9cc7-ec486c49a281</iia-id>
        <message>We sent Approval-CNR for hash code XX now you hashcode is YY</conditions-hash>
    </not_approved>

Also both IIA ids in response need not be mandatory if it is a rejection.

I agree (as concern them, I accept to add both of them if they want, but I don't realize what they could helpful for).

@demilatof there is a proposal to make both the IIAs mandatory. I am saying the opposite. Both should not be mandatory. I should be able to pass message even if I have not linked the IIAs

demilatof commented 1 year ago

@demilatof there is a proposal to make both the IIAs mandatory. I am saying the opposite. Both should not be mandatory. I should be able to pass message even if I have not linked the IIAs

@umesh-qs How can you manage a proper response without either an IIA ID, the one you received in the request? If you don't receive the partner IIA-ID (the one to be approved), your API answers with HTTP Error 400. If you receive the IIA-ID, your response must use it to let the partner knows what you are talking about.

In my opinion, the link doesn't matter, but the partner's IIA ID must be present in the response. At this point even for "not_approved".

umesh-qs commented 1 year ago
0f7a5682-faf7-49a7-9cc7-ec486c49a281 We sent Approval-CNR for hash code XX now you hashcode is YY

@demilatof I am saying mapping both IIAs should not be necessary if I just want to comment about your IIA without creating my internal IIA.

demilatof commented 1 year ago

@demilatof I am saying mapping both IIAs should not be necessary if I just want to comment about your IIA without creating my internal IIA.

Both IIAs are not necessary, but mine yes; I need to know what you're talking about in your comment

umesh-qs commented 1 year ago

@demilatof I am saying mapping both IIAs should not be necessary if I just want to comment about your IIA without creating my internal IIA.

Both IIAs are not necessary, but mine yes; I need to know what you're talking about in your comment

Yes that is what I meant, which is already the case in current approval API GET response

janinamincer-daszkiewicz commented 1 year ago

Sending comments in context of IIAs and their approval

As reported during the Infrastructure Forum on 2023-01-18: