search

erasmus-without-paper / general-issues

An empty project for tracking issues not related to any specific project.
0 stars 1 forks source link

A messenger API proposal or message functionality in CNRs #38

Closed sascoms closed 1 year ago

sascoms commented 2 years ago

During the renewal process of the IIAs, it became certain that either a messenger API is needed or CNRs should be able to carry messages between HEIs.

Both solutions would be good.

The need arises especially during the IIA renewal process or the negotiation process of the IIA.

A HEI needs to ask something to its partner or explain a change in the IIA revision/proposal. And this is not possible. Therefore, HEIs return to emails which makes it more difficult.

We are aware that there is a "other info terms" field in the conditions of the IIAs to convey some information about the condition. But this is not meant for general notes, explanations or short queries. Instead, it is related to the terms of the condition and should be kept the same during the negotiation or renewal process.

"we have increased/decreased the quota as ..."; "removed the Ph.D. degree for the SMS condition as this program was terminated.." etc.

Such functionality would help and speed up the communication between HEIs. And also a track of the messages would be kept under each IIA or HEI record as an archive or log.

This can be done either by a new messenger API (general use) or a message can be added to CNRs for specific APIs such as IIAs.

kamil-olszewski-uw commented 2 years ago

If the situation requires it, communicating outside of EWP is nothing wrong.

@mkurzydlowski can explain exactly why adding anything to CNRs is a bad idea.

And when it comes to a separate API that would be used for communication - at the very beginning of the EWP project, we considered these kinds of things, but concluded that it would be more trouble than good. In fact, it would mean duplicating the functionality that emails provide and dealing with all the things email communication does well.

janinamincer-daszkiewicz commented 2 years ago

In my humble opinion CNR is not supposed to carry any information but a flag that something has changed. HEIs probably remember only the last one of the series of CNRs and the date it has come. We shouldn't add more business logic to it.

A separate API - in that respect I agree with Kamil.

Adding extra fields for comments to 'get' - why not, if more developers will support such idea with convincing arguments.

jiripetrzelka commented 2 years ago

I am in favor of adding an extra optional comment field to the specification of the IIA (version 7) that would allow coordinators to write for example "I added the language requirements that we agreed upon by e-mail on 15 February, please approve". Currently, some coordinators insert this type of information into the other-info-terms, which is very unhandy.

janinamincer-daszkiewicz commented 2 years ago

Sounds like something simple, however there is a hidden obstacle - this comment would be another element of the IIA which would have to be excluded before calculating hash. I wouldn't risk extending specification in this direction.

I wouldn'd be so much again such extra comments, but on the other hand sending comments by XML document is like XML taking functionalities of email. Do we really need them? If the answer is YES, it is hard to operate without these extra comments, let's find a better place for them, not inside the part wich is the basis for calculating hash.

Also implementing such comments will rise other open questions. Should we keep the whole history of comments? Only the last one? When to stop sending them to the partner? We have such comments in nominations and honestly speaking the worflow is not that straightforward. Small but painful, I would say ;)

jiripetrzelka commented 2 years ago

My idea was to include the comment outside of the hashed portion of the IIA, so it would be at the same level as for example the 'in-effect' element. There would be no history, just one text field. The contents would not necessarily have to be related to the approval process but could include any additional information that the coordinator thinks might be useful for the counterparty. The owning side of the IIA could change or remove the comment at any time.

janinamincer-daszkiewicz commented 2 years ago

That would be nice to get some support for this change :) Who else is interested?

janinamincer-daszkiewicz commented 2 years ago

@jiripetrzelka It looks that there is not much interest in this idea, at least now. Let's make the last try. Please, describe how you would implement this new feature in your system. I will also ask you to present it on 2022-04-06 at the next tech meeting. Then we will see if there are more developers willing to implement such change.

umesh-qs commented 2 years ago

In principle we agree with the use case but implementation timeframe would be a concern.

umesh-qs commented 2 years ago

And this should be extended to communication across other APIs and not limited to IIA

jiripetrzelka commented 2 years ago

The implementation would be very simple. A new textarea field would be added to the IIA form and the contents could be edited by users with edit rights to the IIA at any time, regardless of the current status of the approval process (which is much more rigid). There would of course be a listener that would trigger a CNR whenever the contents of comment field would change.

From the counterparty's perspective, the comment would be displayed to users in a visible way. Change in the contents of the field would probably not trigger a notification e-mail to the person responsible for dealing with partner IIAs.

As to the contents of the field, it would be up to coordinators to decide what kind of information they would put in it. I guess some might insert data that would resemble a log of what has already been decided, while others might be inclined to insert information related to the future (what is to be decided, confirmed, approved etc.).

The question is how to ensure that the future-related information don't get outdated once the IIA is approved. For example, the system could ask the user to review the comment once the IIA is approved.

Perhaps others might think of other workflows. @sascoms, you raised this issue, what do you think about this?

sascoms commented 2 years ago

The problem is arisen especially during IIA content negotiation process between partners.

However, as I mentioned in my first message, this need is not limited to the IIAs. It is also applicable to any other API or needed by most data exchange APIs.

If we ourselves were planning this change, we would like to have this functionality in two ways:

  1. a separate API where HEIs can communicate over EWP just like async chat or email. [EWP Messaging Inbox]

    • you do not need to know the IRO email or the responsible person's email.
    • no more problems if the IRO staff changes or the email addresses change.
    • can be used to convey messages and text information for any topic or issue between HEIs. (EWP, mobilities, nominations, LAs, IIAs, factsheet, dates, general info, or inquiries)

  2. Add IIA API its special comment field (even if this is added, the # 1 solution is still needed for common and general messages)

    • should not be inside the blocks used for hash calculation
    • should not be reflected by the partner. but the partner can save this comment/notes in their system if needed just like history or log of messaging in between.
    • field format: free text
    • all IIA related communication will be carried out under one platform (no need to switch from EWP to email/inboxes -> where is that email I have sent for x IIA?)

Use cases:

//etc.//

3PP systems sample screens/uses:

for IIAs

Separate API - Sample Basic Structure (no need to complicate it):

<message xmlns="..">
    <content>
       Parte nam nabataeaque litora sublime nabataeaque fontes humanas. Facientes nullaque litem divino ligavit: adspirate poena ab. Nullo peragebant quisquis deerat aeris quicquam effigiem. Terram convexi habitabilis tum ita coegit? Instabilis feras. Onerosior numero. Mutatas iners coercuit quarum subsidere omni pendebat non quam fossae.
    </content>
</message>

IIA API - field added:

<iia>
        ...
        <in-effect>true</in-effect>
        <cooperation-conditions>
            ...
        </cooperation-conditions>
        <conditions-hash>7c045bc4ca23b3b9953adb27374aa27dcd41cfdda74fff9d2240a813a80443ae</conditions-hash>

        **<message>....<message>**

 </iia>

Now some answers to the responses above:

umesh-qs commented 2 years ago

We are in agreement with most of what @sascoms has drafted. This is the direction we should be looking towards. Just one point on the separate communication API is that it should include unique id like IIA ID, Mobility ID in the context for which the comments are.

lioliosn commented 2 years ago

I am not sure about the effectiveness of such a solution, as these discussions about IIAs, LAs, ToRs etc involve also HEI parties (faculty members, Schools' secretariats) that are not users of the Dashboard or third party mobility software. So, emails with multiple recipients (irrespectively of their access to EWP) is the fastest and most effective way to clarify/decide for the actions that will take place in EWP.

umesh-qs commented 2 years ago

It will have an effect on achieving the digital goal of EWP. We must integrate more and more processes that are currently outside of current EWP architecture. Otherwise end users will get frustrated switching back and forth and tying non EWP communication with EWP exchange.

lioliosn commented 2 years ago

Users (IRO staff members) will be even more frustrated if they will be asked to introduce and train hundreds or even thousands of faculty and staff members, that are potential counterparts in the mobility workflows, to yet another online tool. I see this feature as a valuable but peripheral one, for the current point of time, since the focus is to build up/restore the credibility of the EWP network among end users.

janinamincer-daszkiewicz commented 2 years ago

Fully agree. Building new tools to replace others, already existing, well know, used daily, seems to me like wasted effort.

There are so many mail clients, why users keep using Gmail. Should we have built in email client in the mobility modules of our systems? Or new chat functionality? To have it better integrated with the system? Following that thread of thought nobody would be integrated business domains systems with other tools, but would be incorporating this functionality to the domains systems.

umesh-qs commented 2 years ago

Users (IRO staff members) will be even more frustrated if they will be asked to introduce and train hundreds or even thousands of faculty and staff members, that are potential counterparts in the mobility workflows, to yet another online tool. I see this feature as a valuable but peripheral one, for the current point of time, since the focus is to build up/restore the credibility of the EWP network among end users.

We do not see this as a reason for dropping this. Yes, IRO staff will take time to get used to it. But once they are they will find it easier since everything is within EWP. And if we base the entire EWP project on IRO training problem, then nothing will go through.

Yes we do agree that this is not pressing and can be in future roadmap.

lioliosn commented 2 years ago

Users (IRO staff members) will be even more frustrated if they will be asked to introduce and train hundreds or even thousands of faculty and staff members, that are potential counterparts in the mobility workflows, to yet another online tool. I see this feature as a valuable but peripheral one, for the current point of time, since the focus is to build up/restore the credibility of the EWP network among end users.

We do not see this as a reason for dropping this. Yes, IRO staff will take time to get used to it. But once they are they will find it easier since everything is within EWP. And if we base the entire EWP project on IRO training problem, then nothing will go through.

Yes we do agree that this is not pressing and can be in future roadmap.

I was not referring to IRO training, but to IROs' overhead of training their HEIs' faculty and staff members. But even if this was the case, IRO staff members are the key driving forces behind the EWP project adoption, so we need to take their needs into consideration and not plan removing them from the bigger picture.

sascoms commented 2 years ago

This is suggested for the IRO <-> IRO communication. Not for all units / academic units or people related.

Users (IRO staff members) will be even more frustrated if they will be asked to introduce and train hundreds or even thousands of faculty and staff members, that are potential counterparts in the mobility workflows, to yet another online tool. I see this feature as a valuable but peripheral one, for the current point of time, since the focus is to build up/restore the credibility of the EWP network among end users.

lioliosn commented 2 years ago

This is suggested for the IRO <-> IRO communication. Not for all units / academic units or people related.

IRO staff will need to double check these issues with other staff members within their HEI. So, effectively we will ask them to use both communication tools (which the communication API is supposed to solve).

umesh-qs commented 2 years ago

We hear the need from our users a need for such an API. During our conversations with them on IIA & LA, there was always an ask about the ability to communicate with partners before the approval/signing. So, this API would make that ask possible.

We believe this API would make digitization more practical and not just limit itself to just predefined data exchange process.

If there are concerns about how IRO staff will react to it, then better take feedback from the universities. Not sure how, but this is an important step that should be part of API changes that significantly change the process that they follow currently

kamil-olszewski-uw commented 2 years ago

@sascoms The general idea is not bad, we can all reflect on it sometime, although as others have written - we now have many other more priority issues.

It is not as easy as it may seem. The devil is in the details. The more we start considering individual use cases, exceptional situations, etc., the more we encounter problems to be solved. Below I will refer to a few things as an example.

The problem is arisen especially during IIA content negotiation process between partners.

However, as I mentioned in my first message, this need is not limited to the IIAs. It is also applicable to any other API or needed by most data exchange APIs.

If we ourselves were planning this change, we would like to have this functionality in two ways:

  1. a separate API where HEIs can communicate over EWP just like async chat or email. [EWP Messaging Inbox]
  • you do not need to know the IRO email or the responsible person's email.
  • no more problems if the IRO staff changes or the email addresses change.
  • can be used to convey messages and text information for any topic or issue between HEIs. (EWP, mobilities, nominations, LAs, IIAs, factsheet, dates, general info, or inquiries)

In the case of a large university, dropping all messages into one bag is not convenient at all, because different people in the IRO have different tasks and everyone would have to decide what is addressed to them, or someone would have to deal with assigning messages to recipients - it would have to be implemented.

And if we add the ability to assign the sent message to a specific API or to the id of a specific object, the matter of indicating the addressee becomes a little easier. But you still have to think, for example, about marking a message as "done" so as not to miss it when someone reads it first. In addition, in the case of e-mails, the correspondence may relate to several objects or smoothly move to other objects, maintaining the previous context and history of correspondence. It is rarely possible to settle something with a single piece of information, so we would have to deal with the continuity of correspondence in the event that a discussion should arise. How to unequivocally refer to the partner's statement? How to maintain the continuity of correspondence? You would have to e.g. enter the thread id, collect messages from one thread in the system, handle incorrect situations when the response comes from a thread id that does not exist...

By the way: IROs from different universities often have dedicated functional e-mail addresses, sometimes several, so in such cases you don't have to remember personalized e-mails. And the other way - when you have a well-known partner university, a personal e-mail often allows you to get things done faster.

  1. Add IIA API its special comment field (even if this is added, the # 1 solution is still needed for common and general messages)
  • should not be inside the blocks used for hash calculation
  • should not be reflected by the partner. but the partner can save this comment/notes in their system if needed just like history or log of messaging in between.
  • field format: free text
  • all IIA related communication will be carried out under one platform (no need to switch from EWP to email/inboxes -> where is that email I have sent for x IIA?)

How would the university retrieving the object (get requester) notify the author of the comment that it has read the message? It could happen, for example, that the author introduces a new comment after subsequent changes, and the partner site did not read the previous one.

What if your partner wants to respond to a comment because they don't agree with something? Only the IIAs APIs are symmetric. It is difficult to find a universal mechanism of mutual discussion at the level of commentary. Moving it to the messenger API (solution 1) would disrupt the continuity of the discussion. If the author of the comment decides in advance that the matter requires discussion, they can use the new messenger API instead of using the comment field. But the author of the comment may not predict that the matter will require discussion because "things seem obvious".

Use cases:

  • Partner A sends an IIA to partner B. But partner B not responded. Partner A use this API#1 to ask the status of this IIA.
  • Partner A sends an IIA to partner B. Partner B changes some information in the conditions and wants to add some general explanations and notes to be conveyed to the partner A. (vice versa during this content negotiation process)
  • Partner A wants to create a new IIA with a new partner (B) and sends a message to partner B for units, ISCED clarifications, and other conditions settlement before creating an IIA.
  • Partner A wants to terminate an agreement (there is no workflow for the IIA termination in EWP currently). Sends a message to Partner B and informs about the termination.

All of the above are very general use cases. In fact, these are the reasons why partners would need to contact each other, whether using the new mechanisms within the EWP or outside the EWP.

  • A message will clarify why the CNR is sent. In the current implementation of IIA CNRs, a CNR have many meanings: --- I prepared a new IIA and this CNR is to inform you about that. --- I have imported your IIA and I want to notify you about my IIA ID --- I have changed the contents/conditions and this CNR is to inform you about this change. --- I saw your changes but do not accept these due to ... and therefore, sending a CNR to inform you that I have not accepted the changes you proposed. We want to sign it as our version. //etc.//

Here, as I understand it, you say that the comment sent with get response would explain why CNR was sent. What if several CNRs are sent for one object? Will the HEI sending CNR have to remember all reasons for all CNRs and accumulate them in this message (which will then be pulled by the partner) and decide which "old reasons" can already be removed from the message?

  • Partner B wants to contact Partner A about a special case/issue of a nominated student or a problem of their student currently in host HEI.

If something very unusual has happened and there is a serious problem to be resolved, then the EWP short message mechanism will not be a good place to talk about it.

  • Student ToR was not sent yet -> ask partner about that.
  • General LA problems -> Did you receive the LA of our student? if yes, can you review and approve or reject it?

If the partner university did not receive our LA (or any other object), it is probably due to technical problems of one of the two systems in communication with EWP. Troubleshooting using a communication channel that just failed is not advisable :-)

//etc.//

3PP systems sample screens/uses:

  • HEI list page -> select HEI -> send a message over EWP -> text box -> send (via API#1) or
  • HEI list page -> select HEI -> EWP Messaging page -> received/sent messages are listed here. (via API#1)

for IIAs

  • IIA details/edit page -> Add notes to the partner: [text box] -> save (triggers a CNR or manually send a CNR) (via IIA API) [CNRs are still better for sending one-time message]

Separate API - Sample Basic Structure (no need to complicate it):

<message xmlns="..">
   <content>
      Parte nam nabataeaque litora sublime nabataeaque fontes humanas. Facientes nullaque litem divino ligavit: adspirate poena ab. Nullo peragebant quisquis deerat aeris quicquam effigiem. Terram convexi habitabilis tum ita coegit? Instabilis feras. Onerosior numero. Mutatas iners coercuit quarum subsidere omni pendebat non quam fossae.
   </content>
</message>

IIA API - field added:

<iia>
       ...
       <in-effect>true</in-effect>
       <cooperation-conditions>
          ...
       </cooperation-conditions>
       <conditions-hash>7c045bc4ca23b3b9953adb27374aa27dcd41cfdda74fff9d2240a813a80443ae</conditions-hash>

       **<message>....<message>**

</iia>

Now some answers to the responses above:

  • I have not mentioned adding a message field to the CNRs in this message as a possible implementation. Yet, this also would be an easier solution. And I would like to hear the cons of this solution. And this also has the advantage of sending one-time messages (naturally no reflection in the get api + no need to fetch the IIA).

Cons of messages in CNRs:

  • I have read the objections that email communication should be used or XML should not take place of email traffic. Then, why do we use WhatsApp, Telegram, Slack, and all other similar apps? We should keep using emails for all communication and not these messaging apps? This point of view means we should not have invented the cellphones, either.

We have never argued that e-mail is the only correct form of communication on any matter. We argued that e-mails are a better form than an e-mail solution that we would have developed ourselves in EWP, and thus it would probably be worse than a good old solution widely used all over the world ;-) And every provider would have to implement an equivalent of e-mail client/server.

Admittedly, in a very simple form that you propose, it would not be breakneck implementation work, because you see it rather in the form of a very simple instant messaging service. But I am afraid to say that in the form you propose it would be of little use in business ;-) Hence all my questions and doubts above.

Because yes, I believe that when dealing with this type of matter, e-mail correspondence is much better than external commercial messaging services, which is shown by the complex correspondence between universities in recent months. Instant messaging works well for quick ad hoc correspondence when something needs to be done quickly. In addition, work e-mails remain on university servers, so they are very often a more trusted tool in the context of the regulations of a given university than instant messaging in the context of sensitive data.

However, if the world looked like WhatsApp would be better suited to all this than e-mails, we would suggest using WhatsApp at this stage, instead of implementing a similar functionality in EWP. Because WhatsApp gives us, for example, a correspondence history and a whole lot of other goodies that our users would immediately lack. And no provider would need to implement instant messaging client/server from scratch. The changes in the API that you propose would not give us the messenger functionality, rather something like classic SMS (or even pager) messages, for which we do not see our own responses :-)

  • If we are talking about digital Erasmus and digitalisation, then we should facilitate and leverage the work of our HEIs.
  • We are proud that our HEI/IRO users have really really high-level digital skills and continuously ask for such improvements. So, this is not only our proposal and it is a request from our HEIs.

We absolutely do not deny that you have had such requests from your clients (we just haven't met a single one). Nevertheless, it may be a politically incorrect remark, but our experience shows that digital skills are sometimes quite debatable, both at our universities and their partners, but everyone is great at reading and writing e-mails ;-) Do universities requesting this functionality fully realize what the limitations would be compared to e-mail and how useful it would be for them?

janinamincer-daszkiewicz commented 1 year ago

Infrastructure Forum meeting 2022-12-14. We voted the proposal to add new Comment API (either specific for IIA or generic for all APIs). The voting results:

(the results of voting are summarised in an Excel file uploaded to the IF shared drive).

The proposal is rejected.

I close this issue, but other ways of sharing comments by partners will be discussed in a new issue.