search

erasmus-without-paper / ewp-specs-api-iias

Specifications of EWP's Interinstitutional Agreements API.
MIT License
4 stars 13 forks source link

not-yet-defined: dangerous side effect #148

Open demilatof opened 11 months ago

demilatof commented 11 months ago

As described here, the not-yet-defined is 


    <xs:attribute name="not-yet-defined" type="xs:boolean">
        <xs:annotation>
            <xs:documentation>
                If given MUST be set to "true". Flags an element value as not yet defined
                and MAY be set only for mutually approved agreements that
                have been approved in the old IIA API version and have not been modified since,
                and MUST NOT be set otherwise.
            </xs:documentation>
        </xs:annotation>
    </xs:attribute>

This is not what I've suggested when I proposed the not-yet-defined attribute. It should be used EVEN for the IIAs exposed with version 6 but without some mandatory fields required by version 7. Doing so we can exchange those IIAs even with v. 7 without breaking the XSD as soon as we put into production the version 7.

Otherwise we cannot expose those IIAs, but according the new rules this means that they are deleted and so on. But they have to be approved in that form, this is not a real deletion! And I may have my IIA already v. 7 compatible, but if my partner deletes his IIA and expose it again with a new IIA ID my IROs have to redo all the checking and mapping job.

skishk commented 11 months ago

in my opinion is easier and enough using nyd (not-yet-defined) on all the new attributes or new required attributes.

now is declared on "mobilities-per-year" and "language". and why not on "isced-f-code"?

why on "isced-f-code" is used another attribute "v6-value" ? that have the same meaning with another name!?

janinamincer-daszkiewicz commented 11 months ago

The attribute 'not-yet-defined' should be used only when absolutely necesary. It is the case when IIAs are mutually approved. It is not the case when they are not, as when switching in the local system from v6 to v7 we can ask users to deliver missing values.

janinamincer-daszkiewicz commented 11 months ago

why on "isced-f-code" is used another attribute "v6-value" ? that have the same meaning with another name!?

There is a difference between a value not yet defined and a value which is defined but too short.

skishk commented 11 months ago

The attribute 'not-yet-defined' should be used only when absolutely necesary. It is the case when IIAs are mutually approved

for all the IIAs already shared on the EWP network with no mutual approval which don't have the new required values? have we hide them? and partner consider them deleted? and then we can not use the same IIA-ID only because nyd is not usable for no mutual approved ?! I think it will create a bit of confusion in the exchange of the IIAs.

janinamincer-daszkiewicz commented 11 months ago

for all the IIAs already shared on the EWP network with no mutual approval which don't have the new required values?

Yes

have we hide them? and partner consider them deleted? and then we can not use the same IIA-ID only because nyd is not usable for no mutual approved ?!

No hide, but complete the missing data

jiripetrzelka commented 11 months ago

No hide, but complete the missing data

You mean ask IROs to fill in the data until a certain date? What if they ignore this request?

janinamincer-daszkiewicz commented 11 months ago

You mean ask IROs to fill in the data until a certain date? What if they ignore this request?

Yes, this is what I mean and this is what we plan to do. If they ignore that means that they do not care, which may mean that you can fill the missing data automatically.

demilatof commented 11 months ago

You mean ask IROs to fill in the data until a certain date? What if they ignore this request?

Yes, this is what I mean and this is what we plan to do. If they ignore that means that they do not care, which may mean that you can fill the missing data automatically.

How can it be possible to fill missing data automatically?

E.g.: during the version 6 period, my partner has exposed an IIA (not mutually approved) without the "recommended-language-skill" in the StudentMobilitySpecification section. His IROs doesn't care of filling it before the deadline (31 march). What could happen?

  1. If I call his IIA-Get API he cannot answer me with a valid XML (because the recommended-language-skill element is mandatory in the XSD)
  2. If he provides me an invalid XML, I have to report it (I suppose)
  3. His system fills data automatically, as you suggest, just to make a valid XML
  4. His system doesn't return anything and I considered it as deleted

As concern point 3, this means (for example) that his system can set Russian as language skill, even if the country is Spain. How can my IROs recognize that this is a wished value and not a random value? If we flag it with "not-yet-defined" as I said months ago, my IROs can know that it's just a pad, but if we don't flag it, as you say, my IROs are legitimate to approve it.

janinamincer-daszkiewicz commented 11 months ago

There is enough time to have the data filled by the users. Filling automatically should be avoided, I mentioned it only as the last resort solution. Anyway, this issue should be resolved locally by each provider. We already know that in version 7 the network will not accept missing values.

demilatof commented 10 months ago

There is enough time to have the data filled by the users. Filling automatically should be avoided, I mentioned it only as the last resort solution. Anyway, this issue should be resolved locally by each provider. We already know that in version 7 the network will not accept missing values.

As always, it's an optimistic view. The IROs have only three months to check all their IIAs while they have other to do... And the same for the developers: they have to report to IROs all the IIAs not compatible with version 7, and this is time consuming also.

And if they don't? We use the last resort solution, filling automatically with the problems I described?

The XSLT already takes care of the not-yet-defined attribute: if it is used, the IIA is flagged as

valid-for-approval>false</valid-for-approval>

Therefore why you have decided to impose such a limit on the use of the not-yet-defined attribute?

fioravanti-unibo commented 10 months ago

By the way, the attributes 'not-yet-defined' and 'v6-value' are described and processed differently in the iia-get v. 7 specs and in the iias-approval v. 2 specs:

Clients and servers MUST NOT use IIA Approval API for agreements that: [...] contain an element with not-yet-defined attribute set to "true" or an value-v6 attribute / this rule does not apply to IIAs mutually approved in version 6 and unmodified since,

...implying that these attributes can be applied to agreements not mutually approved and we must check if it is the case.

That's simply not true: in the XSD for the iia-get API is clearly stated that not-yet-defined and v6-value must be used only for mutually approved agreements, so the phrase is useless or misleading.

janinamincer-daszkiewicz commented 10 months ago

The IROs have only three months to check all their IIAs while they have other to do...

The sooner they start, the better.

Therefore why you have decided to impose such a limit on the use of the not-yet-defined attribute?

The specification has been publicly available, we asked for review and comments. Nobody opposed. The conclusion is that this is what providers accept. Now we all need quiet time for implementation.

janinamincer-daszkiewicz commented 10 months ago

first: in the iias-approval spec is cited the attribute 'value-v6', which simply doesn't exist: the attribute name, as stated by the XSD for the iia-get API, is 'v6-value'

This is typo, which will be corrrected.

second: in the approval API description we can read the phrase ...implying that these attributes can be applied to agreements not mutually approved and we must check if it is the case.

First sentence: "Clients and servers MUST NOT use IIA Approval API for agreements that: [...] contain an element with not-yet-defined attribute set to "true" or an value-v6 attribute" was supposed to mean that only when someone manually corrects the data will the attributes disappear and it will be possible to accept such an agreement. However, if these attributes cannot be used for unaccepted contracts, this sentence may be considered somewhat misleading.

We will reformulate this sentence to make it fully clear.

@fioravanti-unibo thank you for this valuable input. @mkurzydlowski please prepare these clarifications and link them here.

demilatof commented 10 months ago

The IROs have only three months to check all their IIAs while they have other to do...

The sooner they start, the better.

It's up to you and DG EAC informing all the IROs about this technical requirement and of what will happen if they don't settle everything before the deadline. Instead, I only see a useless spot of the kind "how are we marvellous" when there are Erasmus Goes Digital seminars.

Therefore why you have decided to impose such a limit on the use of the not-yet-defined attribute?

The specification has been publicly available, we asked for review and comments. Nobody opposed. The conclusion is that this is what providers accept. Now we all need quite time for implementation.

This doesn't answer my question and at the same time is the worst justification I've ever heard. If a specification is wrong, it's wrong. Claiming that nobody has opposed before isn't enough to transform a nonsense in a good requirement. We approach specification while we implement them (first round) and we dig them in deeper while we examine all the possible scenario we could have to face (second round). You cannot expect that we can have the time, and the duty, to inspect them as soon as they are published and, even worse, in a not definitive way; we need our time, even according with our customers' priorities.

demilatof commented 10 months ago

We will reformulate this sentence to make it fully clear.

Are you sure that @mkurzydlowski has to reformulate this sentence and not the other one in the get-response.xsd? Anyway, there is no need to reformulate anything, since nobody has opposed before...

janinamincer-daszkiewicz commented 10 months ago

It's up to you and DG EAC informing all the IROs about this technical requirement and of what will happen if they don't settle everything before the deadline.

It is up to providers to prepare proper solution and inform their customers.

demilatof commented 10 months ago

It's up to you and DG EAC informing all the IROs about this technical requirement and of what will happen if they don't settle everything before the deadline.

It is up to providers to prepare proper solution and inform their customers.

Have you informed providers of this aspect? I can make a proper solution for my system, but I cannot impose anything to other part if not officially declared.

E.g.: my solution could be to already make mandatory the recommended-language-skills, and not approve my partner's IIA if those values are missing. Might I? Because doing so, I would be against the current version, the one we are using, and my partner could answer that currently I cannot force him yet. And technically, he's right.

janinamincer-daszkiewicz commented 10 months ago

P.S. By 'reject' I mean - treat as an error.

demilatof commented 10 months ago
  • You can make recommended-language-skills mandatory in your system in v6.
  • You cannot reject partner's IIA without recommended-language-skills in v6 because they are compliant with the specification.
  • You don't have to approve. These values will be needed soon anyway.
  • You can reject partner's IIA without recommended-language-skills in v7 because they are not compliant with the specification.

Yes, I know, but this mean that we cannot prepare a proper solution if the partner is not collaborative. Because if a partner is small, with few mobilities, we can say that we don't approve his IIAs. But if a partner is important, we have all the interest in exchanging students and if he doesn't accept our solution to make these fields mandatory, we have to change the software to relax our solution.

This is why I think it could be useful a strong and official recommendation to all providers: from now on, please already code your IIAs as if they must be exchanged according to v7.

janinamincer-daszkiewicz commented 10 months ago

But if a partner is important, we have all the interest in exchanging students and if he doesn't accept our solution to make these fields mandatory, we have to change the software to relax our solution.

Such behavior of the partner would be difficult to explain when we have these fields officially 'to be' mandatory very soon.

This is why I think it could be useful a strong and official recommendation to all providers: from now on, please already code your IIAs as if they must be exchanged according to v7.

I agree, spreading this information using all possible channels can help the community.

skishk commented 10 months ago
  • You cannot reject partner's IIA without recommended-language-skills in v6 because they are compliant with the specification.

@janinamincer-daszkiewicz i opened this tkt ESCI-9862 because what you said here, and i completely agree with you, unfortunately is not respected from all providers and creating blocking and confusing situation on the IROs working. the specification of V6 it was clear so i think is better to not change the behavior of it because now there are a lot of problems in production and blocking situation on different not required values. the details are in the tkt ESCI-9862 .

janinamincer-daszkiewicz commented 10 months ago

so the phrase is useless or misleading.

We will remove this part.

janinamincer-daszkiewicz commented 10 months ago

Shared by Relationship manager on the providers mailing list:

I would like to take this opportunity to remind about the article on the ESCI portal that is written for IROs to clarify the implications of version 7 on existing IIAs depending on their approval status: https://erasmus-plus.ec.europa.eu/news/updating-the-inter-institutional-agreement-ewp-api-version-7

Section relevant to discussion in this issue:

What does the update mean for IIAs initiated but not mutually approved before the update?

All inter-institutional agreements initiated before the update need to follow the new standards for them to be mutually approved. In many cases, the draft agreement already follows the new standard (e.g. 4-digit ISCED codes and language requirements are provided) and in this case, no additional action is required by the users.

Published: 26 Sep 2023

demilatof commented 10 months ago

What does the update mean for IIAs initiated but not mutually approved before the update?

All inter-institutional agreements initiated before the update need to follow the new standards for them to be mutually approved.

This doesn't mean that before the 31 march the IIAs need to follow the new standards but only that after the 31 march they "need to follow the new standards for them to be mutually approved". And that sentence doesn't stress at all the fact that they cannot be exchanged after the 31 march if they don't follow the new standards. The not-yet-defined, as intended by me, was instead fully compliant with the above sentence because it allowed to exchange the IIAs as they were, but to not approve them thanks to the XSLT.

JoelBlancoCloserIdeas commented 10 months ago

Just to make sure I understood everything. If we have an IIA NOT mutually approved after 31 march without "mobilities-per-year" or "recommended-language-skills" (or isced code with 4 digits) because the users didn't change the data, we CAN'T use the attribute "not-yet-defined". If the partner calls our GET API, we answer with an invalid XML until our user makes changes to the IIA.

Is this behavior correct?

It is up to the provider to decide if he wants to automatically fill the missing data in NOT mutually approved IIAS. In my opinion, it doesn't make any sense to fill automatically without using the attribute "not-yet-defined" (that is forbidden for an IIA that is not mutually approved).

janinamincer-daszkiewicz commented 10 months ago

If we have an IIA NOT mutually approved after 31 march without "mobilities-per-year" or "recommended-language-skills" (or isced code with 4 digits) because the users didn't change the data, we CAN'T use the attribute "not-yet-defined".

Yes

If the partner calls our GET API, we answer with an invalid XML until our user makes changes to the IIA. Is this behavior correct?

No. We should not intentionally spread incorrect XMLs in the network.

It is up to the provider to decide if he wants to automatically fill the missing data in NOT mutually approved IIAS.

DG EAC recommendation was not to fill anything automatically.

skishk commented 10 months ago

No. We should not intentionally spread incorrect XMLs in the network.

So how do we have to answer in this scenario if we can not use not-yet-defined?

janinamincer-daszkiewicz commented 10 months ago

See https://github.com/erasmus-without-paper/ewp-specs-api-iias/issues/148#issuecomment-1828641138

skishk commented 10 months ago

See #148 (comment)

So we treat as an error a IIA-GET of the partner? On something existing and not deleted? And not changed… I think the attribute not-yet-defined have this scope… to solve scenarios like this…

In the answer we can not use not-yet-defined and we can not fill automatically the data. So the network will be filled with a lot of errors :(

janinamincer-daszkiewicz commented 10 months ago

First of all there is still quite a lot of time to complete missing data. Second, these IIAs which are not ready to be exchanged in the network may be withdrawn. Each provider will decide which way to choose.

Those incomplete IIAs should be completed as soon as possible.

skishk commented 10 months ago

Second, these IIAs which are not ready to be exchanged in the network may be withdrawn.

How? As deleted? And the when it will be complete could be re-shared in the network with all the same data (and completed with the new one) ?

janinamincer-daszkiewicz commented 10 months ago

How? As deleted? And the when it will be complete could be re-shared in the network with all the same data (and completed with the new one) ?

Yes, they can be deleted and come back with new ids and all mandatory data completed.

demilatof commented 10 months ago

First of all there is still quite a lot of time to complete missing data. Second, these IIAs which are not ready to be exchanged in the network may be withdrawn. Each provider will decide which way to choose.

Those incomplete IIAs should be completed as soon as possible.

Sadly, I think you don't have a great consideration of IROs work... Your approach doesn't consider four important aspects:

  1. Withdrawing an IIA means waste of time that IROs have spent in checking and filling IIAs that aren't mutually approved yet due to the other party. They have to repeat all...
  2. You can complete missing data, but if your partner doesn't the same, it is useless: you have to examine again the new IIA (new IIA Id) when it will become available
  3. It's not easy to fill missing data, because I can know what language skill is requested for incoming students (coming to my HEI), but if my partner hasn't filled in yet his incoming mobilities, I have no element to fill in the language skills for my outgoing mobilities. And they are mandatory. So my IROs have to spend more extra time in contacting partner

  4. Saying that there is a lot of time to complete missing data means not considering that new IIAs could be proposed still in March and in that period we cannot impose the partner to fill in data that is not mandatory yet. But if my IROs receive an IIA in March, they have to manage it.

    Or do DG EAC authorize us to reject the IIAs without mandatory data for version 7, starting from 1st January?

janinamincer-daszkiewicz commented 10 months ago

MUCI has more than 90 customers in all, more than 60 are in the EWP network, more than 40 have IIAs in PROD. I have a great consideration for their work.

skishk commented 10 months ago

Yes, they can be deleted and come back with new ids and all mandatory data completed.

I think it’s not the right and complaint solution for IROs… That means more work for all not justifiable… I’m sorry but I don’t agree. We will study a custom solution for us if the specification does not provide for the right management of this scenario.

umesh-qs commented 10 months ago

First of all there is still quite a lot of time to complete missing data. Second, these IIAs which are not ready to be exchanged in the network may be withdrawn. Each provider will decide which way to choose. Those incomplete IIAs should be completed as soon as possible.

Sadly, I think you don't have a great consideration of IROs work... Your approach doesn't consider four important aspects:

  1. Withdrawing an IIA means waste of time that IROs have spent in checking and filling IIAs that aren't mutually approved yet due to the other party. They have to repeat all...
  2. You can complete missing data, but if your partner doesn't the same, it is useless: you have to examine again the new IIA (new IIA Id) when it will become available
  3. It's not easy to fill missing data, because I can know what language skill is requested for incoming students (coming to my HEI), but if my partner hasn't filled in yet his incoming mobilities, I have no element to fill in the language skills for my outgoing mobilities. And they are mandatory. So my IROs have to spend more extra time in contacting partner
  4. Saying that there is a lot of time to complete missing data means not considering that new IIAs could be proposed still in March and in that period we cannot impose the partner to fill in data that is not mandatory yet. But if my IROs receive an IIA in March, they have to manage it.

Or do DG EAC authorize us to reject the IIAs without mandatory data for version 7, starting from 1st January?

Usability was never a focus of this project. EWP was and is a technical project. This is not the first time I see such blunt disregard of the genuine problem. I think DG EAC should reach out to the IROs and mention that they will have to do the rework.

demilatof commented 10 months ago

MUCI has more than 90 customers in all, more than 60 are in the EWP network, more than 40 have IIAs in PROD. I have a great consideration for their work.

Maybe for yours; but there are 4000 HEIs involved, more or less. Not-yet-defined was proposed to save their work, limiting it to only approved IIAs without a technical reason means to not consider all scenarios in the real world. It's like if there is the fear that admitting that the description in the XSD needs to be rewritten could be considered as the proof that the specifications are not so stable as claimed. Maybe; the alternative is not solving problems that will arise until March (and later)

JoepDemey commented 6 months ago

I'm looking for an answer but can't find it, so sorry if I'm asking a question that has already been answered:

We have V6 IIA's that were already approved, but are missing recommended language skills. What should happen with them in V7 API? Can the partner still get this IIA with the get request? The XML is invalid if I have no recommended language skills. If I add an empty recommended language skill with the not-yet-defined attribute set to true, the XML is still invalid without a language and cefr level. So, do I just put random values there that don't matter because the not-yet-defined attribute is set to true? Or is this IIA not visible at all in our API? If not, how could the partner ever know if this IIA is still active or not?

janinamincer-daszkiewicz commented 6 months ago

We have V6 IIA's that were already approved

I guess that you consider unilateraly approved IIAs.

Can the partner still get this IIA with the get request?

Yes, but only when the missing fields are filled with data by the users.

The XML is invalid if I have no recommended language skills. If I add an empty recommended language skill with the not-yet-defined attribute set to true, the XML is still invalid without a language and cefr level. So, do I just put random values there that don't matter because the not-yet-defined attribute is set to true?

No, it is against specification.

Or is this IIA not visible at all in our API? If not, how could the partner ever know if this IIA is still active or not?

No, that would mean that it is deleted (of course you can delete if this is what users want).

You can trigger an error in your system when anybody wants to get this IIA, and send the message to your user asking them to fill in the data.

You can also proactively inform the users about such IIAs pending for missing data.

JoepDemey commented 6 months ago

So we need to change IIA's (fill in "missing" fields (they were not mandatory in V6), meaning that the hash also changes and we need a new approval from the partner) that were already mutually approved in V6? If we don't fill in the missing data, what should we show the partner that wants to get this IIA in our API? If we show an empty response, that means the IIA was deleted, right?

janinamincer-daszkiewicz commented 6 months ago

So we need to change IIA's (fill in "missing" fields (they were not mandatory in V6), meaning that the hash also changes and we need a new approval from the partner) that were already mutually approved in V6?

For mutually approved you can fill the fields with random data and use not-yet-defined. The fields will have to be filled with data only when new negotiations start. For unilateraly approved the users should fill in the data, the hash will change and you will need a new approval.

If we don't fill in the missing data, what should we show the partner that wants to get this IIA in our API? If we show an empty response, that means the IIA was deleted, right?

Right.

demilatof commented 6 months ago

For unilateraly approved the users should fill in the data, the hash will change and you will need a new approval.

If we don't fill in the missing data, what should we show the partner that wants to get this IIA in our API? If we show an empty response, that means the IIA was deleted, right?

Right.

This means that a provider cannot activate the new IIA API v. 7, at least until all of the not mutually approved IIAs will be filled in with missing values, otherwise what could happen is:

  1. The IIA is returned as broken (as per XSD specification)
  2. The IIA is not returned at all (that means it is deleted)
janinamincer-daszkiewicz commented 6 months ago

This means that a provider cannot activate the new IIA API v. 7, at least until all of the not mutually approved IIAs will be filled in with missing values, otherwise what could happen is:

A provider can activate the new IIA API v.7, but cannot sent XSD which is not compliant with the specification. Triggering error (which will be observed by the partner) is OK - partner will know that this object exists but there is a problem with it.

Anyway, the sooner end users fill in missing data, the better.