search

ConnectingEurope / eInvoicing-EN16931

Validation artefacts for the European eInvoicing standard EN 16931
Other
129 stars 52 forks source link

CII Schematron - BT-8 - [CII-SR-462] - Only one DueDateTypeCode shall be present - Not well implemented - URGENT #387

Closed csautereau closed 1 day ago

csautereau commented 1 month ago

The attached CII invoices do not pass the last CII shematron.

They have many VAT breakdown lines, all with BT-8 = 72, but the schematron says

[CII-SR-462] - Only one DueDateTypeCode shall be present (which is the case : only 72)

The schematron counts the occurrence of DueDateTypeCode, where in CII this value is present in each occurrence of VAT Breakdown.

Then, any invoice with MORE than 1 VAT breakdown line is REJECTED !!

URGENT CORRECTION NEEDED

          <ram:ApplicableTradeTax>
                <ram:CalculatedAmount>390.00</ram:CalculatedAmount>
                <ram:TypeCode>VAT</ram:TypeCode>
                <ram:BasisAmount>1950.00</ram:BasisAmount>
                <ram:CategoryCode>S</ram:CategoryCode>
                <ram:DueDateTypeCode>5</ram:DueDateTypeCode>
                <ram:RateApplicablePercent>20.00</ram:RateApplicablePercent>
           </ram:ApplicableTradeTax>
           <ram:ApplicableTradeTax>
                <ram:CalculatedAmount>199.50</ram:CalculatedAmount>
                <ram:TypeCode>VAT</ram:TypeCode>
                <ram:BasisAmount>1995.00</ram:BasisAmount>
                <ram:CategoryCode>S</ram:CategoryCode>
                <ram:DueDateTypeCode>5</ram:DueDateTypeCode>
                <ram:RateApplicablePercent>10.00</ram:RateApplicablePercent>
           </ram:ApplicableTradeTax>

-- factur-x_F2023000152.xml.zip

facture_F20220023.xml.zip

csautereau commented 1 month ago

correction : BT-8 is only 5 in F2023000152 and only 72 in F20230023

oriol commented 1 month ago

DueDateTypeCode is an optional value, the solution is to inform only once this element.

csautereau commented 1 month ago

YES, it is a solution, but not the only one. There is no reason that an evolution of the schematron suddenly force to change implementation which is compliant to the EN16931. In addition without any information or time to comply (and we MUST have backward compatibility, except when correcting a bug, which is not the case here).

Schematron can be corrected in order to accept multiple BT-8 in CII with the same value (like Ite, quantity base has to be provided twice the same for Gross Price and Net price (BT-150).

KR

Cyrille

Cordialement

Cyrille SAUTEREAU

Président ADMAREL Conseil

Président FNFE-MPE (Forum National de la Facture Électronique et des Marchés Publics Électronique)

Mob : +33 6 07 53 32 85 E-mail : @.**@.> P avant d'imprimer, pensez à l'environnement, dématérialisez ! Cet e-mail est confidentiel. Si vous avez reçu ce message par erreur ou si vous n'êtes pas le destinataire de ce message, merci de bien vouloir nous en informer par retour de mail et détruire ce message. Vous ne devez imprimer aucun message, ni copier ni transférer le contenu d'un message qui ne vous est pas destiné. Merci de noter que l'intégrité ou la sécurité de ce message ne peuvent pas être garanties sur Internet. This e-mail is confidential. If you have received this message by mistake or if you are not the intended recipient of this message, please inform us by reply e-mail and delete this message from your system. You may not print any message, nor copy or disclose its content to anyone if you are not its intended recipient. Please note that the integrity and security of this message cannot be guaranteed on the Internet.

De : Oriol Bausa @.> Date : jeudi, 18 juillet 2024 à 16:02 À : ConnectingEurope/eInvoicing-EN16931 @.> Cc : csautereau @.>, Author @.> Objet : Re: [ConnectingEurope/eInvoicing-EN16931] CII Schematron - BT-8 - [CII-SR-462] - Only one DueDateTypeCode shall be present - Not well implemented - URGENT (Issue #387)

DueDateTypeCode is an optional value, the solution is to inform only once this element.

— Reply to this email directly, view it on GitHubhttps://github.com/ConnectingEurope/eInvoicing-EN16931/issues/387#issuecomment-2236613701, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ALFYEPMP3OCRIKGFV27T7X3ZM7DF7AVCNFSM6AAAAABLCIUNCSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMZWGYYTGNZQGE. You are receiving this because you authored the thread.Message ID: @.***>

oriol commented 1 month ago

This was added because of #365.

csautereau commented 1 month ago

YES, but I fully disagree the way it has been done.

The rule is that there must be a unique value of BT-8 and CII is not like UBL. UBL provide BT-8 on document level, CII provides it on Breakdown Group which is 0..n.

Then in CII, the implementation should be : For each VAT breakdown (/rsm:CrossIndustryInvoice/rsm:SupplyChainTradeTransaction/ram:ApplicableHeaderTradeSettlement/ram:ApplicableTradeTax), The # of ram:DueDateTypeCode must be equal to # of value of ram:DueDateTypeCode

And the same for ram:TaxPointDate (BT-7)

KR

Cyrille.

Cordialement

Cyrille SAUTEREAU

Président ADMAREL Conseil

Président FNFE-MPE (Forum National de la Facture Électronique et des Marchés Publics Électronique)

Mob : +33 6 07 53 32 85 E-mail : @.**@.> P avant d'imprimer, pensez à l'environnement, dématérialisez ! Cet e-mail est confidentiel. Si vous avez reçu ce message par erreur ou si vous n'êtes pas le destinataire de ce message, merci de bien vouloir nous en informer par retour de mail et détruire ce message. Vous ne devez imprimer aucun message, ni copier ni transférer le contenu d'un message qui ne vous est pas destiné. Merci de noter que l'intégrité ou la sécurité de ce message ne peuvent pas être garanties sur Internet. This e-mail is confidential. If you have received this message by mistake or if you are not the intended recipient of this message, please inform us by reply e-mail and delete this message from your system. You may not print any message, nor copy or disclose its content to anyone if you are not its intended recipient. Please note that the integrity and security of this message cannot be guaranteed on the Internet.

De : Oriol Bausa @.> Date : jeudi, 18 juillet 2024 à 16:54 À : ConnectingEurope/eInvoicing-EN16931 @.> Cc : csautereau @.>, Author @.> Objet : Re: [ConnectingEurope/eInvoicing-EN16931] CII Schematron - BT-8 - [CII-SR-462] - Only one DueDateTypeCode shall be present - Not well implemented - URGENT (Issue #387)

This was added because of #365https://github.com/ConnectingEurope/eInvoicing-EN16931/issues/365.

— Reply to this email directly, view it on GitHubhttps://github.com/ConnectingEurope/eInvoicing-EN16931/issues/387#issuecomment-2236785532, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ALFYEPIHHRXMQ4BQ3B6CLP3ZM7JKTAVCNFSM6AAAAABLCIUNCSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMZWG44DKNJTGI. You are receiving this because you authored the thread.Message ID: @.***>

oriol commented 1 month ago

The EN16931 is a semantic model, and the semantic model is what drives the European Norm. The fact that a specific syntax handles the data in a specific way does not mean that the semantic model has to be modified to support that particular syntax specification, but the other way around. The syntax shall stick to the semantic model. And in this case the semantic model states that there is only one Due Date Code, and this is what has been implemented in the syntax bindings.

The issue could be in this particular case that we could end up with different Due Date Codes in a single invoice, and then an automated system would not know which one to apply.

If you want to to modify the semantics, I think that the best way is to do it through the CEN TC 434. I'd suggest to a request a change of the cardinality to the CEN TC and when approved it will be applied in the syntax bindings.

csautereau commented 1 month ago

Again, the subject is not to change the EN16931, but how it is implemented in CII or UBL

We have already the same subject for Item base Quantity (BT-150) which is unique in the EN16931. But in CII, this data is present in the Gros Price Group AND in the Net Price Group. Then, there is a rule which says that the 2 values MUST be equal.

This is just the same for BT-8 and BT-7. In CII, the place for those values is in a group which is repeatable. Then, in case it is repeated, the value MUST BE the same. That's all.

It is not written in the syntax binding that only the first or one occurrence of VAT Breakdown must contain the value of BT-8 or BT-7.

And it does not seem that implementing a rule controlling that only ONE VALUE of BT-8 (or BT-7) is present is undoable.

It is required to use the last version of EN16931 Schematron. The consequence is a minimum concern about changes which are not backward compatible and makes compliant invoices suddenly not compliant, with NO INFORMATION and clear notice in advance. CII with multiple VAT Breakdown including the same value for BT-8 is fully compliant semantically.

And more generally, any change which will add new control should be done carefully. There are many thousands of solutions which are issuing EN16931 compliant invoices based on documentation and current Schematron. If schematrons are changed against documentation and with no time to organize a correction, it is just not professional.

svanteschubert commented 1 week ago

As just mentioned in our TAG meeting and as indicated by Cyrille's given test files. The current test is counting the number of ram:DueDateTypeCode within the complete document by using count(ram:DueDateTypeCode[parent::ram:ApplicableTradeTax]) &lt;= 1 instead of testing the number of ram:DueDateTypeCode children within a ram:ApplicableTradeTax therefore the test should be count(ram:ApplicableTradeTax/ram:DueDateTypeCode[2]) &lt; 1! As Cyrille indicated there are many solutions, but this is new one is working one according to EN16931!

Going to provide a pull request.

svanteschubert commented 1 week ago

@oriol Please remove the label "CEN/TC 434 Discussion needed" as it had been an XPATH problem as I do not have the editing rights for labels.

csautereau commented 1 week ago

For svante : The correction should allow a single value for BT-8, in case it is present more than 1 time (for instance where there are more than 1 VAT breakdown line). The current implantation is only counting the number of BT-8.

Do your modification correct this ?

svanteschubert commented 1 week ago

Your test documents validate fine, and I modified one example and duplicated the ram:DueDateTypeCode to test that the new rule works in finding invalid documents with multiple ram:DueDateTypeCode children.

csautereau commented 1 week ago

2xml_CII_files.zip

I am not sure the correction works. Attached 2 xml CII files factur-x_F2024000223.xml has 4 value of BT-8 (4 VAT breakdown : ram:ApplicableTradeTax/ram:DueDateTypeCode) all equal to 72 => your modification approve it

factur-x_F2024000224.xml has 4 value of BT-8 (4 VAT breakdown : ram:ApplicableTradeTax/ram:DueDateTypeCode) the first 3 equal to 72 and the 4th = 5 => your modification approve it also where it should reject.

The rule should count the number of different value of ram:ApplicableTradeTax/ram:DueDateTypeCode, which must be 1 on factur-x_F2024000223.xml : there is only 1 value (as the 4 value are equal to 72) on factur-x_F2024000224.xml : there are 2 values : 72 (repeated 3 times) and 5

KR

csautereau commented 1 week ago

Mayube a way to do it, as we know that ram:ApplicableTradeTax/ram:DueDateTypeCode can only be equal ro 5, 29 or 72 (code list check BR-CL-06), maybe should we: Count the # of value = 5, then 29 and 72 and count that there are at least 2 values of these 3 which are equal to 0

svanteschubert commented 1 week ago

I was indeed only fixing the problem that more than one VAT BREAKDOWN (BG-23) with a BT-7 or BT-8 was no longer possible - which must be possible as it is allowed in the EN16931 norm (see my comment in #365).

But the EN16931-1 states on BT-8 also the following:

The code shall distinguish between the following entries of UNTDID 2005 [6]:
- Invoice document issue date
- Delivery date, actual
- Paid to dateThe Value added tax point date code is used if the Value added tax point date is not known when the invoice is issued. The use of BT-8 and BT-7 is mutually exclusive.

Therefore, I might add a validation of the mutually exclusive use of BT-8 and BT-7.

Is there more that needs to be validated, Cyrille? It seems to me - from your new example - that you like to ensure to have only the same BT-8 code for all occurrences in the document. If this is the case, is your demand also demanded by the law? And if it is in the law, we still have to update it in EN16931-1, as I can not find this rule in the current norm or its current internal draft.

csautereau commented 1 week ago

Svante

There is a difference between the Norm EN16931, and its implementation in CII (Syntax Binding).

The Norm obliges 1 occurrence of BT-8 value, BUT

Consequence for CII : in case there are more than 1 occurrence of VAT Breakdown, it is possible to repeat ram:DueDateTypeCode in each occurrence of ram:ApplicableTradeTax. In that case it should be allowed to have multiple values (which must be inside 5, 29, 72 (the 3 values already checked)) BUT they MUST be all equal together.

The current implementation is a simplification which forces to have only 1 occurrence ram:ApplicableTradeTax with one occurrence of ram:DueDateTypeCode.

Then, the implementation of the rule “Only 1 value of BT-8 in an invoice” should become in CII : “in case of multiple occurrences of BT-8, the value Must be always the same” It is a validation artefact subject. Technical implementation only.

Mutually exclusion is already implemented : BR-CO-03

KR

Cyrille

Cordialement

Cyrille SAUTEREAU

Président ADMAREL Conseil

Président FNFE-MPE (Forum National de la Facture Électronique et des Marchés Publics Électronique)

Mob : +33 6 07 53 32 85 E-mail : @.**@.> P avant d'imprimer, pensez à l'environnement, dématérialisez ! Cet e-mail est confidentiel. Si vous avez reçu ce message par erreur ou si vous n'êtes pas le destinataire de ce message, merci de bien vouloir nous en informer par retour de mail et détruire ce message. Vous ne devez imprimer aucun message, ni copier ni transférer le contenu d'un message qui ne vous est pas destiné. Merci de noter que l'intégrité ou la sécurité de ce message ne peuvent pas être garanties sur Internet. This e-mail is confidential. If you have received this message by mistake or if you are not the intended recipient of this message, please inform us by reply e-mail and delete this message from your system. You may not print any message, nor copy or disclose its content to anyone if you are not its intended recipient. Please note that the integrity and security of this message cannot be guaranteed on the Internet.

De : Svante Schubert @.> Date : jeudi, 5 septembre 2024 à 18:06 À : ConnectingEurope/eInvoicing-EN16931 @.> Cc : csautereau @.>, Author @.> Objet : Re: [ConnectingEurope/eInvoicing-EN16931] CII Schematron - BT-8 - [CII-SR-462] - Only one DueDateTypeCode shall be present - Not well implemented - URGENT (Issue #387)

I was indeed only fixing the problem that more than one VAT BREAKDOWN with a BT-7 and BT-8 was no longer possible - which must be possible as it is allowed in the EN16931 norm (see my comment https://github.com/ConnectingEurope/eInvoicing-EN16931/issues/365#issuecomment-2332082280 in #365https://github.com/ConnectingEurope/eInvoicing-EN16931/issues/365).

But the EN16931-1 states on BT-8 also the following:

The code shall distinguish between the following entries of UNTDID 2005 [6]:

Therefore, I might add a validation of the mutually exclusive use of BT-8 and BT-7.

Is there more that needs to be validated, Cyrille? It seems to me - from your new example - that you like to ensure to have only the same BT-8 code for all occurrences in the document. If this is the case, is your demand also demanded by the law? And if it is in the law, we still have to update it in EN16931-1, as I can not find this rule in the current norm or its current internal draft.

— Reply to this email directly, view it on GitHubhttps://github.com/ConnectingEurope/eInvoicing-EN16931/issues/387#issuecomment-2332124447, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ALFYEPNAF5DDLICGLODV7X3ZVB6RFAVCNFSM6AAAAABLCIUNCSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMZSGEZDINBUG4. You are receiving this because you authored the thread.Message ID: @.***>

svanteschubert commented 1 week ago

Yes, I found and tested BR-CO-03 meanwhile and tested it, and it works.

  1.  We both agree that the norm obliges 1 occurrence of BT-8 (Value added tax point date code) in a BT-32 (VAT BREAKDOWN) and that there are multiple BT-32 allowed in the norm.
  2. The Syntax Binding of UBL states a 1..n cardinality, which is different from what you are stating to the 1:1 cardinality on the document, level, but UBL does not concern this issue, so let's drop this.
  3. I asked about the law that all element values must be equal - I find it of course natural what you demand, but theoretically why not split the choice of BT-8 for each VAT BREAKDOWN using a different value for <ram:DueDateTypeCode>? Unless it is not forbidden, it is possible by the given grammar. In any case, the rules, we are testing here, must be found as well in EN16931, right?
svanteschubert commented 1 week ago

The solution for your request would be:

    <param name="CII-SR-463" value="count(ram:ApplicableTradeTax/ram:DueDateTypeCode) = count(ram:ApplicableTradeTax/ram:DueDateTypeCode[text() = '5']) or 
                                    count(ram:ApplicableTradeTax/ram:DueDateTypeCode) = count(ram:ApplicableTradeTax/ram:DueDateTypeCode[text() = '29']) or 
                                    count(ram:ApplicableTradeTax/ram:DueDateTypeCode) = count(ram:ApplicableTradeTax/ram:DueDateTypeCode[text() = '72'])"/>

I should assume if we require this criteria for BT-8, we should require this for BT-7 as well, right? Guess that is what the KoSIT asked for in #365

Thinking it over, I believe the core should have this requirement and an extension might do differently (if the law is not forbidding it). Will address this to CEN TC 434 WG1. @oriol / @phax Please re-add the CEN/TC 434 Discussion needed :-)

svanteschubert commented 1 week ago

Going to drop my prior commits testing for uniqueness as this is already tested by the XSD. Instead, I will test for the same value of BT-7 and BT-8 like the following:

For BT-8:

<param name="CII-SR-461" value="count(ram:ApplicableTradeTax/ram:DueDateTypeCode) = count(ram:ApplicableTradeTax/ram:DueDateTypeCode[text() = '5']) or
count(ram:ApplicableTradeTax/ram:DueDateTypeCode) = count(ram:ApplicableTradeTax/ram:DueDateTypeCode[text() = '29']) or
count(ram:ApplicableTradeTax/ram:DueDateTypeCode) = count(ram:ApplicableTradeTax/ram:DueDateTypeCode[text() = '72'])"/>

For BT-7:

<param name="CII-SR-462" value="count(ram:ApplicableTradeTax/ram:TaxPointDate/udt:DateString) = 0 or ram:ApplicableTradeTax/ram:TaxPointDate/udt:DateString[text() = preceding::ram:ApplicableTradeTax/ram:TaxPointDate/udt:DateString/text()]"/>

Different approaches because the solution for BT-8 is likely faster than the one of BT-7 using the preceding:: axis.

svanteschubert commented 1 week ago

The previous solution for BT-7 to check for a unique value within the document, would not work for 3 or more occurrences of BG-23, when the last two are the same, the following would do:

<param name="CII-SR-462" value="count(ram:ApplicableTradeTax/ram:TaxPointDate/udt:DateString) = count(ram:ApplicableTradeTax/ram:TaxPointDate/udt:DateString[text() = (//ram:ApplicableTradeTax/ram:TaxPointDate/udt:DateString)[1]/text()])"/>