search

wes4m / zatca-xml-js

An implementation of Saudi Arabia ZATCA's E-Invoicing requirements, processes, and standards in TypeScript.
MIT License
68 stars 58 forks source link

NEW VERSION OF SANDBOX (ISSUES AFTER THAT) #25

Closed asim009 closed 1 year ago

asim009 commented 1 year ago

@wes4m I have been working on this package since very long time and everything was good and running fine. Yesterday, I think zatca has updated the sandbox because of the there was no service available till today. Below message was in response everytime when I hit the API.

<html>
      <head><title>503 Service Temporarily Unavailable</title></head>
      <body>
          <center><h1>503 Service Temporarily Unavailable</h1></center>
          <hr><center>nginx</center>
      </body>
</html>

Now that the server is up and running again. I tried to execute the same request and I am facing below errors 

{
  "infoMessages": [
      {
          "type": "INFO",
          "code": "XSD_ZATCA_VALID",
          "category": "XSD validation",
          "message": "Complied with UBL 2.1 standards in line with ZATCA specifications",
          "status": "PASS"
      }
  ],
  "warningMessages": [],
  "errorMessages": [
      {
          "type": "ERROR",
          "code": "GENERAL",
          "category": "BUSINESS_RULES",
          "message": "Unable to execute Business Rules validation",
          "status": "ERROR"
      },
      {
          "type": "ERROR",
          "code": "BR-KSA-EN16931-11",
          "category": "KSA",
          "message": "Invoice line net amount (BT-131) must equal (Invoiced quantity (BT-130) * (Item net price (BT-146) / item price base quantity (BT-149))-))- Sum of invoice line allowance amount (BT-136)",
          "status": "ERROR"
      },
      {
          "type": "ERROR",
          "code": "BR-KSA-EN16931-06",
          "category": "KSA",
          "message": "Charge on price level (BG-29) is NOT allowed. Only value 'false' allowed.",
          "status": "ERROR"
      },
      {
          "type": "ERROR",
          "code": "BR-KSA-EN16931-06",
          "category": "KSA",
          "message": "Charge on price level (BG-29) is NOT allowed. Only value 'false' allowed.",
          "status": "ERROR"
      }
  ],
  "status": "ERROR"
}

Don't understand what went wrong. After some research I see that it is related to lineitem Discount 

"discounts": [
    { "amount": 2, "reason": "A discount" },
    { "amount": 2, "reason": "A second discount" }
]

I think there is some miscalculation while calculating the discounts or some new features they have updated in this publish Strange is that when I don't pass any discount the Invoice is getting reported.

asim009 commented 1 year ago

Request sample which through error

{
"uuid": "6f4d20e0-6bfe-4a80-9389-7dabe6620f12",
"custom_id": "EGS1-886431145",
"model": "IOS",
"CRN_number": "454634645645654",
"VAT_name": "Wesam Alzahir",
"VAT_number": "301121971500003",
"location": {
"city": "Khobar",
"city_subdivision": "West",
"street": "King Fahahd st",
"plot_identification": "0000",
"building": "0000",
"postal_zone" : "31952"
},
"branch_name": "My Branch Name",
"branch_industry": "Food",
"invoice_counter_number": "1",
"invoice_serial_number": "EGS1-886431145-1",
"issue_date": "2022-03-13",
"issue_time": "14:40:40",
"previous_invoice_hash": "NWZlY2ViNjZmZmM4NmYzOGQ5NTI3ODZjNmQ2OTZjNzljMmRiYzIzOWRkNGU5MWI0NjcyOWQ3M2EyN2ZiNTdlOQ==",
"line_items": [
{
        "id": "1",
        "name": "TEST NAME",
        "quantity": 5,
        "tax_exclusive_price": 10,
        "VAT_percent": 0.15,
        "other_taxes": [
        ],
        "discounts": [
            { "amount": 1, "reason": "A discount" },
            { "amount": 1, "reason": "A second discount" }
        ]
}
]
}

Request successful Reported

{
"uuid": "6f4d20e0-6bfe-4a80-9389-7dabe6620f12",
"custom_id": "EGS1-886431145",
"model": "IOS",
"CRN_number": "454634645645654",
"VAT_name": "Wesam Alzahir",
"VAT_number": "301121971500003",
"location": {
"city": "Khobar",
"city_subdivision": "West",
"street": "King Fahahd st",
"plot_identification": "0000",
"building": "0000",
"postal_zone" : "31952"
},
"branch_name": "My Branch Name",
"branch_industry": "Food",
"invoice_counter_number": "1",
"invoice_serial_number": "EGS1-886431145-1",
"issue_date": "2022-03-13",
"issue_time": "14:40:40",
"previous_invoice_hash": "NWZlY2ViNjZmZmM4NmYzOGQ5NTI3ODZjNmQ2OTZjNzljMmRiYzIzOWRkNGU5MWI0NjcyOWQ3M2EyN2ZiNTdlOQ==",
"line_items": [
{
        "id": "1",
        "name": "TEST NAME",
        "quantity": 5,
        "tax_exclusive_price": 10,
        "VAT_percent": 0.15,
        "other_taxes": [
        ],
        "discounts": [
        ]
}
]
}
wes4m commented 1 year ago

@asim009 Ah nice one ZATCA

Will have to see what new fucked up calculation changed now ..

asim009 commented 1 year ago

Yes we have to see !:-(

asim009 commented 1 year ago

calculation error when calculate as per the formulae of zatca

{
  "type": "ERROR",
  "code": "BR-KSA-EN16931-11",
  "category": "KSA",
  "message": "Invoice line net amount (BT-131) must equal (Invoiced quantity (BT-130) * (Item net price (BT-146) / item price base quantity (BT-149))-))- Sum of invoice line allowance amount (BT-136)",
  "status": "ERROR"
}

BT-131=(BT-130 (BT-146 / BT-149))-BT-136 48=(5(48/5))-2 value of (5*(48/5))-2 is 46 the result is 48=46

**Note values taken as per the above example of json which has errors

HaimenToshi commented 1 year ago

Hi. Strangely I don't get any calculation errors, the only error that is popping is the BUSINESS RULES one. I'm building the XML manually from the ground up (not using the canonalizer or the purifier functions of this package).

wes4m commented 1 year ago

Did you include discounts in the XML ?

HaimenToshi commented 1 year ago

@wes4m Yes, a general discount on the whole invoice, not per item.

wes4m commented 1 year ago

I tried changing discounts from per item method to the whole invoice instead, but this results in a useless error

{
  "validationResults": {
    "infoMessages": [
      {
        "type": "INFO",
        "code": "XSD_ZATCA_VALID",
        "category": "XSD validation",
        "message": "Complied with UBL 2.1 standards in line with ZATCA specifications",
        "status": "PASS"
      }
    ],
    "warningMessages": [],
    "errorMessages": [
      {
        "type": "ERROR",
        "code": "GENERAL",
        "category": "BUSINESS_RULES",
        "message": "Unable to execute Business Rules validation",
        "status": "ERROR"
      }
    ],
    "status": "ERROR"
  },
  "reportingStatus": "NOT_REPORTED",
  "clearanceStatus": null,
  "qrSellertStatus": null,
  "qrBuyertStatus": null
}

Any ideas ?

HaimenToshi commented 1 year ago

@wes4m This is exactly what I'm getting right now. Maybe it is an issue on the API end. I suggest waiting a few days before attempting to fix things on our end, hopefully ZATCA will fix their API.

wes4m commented 1 year ago

Yeah, I will ignore this for a few days instead and hope they fix it. looking forward to them doing this type of stuff during production

HaimenToshi commented 1 year ago

@wes4m Oh they will keep f*cking things up even when Phase 2 goes live, as they enjoy keeping us on our toes.

asim009 commented 1 year ago

@HaimenToshi @wes4m How can we post this bug to them any idea?

HaimenToshi commented 1 year ago

@wes4m ZATCA fixed the "BUSINESS RULES" issue.

wes4m commented 1 year ago

@HaimenToshi nice. good suggestion on waiting and not do any changes. Example is working fine again with discount @asim009.

> npm run example
{
  validationResults: {
    infoMessages: [ [Object] ],
    warningMessages: [],
    errorMessages: [],
    status: 'PASS'
  },
  reportingStatus: 'REPORTED',
  clearanceStatus: null,
  qrSellertStatus: null,
  qrBuyertStatus: null
}
wes4m commented 1 year ago

@HaimenToshi @wes4m How can we post this bug to them any idea?

They have a developer support ticketing portal. I don't have access to it, and not sure how to get access to it. I know someone who uses it though, will check with him about it.

adriantoro26 commented 1 year ago

Excellent news, Also I wanted to ask you about this error I'm getting when validating the signed xml with Zatca's web based validator. I'm pretty sure it is related to the expired certificate we receive from them using the production CSID API:

Validation XML

invoice.xml

Valid : false

  category : SIGNATURE_ERROR
  code :X509SerialNumber
  message : wrong X509SerialNumber 

  category : SIGNATURE_ERROR
  code :signatureValue
  message : wrong signature Value

I checked the signing process used on this code and it is solid and I verified throughly the whole process on my end:

Lastly I personal doubt when using crypto.createSign and we select "SHA-256" that is only the hashing method but not the algorithm for signing (i.e., ECDSA). When selecting "SHA-256" are we using ECDSA with SHA-256?

Because when using secp256k1 package and the same generated private key the length of the signature was exactly 64 bytes which the right length but the number of bytes from crypto is always 70 bytes.

adriantoro26 commented 1 year ago

@adriantoro26 Web based validator is broken and uses hardcoded cert. Even their sample signed invoice provided with latest SDK does not pass it. It shows same error you're getting.

Thank you very much! I imagined it. Also this is a personal doubt of mine just for learning purposes: when using crypto.createSign and we select "SHA-256" that is only the hashing method but not the algorithm for signing (i.e., ECDSA). When selecting "SHA-256" are we using ECDSA with SHA-256?

Because when using secp256k1 package and the same generated private key the length of the signature was exactly 64 bytes which the right length but the number of bytes from crypto is always 70 bytes.

wes4m commented 1 year ago

@adriantoro26 Web based validator is broken and uses hardcoded cert. Even their sample signed invoice provided with latest SDK does not pass it. It shows same error you're getting.

Thank you very much! I imagined it. Also this is a personal doubt of mine just for learning purposes: when using crypto.createSign and we select "SHA-256" that is only the hashing method but not the algorithm for signing (i.e., ECDSA). When selecting "SHA-256" are we using ECDSA with SHA-256?

Because when using secp256k1 package and the same generated private key the length of the signature was exactly 64 bytes which the right length but the number of bytes from crypto is always 70 bytes.

NodeJS crypto will use ECDSA by default for SHA256 digest, according to the docs https://nodejs.org/api/crypto.html#cryptocreatesignalgorithm-options

Unless specified otherwise like createSign('RSA-SHA256')

Answer here probably explains the byte length difference. I didn't research it enough. https://stackoverflow.com/questions/39554165/ecdsa-signatures-between-node-js-and-webcrypto-appear-to-be-incompatible

Would be helpful If someone can verify. However, so far it works fine with the API

asim009 commented 1 year ago

sandbox is again down!!

wes4m commented 1 year ago

sandbox is again down!!

Maybe I should make a tracker web hook for API status that tests every X minutes or something

asim009 commented 1 year ago

@wes4m I want to work on Standard Invoices by using the same library. What will be challenges? How do you want me to start ?

adriantoro26 commented 1 year ago

@wes4m @asim009 Is there any forum to be up-to-date with any changes zatca does?

farheenagithub commented 1 year ago

I am getting e [AxiosError: Request failed with status code 400] { code: 'ERR_BAD_REQUEST', config: { transitional: { silentJSONParsing: true, forcedJSONParsing: true, clarifyTimeoutError: false }, adapter: [Function: httpAdapter], transformRequest: [ [Function: transformRequest] ], transformResponse: [ [Function: transformResponse] ], timeout: 0, xsrfCookieName: 'XSRF-TOKEN', xsrfHeaderName: 'X-XSRF-TOKEN', maxContentLength: -1, maxBodyLength: -1, env: { FormData: [Function] }, validateStatus: [Function: validateStatus], ................ for this call 'POST /e-invoicing/developer-portal/invoices/reporting/single HTTP/1.1\r\n'

The json is the same example one. Any directions?

asim009 commented 1 year ago

@adriantoro26 No I haven't came across @farheenagithub Zatca request url are secured. See that the request should be over HTTPS and not HTTP.

BrainDamageCRO commented 1 year ago

Hello everyone,

First question Has any of you succeeded to create production certificate? We are facing issue with this but we think it's sandbox issue, so I was wondering are others facing the same issue.

Second question Has any of you done anything about Tobacco fee? Since there is some special regulation around that and there aren't any examples on that matter, so I was wondering if someone has any experience with it. Would be helpful a lot.

Third question I've seen some comments in other issue, that there is some talking about postponing the production for 3-4 months, so I was wondering where that information comes from since we didn't find any information regarding that and right now there is only 10 days until production and they still haven't put the production URL online for us to prepare it for our production machines.

Forth question We aren't able to send multiple payments, if someone has an XML example with multiple payments I would appreciate that. Of course if that is even possible to do.

I assume I won't get all the answers but I think it's worth a try.

Thanks all in advance!

farheenagithub commented 1 year ago

Detailed error response coming with 400 : BAD REQUEST

{ type: 'ERROR', code: 'certificate-permissions', category: 'CERTIFICATE_ERRORS', message: 'Production CSID does not cover Simplified documents', status: 'ERROR' }, { type: 'ERROR', code: 'certificate-permissions', category: 'CERTIFICATE_ERRORS', message: 'User only allowed to use the vat number that exists in the authentication certificate', status: 'ERROR' }

asim009 commented 1 year ago

I am in phase of implementing Standard invoice with in this library and reached to an extend! below are some of my observation which will help others in implementation of standard invoices As of now, 1) In production only standard invoice are getting reported. 2) Clearance is deactivated there is a message from zatca which says Clearance is deactiviated. Please use the /invoices/reporting/single endpoint instead.

This library only support simplified Invoice (Reporting) and for Standard Invoice (Clearance) one have to make the necessary changes to xml as per the standard of zatca.

@wes4m can you please create one dummy (standard Invoice) template under templates folder. So that I can use that template and integrate standard invoices.

@farheenagithub Production CSID only cover standard invoice.

adriantoro26 commented 1 year ago
  1. In production only standard invoice are getting reported.

So basically the clearance of standard invoices can't be done, and you used the reporting API to report an standard invoice?. Yes, I think we can tackle standard invoices, @asim009 I can also help on this matter.

asim009 commented 1 year ago

So basically the clearance of standard invoices can't be done, and you used the reporting API to report an standard invoice? YES FOR NOW I think we can tackle standard invoices, @asim009 I can also help on this matter. I have reached to an extend. I am weak in understanding how to use git repos pull and push, I have cloned the repo. what next?

farheenagithub commented 1 year ago

"// Report invoice production // Note: This request currently fails because ZATCA sandbox returns a constant fake production certificate"

What does this note mean in the example, can you please explain. How will the reporting get success?

wes4m commented 1 year ago

Hey sorry guys I'm a bit busy these days. For anyone working on standard invoice. I would say look at the ZATCASimplifiedInvoice.ts and start from there. @asim009 you need to fork the repo, make a new branch to do your work then try to send a pull request. Just google the process.

Maybe I will make a template when I have the time, probably not before the deadline though.

wes4m commented 1 year ago

Hello everyone,

Second question Has any of you done anything about Tobacco fee? Since there is some special regulation around that and there aren't any examples on that matter, so I was wondering if someone has any experience with it. Would be helpful a lot.

What we did before phase two was to just add an extra 100% TAX on the invoice, or item. To accommodate for this, I added the option in this lib to add additional taxes. If you look at the example there is an extra tax with percent = 1 (100%)

Not sure about other questions.

asim009 commented 1 year ago

"// Report invoice production // Note: This request currently fails because ZATCA sandbox returns a constant fake production certificate"

What does this note mean in the example, can you please explain. How will the reporting get success? In production only standard invoice are getting reported. Clearance is deactivated there is a message from zatca which says Clearance is deactiviated. Please use the /invoices/reporting/single endpoint instead.

Simplified Invoices are not getting reported and zatca returns constant fake production certificate even I am not sure of what is that. But, for Standard Invoices if you are using this package you need to implement standard invoice xml.

asim009 commented 1 year ago

calculation error when calculate as per the formulae of zatca

{
  "type": "ERROR",
  "code": "BR-KSA-EN16931-11",
  "category": "KSA",
  "message": "Invoice line net amount (BT-131) must equal (Invoiced quantity (BT-130) * (Item net price (BT-146) / item price base quantity (BT-149))-))- Sum of invoice line allowance amount (BT-136)",
  "status": "ERROR"
}

BT-131=(BT-130 (BT-146 / BT-149))-BT-136 48=(5(48/5))-2 value of (5*(48/5))-2 is 46 the result is 48=46

**Note values taken as per the above example of json which has errors

--------------------------------------------------------_____---------------------- @wes4m @HaimenToshi @adriantoro26 There is an error from zatca when I tried to execute below calculation "message": "The line VAT amount (KSA-11) must be Invoice line net amount (BT-131) x (Line VAT rate (BT-152)/100).", 3000.07=20000.50*15/100;> 3000.075 (calculator value) I assume Zatca calculation is rounding up the value to 3000.075 to 3000.08. I tried to round up the thing with in this package but entire calculation is getting messed up. Suggest me what we can do

Test case Json for line 

"line_items": [
        {
            "id": "1",
            "name": "NPR75L-KL5VAYSN",
            "quantity": 1,
            "Sales_unit": "UN",
            "tax_exclusive_price": 20000.5,
            "net_price": 20000.5,
            "line_amount_without_tax": 20000.5,
            "line_amount_withtax": 23000.575,
            "advance%": 0,
            "advance_amount": 0,
            "advance_vat": 0,
            "advance_amount_after_tax": 0,
            "model_year": "",
            "tax_level1": "SR15",
            "VAT_percent": 0.15,
            "other_taxes": [],
            "discounts": [
                {
                    "amount": 0,
                    "reason": 0
                },
                {
                    "amount": 0,
                    "reason": 0
                }
            ]
        }
    ]
wes4m commented 1 year ago

calculation error when calculate as per the formulae of zatca

{
  "type": "ERROR",
  "code": "BR-KSA-EN16931-11",
  "category": "KSA",
  "message": "Invoice line net amount (BT-131) must equal (Invoiced quantity (BT-130) * (Item net price (BT-146) / item price base quantity (BT-149))-))- Sum of invoice line allowance amount (BT-136)",
  "status": "ERROR"
}

BT-131=(BT-130 (BT-146 / BT-149))-BT-136 48=(5(48/5))-2 value of (5*(48/5))-2 is 46 the result is 48=46 **Note values taken as per the above example of json which has errors

--------------------------------------------------------_____---------------------- @wes4m @HaimenToshi @adriantoro26 There is an error from zatca when I tried to execute below calculation "message": "The line VAT amount (KSA-11) must be Invoice line net amount (BT-131) x (Line VAT rate (BT-152)/100).", 3000.07=20000.50*15/100;> 3000.075 (calculator value) I assume Zatca calculation is rounding up the value to 3000.075 to 3000.08. I tried to round up the thing with in this package but entire calculation is getting messed up. Suggest me what we can do

Test case Json for line

"line_items": [
       {
           "id": "1",
           "name": "NPR75L-KL5VAYSN",
           "quantity": 1,
           "Sales_unit": "UN",
           "tax_exclusive_price": 20000.5,
           "net_price": 20000.5,
           "line_amount_without_tax": 20000.5,
           "line_amount_withtax": 23000.575,
           "advance%": 0,
           "advance_amount": 0,
           "advance_vat": 0,
           "advance_amount_after_tax": 0,
           "model_year": "",
           "tax_level1": "SR15",
           "VAT_percent": 0.15,
           "other_taxes": [],
           "discounts": [
               {
                   "amount": 0,
                   "reason": 0
               },
               {
                   "amount": 0,
                   "reason": 0
               }
           ]
       }
   ]

Please post this in another issue to make tracking easier.

asim009 commented 1 year ago

@ameenpv only wait will solve this problem as the continuously updating there system.