search

edissyum / opencapture

Open-Capture is the one and only 100% Open Source intelligent document capture solution
https://kutt.it/OpenCaptureWebSite
GNU General Public License v3.0
24 stars 11 forks source link

Erro 500 when validate an invoice #149

Closed gportail closed 2 years ago

gportail commented 2 years ago

Hi,

I built a simple form. In "Verification" modul, when I validate an invoice, I've got an "Error 500". Nothing in OCforInvoices.log but in /var/log/apache2/error.log I get :

[Mon May 09 18:49:18.470790 2022] [wsgi:error] [pid 536:tid 139816381544192] [remote 192.168.42.10:2237] [2022-05-09 18:49:18,470] ERROR in app: Exception on /ws/verifier/invoices/1/export_xml [POST]
[Mon May 09 18:49:18.470819 2022] [wsgi:error] [pid 536:tid 139816381544192] [remote 192.168.42.10:2237] Traceback (most recent call last):
[Mon May 09 18:49:18.470826 2022] [wsgi:error] [pid 536:tid 139816381544192] [remote 192.168.42.10:2237]   File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 2077, in wsgi_app
[Mon May 09 18:49:18.470831 2022] [wsgi:error] [pid 536:tid 139816381544192] [remote 192.168.42.10:2237]     response = self.full_dispatch_request()
[Mon May 09 18:49:18.470836 2022] [wsgi:error] [pid 536:tid 139816381544192] [remote 192.168.42.10:2237]   File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 1525, in full_dispatch_request
[Mon May 09 18:49:18.470842 2022] [wsgi:error] [pid 536:tid 139816381544192] [remote 192.168.42.10:2237]     rv = self.handle_user_exception(e)
[Mon May 09 18:49:18.470846 2022] [wsgi:error] [pid 536:tid 139816381544192] [remote 192.168.42.10:2237]   File "/usr/local/lib/python3.9/dist-packages/flask_cors/extension.py", line 165, in wrapped_function
[Mon May 09 18:49:18.470851 2022] [wsgi:error] [pid 536:tid 139816381544192] [remote 192.168.42.10:2237]     return cors_after_request(app.make_response(f(*args, **kwargs)))
[Mon May 09 18:49:18.470856 2022] [wsgi:error] [pid 536:tid 139816381544192] [remote 192.168.42.10:2237]   File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 1523, in full_dispatch_request
[Mon May 09 18:49:18.470860 2022] [wsgi:error] [pid 536:tid 139816381544192] [remote 192.168.42.10:2237]     rv = self.dispatch_request()
[Mon May 09 18:49:18.470865 2022] [wsgi:error] [pid 536:tid 139816381544192] [remote 192.168.42.10:2237]   File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 1509, in dispatch_request
[Mon May 09 18:49:18.470870 2022] [wsgi:error] [pid 536:tid 139816381544192] [remote 192.168.42.10:2237]     return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args)
[Mon May 09 18:49:18.470874 2022] [wsgi:error] [pid 536:tid 139816381544192] [remote 192.168.42.10:2237]   File "/var/www/html/opencaptureforinvoices/src/backend/controllers/auth.py", line 106, in wrapped_view
[Mon May 09 18:49:18.470879 2022] [wsgi:error] [pid 536:tid 139816381544192] [remote 192.168.42.10:2237]     return view(**kwargs)
[Mon May 09 18:49:18.470884 2022] [wsgi:error] [pid 536:tid 139816381544192] [remote 192.168.42.10:2237]   File "/var/www/html/opencaptureforinvoices/src/backend/rest/verifier.py", line 90, in export_xml
[Mon May 09 18:49:18.470889 2022] [wsgi:error] [pid 536:tid 139816381544192] [remote 192.168.42.10:2237]     res = verifier.export_xml(invoice_id, data)
[Mon May 09 18:49:18.470894 2022] [wsgi:error] [pid 536:tid 139816381544192] [remote 192.168.42.10:2237]   File "/var/www/html/opencaptureforinvoices/src/backend/controllers/verifier.py", line 545, in export_xml
[Mon May 09 18:49:18.470898 2022] [wsgi:error] [pid 536:tid 139816381544192] [remote 192.168.42.10:2237]     xml_file.write(xml_root)
[Mon May 09 18:49:18.470903 2022] [wsgi:error] [pid 536:tid 139816381544192] [remote 192.168.42.10:2237] UnicodeEncodeError: 'ascii' codec can't encode character '\\xe9' in position 415: ordinal not in range(128)

I think it's because I have a "é" in the name of supplier. ( \xe9 = é ) I my form, supplier is "Alphanumeric extended with accent". Even if I remove the "é", I continue to get an error 500

nathan30 commented 2 years ago

Hi,

Which version do you use ?

Also, can you give us a screenshot of the filled form to try to reproduce the issue please ?

gportail commented 2 years ago

I use version 2.2.2.

here is a screenshot of my simple form

Open-Capture For Invoices

nathan30 commented 2 years ago

I try to reproduce the error and I didn't have any issues while exporting the XML with the supplier name including accent. Could you try to update your installation (with 2.2.3) and try again ?

gportail commented 2 years ago

It's worse. I took my pre-test snapshot and updated to 2.2.3. I loaded the same invoice...it doesn't appear in today's batches! nor in the older ones. However, the logs show that it has been processed.

Log OpenCapture :

[Open-Capture     ] [OCForInvoices.py          line 123] 17-05-2022 17:03:41 INFO Processing file : /var/www/html/opencaptureforinvoices/instance/upload/verifier/11Ob_fact_CCR.pdf
[Open-Capture     ] [FindSupplier.py           line 188] 17-05-2022 17:03:53 INFO No supplier informations found in the header, improve image and retry...
[Open-Capture     ] [FindSupplier.py           line 196] 17-05-2022 17:03:56 INFO No supplier informations found with improved image, try with footer...
[Open-Capture     ] [FindSupplier.py           line 108] 17-05-2022 17:03:56 INFO Supplier found : Couverture Charpente Réunies using VAT Number : FR18382116952
[Open-Capture     ] [FindInvoiceNumber.py      line 87 ] 17-05-2022 17:03:56 INFO Invoice number found : 151212
[Open-Capture     ] [FindDate.py               line 170] 17-05-2022 17:03:56 INFO Invoice date found : 17/12/2015

I had difficulties with the update. OC could not connect to the database for identification. On the 2nd attempt and an OS upgrade, it worked.

nathan30 commented 2 years ago

The LOG should show "Process end after XXX". If not, please check the status of the workers (see documentation : https://edissyum.gitbook.io/open-capture-for-invoices-v2/francais/gestion-des-erreurs)

By the way, we had issue when launching the 2.2.3. If you install it before today you could do the following to update to latest : 

cd /var/www/html/opencaptureforinvoices/
git fetch origin --tags --force
git checkout 2.2.3
gportail commented 2 years ago

Hi!

After reset to last snapshot (v 2.2.2) I resart update.sh then it work.

I see an error during update :

Certains paquets ne peuvent être installés. Ceci peut signifier
que vous avez demandé l'impossible, ou bien, si vous utilisez
la distribution unstable, que certains paquets n'ont pas encore
été créés ou ne sont pas sortis d'Incoming.
L'information suivante devrait vous aider à résoudre la situation :

Les paquets suivants contiennent des dépendances non satisfaites :
 libpq-dev : Dépend: libpq5 (= 13.7-0+deb11u1) mais 14.2-1.pgdg110+1 devra être installé
E: Impossible de corriger les problèmes, des paquets défectueux sont en mode « garder en l'état ».
Requirement already satisfied: pip in /usr/local/lib/python3.9/dist-packages (22.0.4)

After added my invoice, I check services, I think there is an error in OCForInvoices-worker.service here are result of status of all services:

root@opencapture:~# systemctl status OCForInvoices-worker.service
● OCForInvoices-worker.service - Daemon for Open-Capture for Invoices
     Loaded: loaded (/etc/systemd/system/OCForInvoices-worker.service; enabled; vendor preset: enabled)
     Active: active (running) since Fri 2022-05-20 23:02:57 CEST; 4min 50s ago
   Main PID: 1723 (service_workerO)
      Tasks: 4 (limit: 4679)
     Memory: 135.7M
        CPU: 23.048s
     CGroup: /system.slice/OCForInvoices-worker.service
             ├─1723 /bin/bash /var/www/html/opencaptureforinvoices/bin/scripts/service_workerOC.sh
             └─1725 /usr/bin/python3 /usr/local/bin/kuyruk --app src.backend.main.OCforInvoices_worker worker --queue invoices

mai 20 23:05:31 opencapture service_workerOC.sh[1725]:     return task.apply(*args, **kwargs)
mai 20 23:05:31 opencapture service_workerOC.sh[1725]:   File "/usr/local/lib/python3.9/dist-packages/kuyruk/task.py", line 179, in apply
mai 20 23:05:31 opencapture service_workerOC.sh[1725]:     return self.f(*args, **kwargs)
mai 20 23:05:31 opencapture service_workerOC.sh[1725]:   File "/var/www/html/opencaptureforinvoices/src/backend/main.py", line 292, in launch
mai 20 23:05:31 opencapture service_workerOC.sh[1725]:     res = OCForInvoices_process.process(args, path, log, config, files, ocr, regex, database, docservers, configurations, languages)
mai 20 23:05:31 opencapture service_workerOC.sh[1725]:   File "/var/www/html/opencaptureforinvoices/src/backend/process/OCForInvoices.py", line 277, in process
mai 20 23:05:31 opencapture service_workerOC.sh[1725]:     quotation_number = quotation_number_class.run()
mai 20 23:05:31 opencapture service_workerOC.sh[1725]:   File "/var/www/html/opencaptureforinvoices/src/backend/process/FindQuotationNumber.py", line 84, in run
mai 20 23:05:31 opencapture service_workerOC.sh[1725]:     for _invoice in re.finditer(r"" + self.regex['quotationRegex'] + "", line.content.upper()):
mai 20 23:05:31 opencapture service_workerOC.sh[1725]: KeyError: 'quotationRegex'

root@opencapture:~# systemctl status OCForInvoices_Split-worker
● OCForInvoices_Split-worker.service - Splitter Daemon for Open-Capture for Invoices
     Loaded: loaded (/etc/systemd/system/OCForInvoices_Split-worker.service; enabled; vendor preset: enabled)
     Active: active (running) since Fri 2022-05-20 23:02:57 CEST; 6min ago
   Main PID: 1754 (service_workerO)
      Tasks: 2 (limit: 4679)
     Memory: 75.3M
        CPU: 1.122s
     CGroup: /system.slice/OCForInvoices_Split-worker.service
             ├─1754 /bin/bash /var/www/html/opencaptureforinvoices/bin/scripts/service_workerOC_splitter.sh
             └─1755 /usr/bin/python3 /usr/local/bin/kuyruk --app src.backend.main_splitter.OCforInvoices worker --queue splitter

mai 20 23:02:57 opencapture systemd[1]: Started Splitter Daemon for Open-Capture for Invoices.
mai 20 23:02:58 opencapture service_workerOC_splitter.sh[1755]: I kuyruk.kuyruk.connection:85 - Connected to RabbitMQ
mai 20 23:02:58 opencapture service_workerOC_splitter.sh[1755]: I kuyruk.kuyruk.channel:60 - Opened new channel
mai 20 23:02:58 opencapture service_workerOC_splitter.sh[1755]: I kuyruk.worker._consume_messages:139 - Consumer started

root@opencapture:~# systemctl status rabbitmq-server.service
● rabbitmq-server.service - RabbitMQ Messaging Server
     Loaded: loaded (/lib/systemd/system/rabbitmq-server.service; enabled; vendor preset: enabled)
     Active: active (running) since Fri 2022-05-20 23:01:11 CEST; 9min ago
   Main PID: 354 (beam.smp)
     Status: "Initialized"
      Tasks: 86 (limit: 4679)
     Memory: 118.2M
        CPU: 10.028s
     CGroup: /system.slice/rabbitmq-server.service
             ├─354 /usr/lib/erlang/erts-11.1.8/bin/beam.smp -W w -K true -A 64 -MBas ageffcbf -MHas ageffcbf -MBlmbcs 512 -MHlmbcs 512 -MMmcs 30 -P 1048576 -t 5000000 -stbt db -zdbbl 128000 -- -root /usr/lib/e>
             ├─522 erl_child_setup 65536
             ├─814 inet_gethost 4
             └─815 inet_gethost 4

mai 20 23:01:01 opencapture systemd[1]: Starting RabbitMQ Messaging Server...
mai 20 23:01:11 opencapture systemd[1]: Started RabbitMQ Messaging Server.
nathan30 commented 2 years ago

Hi,

We are currently working to improve the invoice process by adding a SQL update script between version. Here you are missing the new regex for quotation number. You can find the SQL INSERT in instance/sql/global.sql