search

NixOS / nixpkgs

Nix Packages collection & NixOS
MIT License
17.47k stars 13.66k forks source link

paperless-ngx pdf import not working #198787

Closed hvraven closed 1 year ago

hvraven commented 1 year ago

Describe the bug

Using unstable paperless-ngx-1.9.2 (on a 22.05 system) I am no longer able to import documents from the web interface as they fail with a MissingDependencyError in unpaper. Sadly parts of the error log appear to be missing, I am not sure how to get the full output.

Steps To Reproduce

Steps to reproduce the behavior:

  1. Configure service.paperless with package = pkgs.unstable.paperless-ngx
  2. Import document from the web interface

Expected behavior

Document should be processed and show up in the document list

Log

Oct 31 15:24:42 nixos paperless-ngx[38670]: [2022-10-31 14:24:42,168] [ERROR] [paperless.consumer] Error while consuming document censored-document-name.pdf: MissingDependencyError: Ran program '/nix/store/s9qywrdw15ihkxw5fwjya47551fs58vh-unpaper-7.0.0/bin/unpaper' but it exited with an error:
Oct 31 15:24:42 nixos paperless-ngx[38670]: Traceback (most recent call last):
Oct 31 15:24:42 nixos paperless-ngx[38670]:   File "/nix/store/dk6m8qsyichiyh4b3v5d5b7kqjjnswgd-python3.10-ocrmypdf-13.7.0/lib/python3.10/site-packages/ocrmypdf/subprocess/__init__.py", line 157, in get_version
Oct 31 15:24:42 nixos paperless-ngx[38670]:     proc = run(
Oct 31 15:24:42 nixos paperless-ngx[38670]:   File "/nix/store/dk6m8qsyichiyh4b3v5d5b7kqjjnswgd-python3.10-ocrmypdf-13.7.0/lib/python3.10/site-packages/ocrmypdf/subprocess/__init__.py", line 58, in run
Oct 31 15:24:42 nixos paperless-ngx[38670]:     proc = subprocess_run(args, env=env, check=check, **kwargs)
Oct 31 15:24:42 nixos paperless-ngx[38670]:   File "/nix/store/fkcl1wzq3106qqgl84bhgk1lp56q6bzg-python3-3.10.7/lib/python3.10/subprocess.py", line 524, in run
Oct 31 15:24:42 nixos paperless-ngx[38670]:     raise CalledProcessError(retcode, process.args,
Oct 31 15:24:42 nixos paperless-ngx[38670]: subprocess.CalledProcessError: Command '['/nix/store/s9qywrdw15ihkxw5fwjya47551fs58vh-unpaper-7.0.0/bin/unpaper', '--version']' died with <Signals.SIGSYS: 31>.
Oct 31 15:24:42 nixos paperless-ngx[38670]: The above exception was the direct cause of the following exception:
Oct 31 15:24:42 nixos paperless-ngx[38670]: Traceback (most recent call last):
Oct 31 15:24:42 nixos paperless-ngx[38670]:   File "/nix/store/za5yxgmxgmf8rdlhni9xg3w7mcwnf9bm-paperless-ngx-1.9.2/lib/paperless-ngx/src/paperless_tesseract/parsers.py", line 277, in parse
Oct 31 15:24:42 nixos paperless-ngx[38670]:     ocrmypdf.ocr(**args)
Oct 31 15:24:42 nixos paperless-ngx[38670]:   File "/nix/store/dk6m8qsyichiyh4b3v5d5b7kqjjnswgd-python3.10-ocrmypdf-13.7.0/lib/python3.10/site-packages/ocrmypdf/api.py", line 339, in ocr
Oct 31 15:24:42 nixos paperless-ngx[38670]:     check_options(options, plugin_manager)
Oct 31 15:24:42 nixos paperless-ngx[38670]:   File "/nix/store/dk6m8qsyichiyh4b3v5d5b7kqjjnswgd-python3.10-ocrmypdf-13.7.0/lib/python3.10/site-packages/ocrmypdf/_validation.py", line 245, in check_options
Oct 31 15:24:42 nixos paperless-ngx[38670]:     _check_plugin_invariant_options(options)
Oct 31 15:24:42 nixos paperless-ngx[38670]:   File "/nix/store/dk6m8qsyichiyh4b3v5d5b7kqjjnswgd-python3.10-ocrmypdf-13.7.0/lib/python3.10/site-packages/ocrmypdf/_validation.py", line 232, in _check_plugin_invariant_options
Oct 31 15:24:42 nixos paperless-ngx[38670]:     check_options_preprocessing(options)
Oct 31 15:24:42 nixos paperless-ngx[38670]:   File "/nix/store/dk6m8qsyichiyh4b3v5d5b7kqjjnswgd-python3.10-ocrmypdf-13.7.0/lib/python3.10/site-packages/ocrmypdf/_validation.py", line 132, in check_options_preprocessing
Oct 31 15:24:42 nixos paperless-ngx[38670]:     check_external_program(
Oct 31 15:24:42 nixos paperless-ngx[38670]:   File "/nix/store/dk6m8qsyichiyh4b3v5d5b7kqjjnswgd-python3.10-ocrmypdf-13.7.0/lib/python3.10/site-packages/ocrmypdf/subprocess/__init__.py", line 331, in check_external_program
Oct 31 15:24:42 nixos paperless-ngx[38670]:     found_version = version_checker()
Oct 31 15:24:42 nixos paperless-ngx[38670]:   File "/nix/store/dk6m8qsyichiyh4b3v5d5b7kqjjnswgd-python3.10-ocrmypdf-13.7.0/lib/python3.10/site-packages/ocrmypdf/_exec/unpaper.py", line 69, in version
Oct 31 15:24:42 nixos paperless-ngx[38670]:     return get_version('/nix/store/s9qywrdw15ihkxw5fwjya47551fs58vh-unpaper-7.0.0/bin/unpaper')
Oct 31 15:24:42 nixos paperless-ngx[38670]:   File "/nix/store/dk6m8qsyichiyh4b3v5d5b7kqjjnswgd-python3.10-ocrmypdf-13.7.0/lib/python3.10/site-packages/ocrmypdf/subprocess/__init__.py", line 173, in get_version
Oct 31 15:24:42 nixos paperless-ngx[38670]:     raise MissingDependencyError(
Oct 31 15:24:42 nixos paperless-ngx[38670]: ocrmypdf.exceptions.MissingDependencyError: Ran program '/nix/store/s9qywrdw15ihkxw5fwjya47551fs58vh-unpaper-7.0.0/bin/unpaper' but it exited with an error:
Oct 31 15:24:42 nixos paperless-ngx[38670]: The above exception was the direct cause of the following exception:
Oct 31 15:24:42 nixos paperless-ngx[38670]: Traceback (most recent call last):
Oct 31 15:24:42 nixos paperless-ngx[38670]:   File "/nix/store/za5yxgmxgmf8rdlhni9xg3w7mcwnf9bm-paperless-ngx-1.9.2/lib/paperless-ngx/src/documents/consumer.py", line 320, in try_consume_file
Oct 31 15:24:42 nixos paperless-ngx[38670]:     document_parser.parse(self.path, mime_type, self.filename)
Oct 31 15:24:42 nixos paperless-ngx[38670]:   File "/nix/store/za5yxgmxgmf8rdlhni9xg3w7mcwnf9bm-paperless-ngx-1.9.2/lib/paperless-ngx/src/paperless_tesseract/parsers.py", line 333, in parse
Oct 31 15:24:42 nixos paperless-ngx[38670]:     raise ParseError(f"{e.__class__.__name__}: {str(e)}") from e
Oct 31 15:24:42 nixos paperless-ngx[38670]: documents.parsers.ParseError: MissingDependencyError: Ran program '/nix/store/s9qywrdw15ihkxw5fwjya47551fs58vh-unpaper-7.0.0/bin/unpaper' but it exited with an error:
Oct 31 15:24:42 nixos paperless-ngx[38670]: 14:24:42 [Q] INFO Process-1:4 stopped doing work
Oct 31 15:24:42 nixos paperless-ngx[38596]: 14:24:42 [Q] ERROR Failed [censored-document-name.pdf] - censored-document-name.pdf: Error while consuming document censored-document-name.pdf: MissingDependencyError: Ran program '/nix/store/s9qywrdw15ihkxw5fwjya47551fs58vh-unpaper-7.0.0/bin/unpaper' but it exited with an error:
Oct 31 15:24:42 nixos paperless-ngx[38596]:  : Traceback (most recent call last):
Oct 31 15:24:42 nixos paperless-ngx[38596]:   File "/nix/store/dk6m8qsyichiyh4b3v5d5b7kqjjnswgd-python3.10-ocrmypdf-13.7.0/lib/python3.10/site-packages/ocrmypdf/subprocess/__init__.py", line 157, in get_version
Oct 31 15:24:42 nixos paperless-ngx[38596]:     proc = run(
Oct 31 15:24:42 nixos paperless-ngx[38596]:   File "/nix/store/dk6m8qsyichiyh4b3v5d5b7kqjjnswgd-python3.10-ocrmypdf-13.7.0/lib/python3.10/site-packages/ocrmypdf/subprocess/__init__.py", line 58, in run
Oct 31 15:24:42 nixos paperless-ngx[38596]:     proc = subprocess_run(args, env=env, check=check, **kwargs)
Oct 31 15:24:42 nixos paperless-ngx[38596]:   File "/nix/store/fkcl1wzq3106qqgl84bhgk1lp56q6bzg-python3-3.10.7/lib/python3.10/subprocess.py", line 524, in run
Oct 31 15:24:42 nixos paperless-ngx[38596]:     raise CalledProcessError(retcode, process.args,
Oct 31 15:24:42 nixos paperless-ngx[38596]: subprocess.CalledProcessError: Command '['/nix/store/s9qywrdw15ihkxw5fwjya47551fs58vh-unpaper-7.0.0/bin/unpaper', '--version']' died with <Signals.SIGSYS: 31>.
Oct 31 15:24:42 nixos paperless-ngx[38596]: The above exception was the direct cause of the following exception:
Oct 31 15:24:42 nixos paperless-ngx[38596]: Traceback (most recent call last):
Oct 31 15:24:42 nixos paperless-ngx[38596]:   File "/nix/store/za5yxgmxgmf8rdlhni9xg3w7mcwnf9bm-paperless-ngx-1.9.2/lib/paperless-ngx/src/paperless_tesseract/parsers.py", line 277, in parse
Oct 31 15:24:42 nixos paperless-ngx[38596]:     ocrmypdf.ocr(**args)
Oct 31 15:24:42 nixos paperless-ngx[38596]:   File "/nix/store/dk6m8qsyichiyh4b3v5d5b7kqjjnswgd-python3.10-ocrmypdf-13.7.0/lib/python3.10/site-packages/ocrmypdf/api.py", line 339, in ocr
Oct 31 15:24:42 nixos paperless-ngx[38596]:     check_options(options, plugin_manager)
Oct 31 15:24:42 nixos paperless-ngx[38596]:   File "/nix/store/dk6m8qsyichiyh4b3v5d5b7kqjjnswgd-python3.10-ocrmypdf-13.7.0/lib/python3.10/site-packages/ocrmypdf/_validation.py", line 245, in check_options
Oct 31 15:24:42 nixos paperless-ngx[38596]:     _check_plugin_invariant_options(options)
Oct 31 15:24:42 nixos paperless-ngx[38596]:   File "/nix/store/dk6m8qsyichiyh4b3v5d5b7kqjjnswgd-python3.10-ocrmypdf-13.7.0/lib/python3.10/site-packages/ocrmypdf/_validation.py", line 232, in _check_plugin_invariant_options
Oct 31 15:24:42 nixos paperless-ngx[38596]:     check_options_preprocessing(options)
Oct 31 15:24:42 nixos paperless-ngx[38596]:   File "/nix/store/dk6m8qsyichiyh4b3v5d5b7kqjjnswgd-python3.10-ocrmypdf-13.7.0/lib/python3.10/site-packages/ocrmypdf/_validation.py", line 132, in check_options_preprocessing
Oct 31 15:24:42 nixos paperless-ngx[38596]:     check_external_program(
Oct 31 15:24:42 nixos paperless-ngx[38596]:   File "/nix/store/dk6m8qsyichiyh4b3v5d5b7kqjjnswgd-python3.10-ocrmypdf-13.7.0/lib/python3.10/site-packages/ocrmypdf/subprocess/__init__.py", line 331, in check_external_program
Oct 31 15:24:42 nixos paperless-ngx[38596]:     found_version = version_checker()
Oct 31 15:24:42 nixos paperless-ngx[38596]:   File "/nix/store/dk6m8qsyichiyh4b3v5d5b7kqjjnswgd-python3.10-ocrmypdf-13.7.0/lib/python3.10/site-packages/ocrmypdf/_exec/unpaper.py", line 69, in version
Oct 31 15:24:42 nixos paperless-ngx[38596]:     return get_version('/nix/store/s9qywrdw15ihkxw5fwjya47551fs58vh-unpaper-7.0.0/bin/unpaper')
Oct 31 15:24:42 nixos paperless-ngx[38596]:   File "/nix/store/dk6m8qsyichiyh4b3v5d5b7kqjjnswgd-python3.10-ocrmypdf-13.7.0/lib/python3.10/site-packages/ocrmypdf/subprocess/__init__.py", line 173, in get_version
Oct 31 15:24:42 nixos paperless-ngx[38596]:     raise MissingDependencyError(
Oct 31 15:24:42 nixos paperless-ngx[38596]: ocrmypdf.exceptions.MissingDependencyError: Ran program '/nix/store/s9qywrdw15ihkxw5fwjya47551fs58vh-unpaper-7.0.0/bin/unpaper' but it exited with an error:
Oct 31 15:24:42 nixos paperless-ngx[38596]: The above exception was the direct cause of the following exception:
Oct 31 15:24:42 nixos paperless-ngx[38596]: Traceback (most recent call last):
Oct 31 15:24:42 nixos paperless-ngx[38596]:   File "/nix/store/28535xrr4xv02r0xlq6b5k9gqy96d9cq-python3.10-asgiref-3.5.2/lib/python3.10/site-packages/asgiref/sync.py", line 280, in main_wrap
Oct 31 15:24:42 nixos paperless-ngx[38596]:     raise exc_info[1]
Oct 31 15:24:42 nixos paperless-ngx[38596]:   File "/nix/store/za5yxgmxgmf8rdlhni9xg3w7mcwnf9bm-paperless-ngx-1.9.2/lib/paperless-ngx/src/documents/consumer.py", line 320, in try_consume_file
Oct 31 15:24:42 nixos paperless-ngx[38596]:     document_parser.parse(self.path, mime_type, self.filename)
Oct 31 15:24:42 nixos paperless-ngx[38596]:   File "/nix/store/za5yxgmxgmf8rdlhni9xg3w7mcwnf9bm-paperless-ngx-1.9.2/lib/paperless-ngx/src/paperless_tesseract/parsers.py", line 333, in parse
Oct 31 15:24:42 nixos paperless-ngx[38596]:     raise ParseError(f"{e.__class__.__name__}: {str(e)}") from e
Oct 31 15:24:42 nixos paperless-ngx[38596]: documents.parsers.ParseError: MissingDependencyError: Ran program '/nix/store/s9qywrdw15ihkxw5fwjya47551fs58vh-unpaper-7.0.0/bin/unpaper' but it exited with an error:
Oct 31 15:24:42 nixos paperless-ngx[38596]: The above exception was the direct cause of the following exception:
Oct 31 15:24:42 nixos paperless-ngx[38596]: Traceback (most recent call last):
Oct 31 15:24:42 nixos paperless-ngx[38596]:   File "/nix/store/0lg74rgck6skkf4p14ivqsk1nb60d5ck-python3.10-django-q-1.3.9/lib/python3.10/site-packages/django_q/cluster.py", line 454, in worker
Oct 31 15:24:42 nixos paperless-ngx[38596]:     res = f(*task["args"], **task["kwargs"])
Oct 31 15:24:42 nixos paperless-ngx[38596]:   File "/nix/store/za5yxgmxgmf8rdlhni9xg3w7mcwnf9bm-paperless-ngx-1.9.2/lib/paperless-ngx/src/documents/tasks.py", line 154, in consume_file
Oct 31 15:24:42 nixos paperless-ngx[38596]:     document = Consumer().try_consume_file(
Oct 31 15:24:42 nixos paperless-ngx[38596]:   File "/nix/store/za5yxgmxgmf8rdlhni9xg3w7mcwnf9bm-paperless-ngx-1.9.2/lib/paperless-ngx/src/documents/consumer.py", line 339, in try_consume_file
Oct 31 15:24:42 nixos paperless-ngx[38596]:     self._fail(
Oct 31 15:24:42 nixos paperless-ngx[38596]:   File "/nix/store/za5yxgmxgmf8rdlhni9xg3w7mcwnf9bm-paperless-ngx-1.9.2/lib/paperless-ngx/src/documents/consumer.py", line 90, in _fail
Oct 31 15:24:42 nixos paperless-ngx[38596]:     raise ConsumerError(f"{self.filename}: {log_message or message}") from exception
Oct 31 15:24:42 nixos paperless-ngx[38596]: documents.consumer.ConsumerError: censored-document-name.pdf: Error while consuming document censored-document-name.pdf: MissingDependencyError: Ran program '/nix/store/s9qywrdw15ihkxw5fwjya47551fs58vh-unpaper-7.0.0/bin/unpaper' but it exited with an error:
Oct 31 15:24:43 nixos paperless-ngx[38594]: 14:24:43 [Q] INFO recycled worker Process-1:4
Oct 31 15:24:43 nixos paperless-ngx[38685]: 14:24:43 [Q] INFO Process-1:5 ready for work at 38685

Notify maintainers

@lukegb, @gador, @erikarvstedt

Metadata

[user@system:~]$ nix-shell -p nix-info --run "nix-info -m"
- system: `"x86_64-linux"`
 - host os: `Linux 5.15.74, NixOS, 22.05 (Quokka), 22.05.3900.26eb67abc9a`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.8.1`
 - channels(hendrik): `""`
 - channels(root): `"nixos-22.05, nixos-hardware, nixpkgs-unstable"`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`
gador commented 1 year ago

I just tested it on the current master as well as nixos-unstable. Both work just fine. I suspect it has something to do with you runnung 22.05 and using a package from the unstable branch. I've seen weird errors when mixing those (e.g. with pgadmin here).

gador commented 1 year ago

@hvraven is this still an issue?

Atemu commented 1 year ago

Swapping out the executable with one from a different release is not supported.