search

MatthiasValvekens / pyHanko

pyHanko: sign and stamp PDF files
MIT License
460 stars 68 forks source link

Support of 64bit PKCS#11 libraries (drivers) #373

Closed KingCZE closed 5 months ago

KingCZE commented 5 months ago

Hi, using pyHanko version 0.21.0. As of now, there is unfortunately no support of 64bit PKCS#11 drivers in pyHanko.

RuntimeError: Cannot open library at C:/Windows/SysWOW64/eop2v1czep11.dll: C:/Windows/SysWOW64/eop2v1czep11.dll is not a valid Win32 application.

The Czech eID (as shown above) fortunately has a 32bit version of the PKCS#11 driver (which works fine), but the problem is that some qualified and commercial certificate issuers (e.g. Komerční banka and their C:\Windows\SysWOW64\mopkcs11.dll) do not even have 32bit versions of their PKCS#11 drivers anymore.

I understand that at least the functionality of the 32bit versions are the priority, but the absence of support of the 64bit PKCS11 libraries makes it unusable for many people and it is really unfortunate.

MatthiasValvekens commented 5 months ago

This is not a pyHanko issue, but almost certainly a Python runtime issue. You can't use 64-bit DLLs with a 32-bit runtime, (and probably vice versa). Are you sure you haven't installed a 32-bit Python on a 64-bit machine?

Either way, the PKCS#11 wiring in pyHanko is completely platform- and architecture-agnostic. If you think you have found a genuine incompatibility, you should file an issue with the author of the Cython extension that pyHanko uses to talk to PKCS#11 interfaces: https://github.com/pyauth/python-pkcs11. But I doubt that that's it, because I've been able to use 64-bit PKCS#11 modules without issue (albeit I haven't personally tested them on Windows).

KingCZE commented 5 months ago

Are you sure you haven't installed a 32-bit Python on a 64-bit machine?

I installed it with python 3.11 on conda from conda-forge that provides 64-bit packages and 64-bit Python. (I was unable to install it with python 3.12 that was as default on my machine)

I am fairly confident that I built everything with 64-bit python:

(myenv64bit) C:\pyHanko\python-pkcs11>python -m pip install dist/python_pkcs11-0.7.1.dev40+gff20b84-cp311-cp311-win_amd64.whl
Processing c:\pyhanko\python-pkcs11\dist\python_pkcs11-0.7.1.dev40+gff20b84-cp311-cp311-win_amd64.whl
Collecting asn1crypto>=1.0.0 (from python-pkcs11==0.7.1.dev40+gff20b84)
  Using cached asn1crypto-1.5.1-py2.py3-none-any.whl (105 kB)
Collecting cached-property (from python-pkcs11==0.7.1.dev40+gff20b84)
  Using cached cached_property-1.5.2-py2.py3-none-any.whl (7.6 kB)
Installing collected packages: cached-property, asn1crypto, python-pkcs11
Successfully installed asn1crypto-1.5.1 cached-property-1.5.2 python-pkcs11-0.7.1.dev40+gff20b84
...

...
(myenv64bit) C:\pyHanko\pyHanko>pip install pyHanko[pkcs11,image-support,opentype,xmp]
Collecting pyHanko[image-support,opentype,pkcs11,xmp]
  Using cached pyHanko-0.21.0-py3-none-any.whl.metadata (9.4 kB)
Requirement already satisfied: asn1crypto>=1.5.1 in c:\programdata\miniconda3\envs\myenv64bit\lib\site-packages (from pyHanko[image-support,opentype,pkcs11,xmp]) (1.5.1)
Collecting qrcode>=7.3.1 (from pyHanko[image-support,opentype,pkcs11,xmp])
  Using cached qrcode-7.4.2-py3-none-any.whl (46 kB)
Collecting tzlocal>=4.3 (from pyHanko[image-support,opentype,pkcs11,xmp])
  Using cached tzlocal-5.2-py3-none-any.whl.metadata (7.8 kB)
Collecting pyhanko-certvalidator<0.27,>=0.26.2 (from pyHanko[image-support,opentype,pkcs11,xmp])
  Using cached pyhanko_certvalidator-0.26.3-py3-none-any.whl.metadata (5.3 kB)
Collecting click>=8.1.3 (from pyHanko[image-support,opentype,pkcs11,xmp])
  Using cached click-8.1.7-py3-none-any.whl.metadata (3.0 kB)
Collecting requests>=2.31.0 (from pyHanko[image-support,opentype,pkcs11,xmp])
  Using cached requests-2.31.0-py3-none-any.whl.metadata (4.6 kB)
Collecting pyyaml>=6.0 (from pyHanko[image-support,opentype,pkcs11,xmp])
  Using cached PyYAML-6.0.1-cp311-cp311-win_amd64.whl.metadata (2.1 kB)
Collecting cryptography>=41.0.5 (from pyHanko[image-support,opentype,pkcs11,xmp])
  Using cached cryptography-42.0.2-cp39-abi3-win_amd64.whl.metadata (5.4 kB)
Collecting Pillow>=7.2.0 (from pyHanko[image-support,opentype,pkcs11,xmp])
  Using cached pillow-10.2.0-cp311-cp311-win_amd64.whl.metadata (9.9 kB)
Collecting python-barcode==0.15.1 (from pyHanko[image-support,opentype,pkcs11,xmp])
  Using cached python_barcode-0.15.1-py3-none-any.whl.metadata (2.3 kB)
Collecting fonttools>=4.33.3 (from pyHanko[image-support,opentype,pkcs11,xmp])
  Using cached fonttools-4.47.2-cp311-cp311-win_amd64.whl.metadata (160 kB)
Collecting uharfbuzz<0.38.0,>=0.25.0 (from pyHanko[image-support,opentype,pkcs11,xmp])
  Using cached uharfbuzz-0.37.3-cp311-cp311-win_amd64.whl.metadata (2.3 kB)
Requirement already satisfied: python-pkcs11~=0.7.0 in c:\programdata\miniconda3\envs\myenv64bit\lib\site-packages (from pyHanko[image-support,opentype,pkcs11,xmp]) (0.7.1.dev40+gff20b84)
Collecting defusedxml~=0.7.1 (from pyHanko[image-support,opentype,pkcs11,xmp])
  Using cached defusedxml-0.7.1-py2.py3-none-any.whl (25 kB)
Collecting colorama (from click>=8.1.3->pyHanko[image-support,opentype,pkcs11,xmp])
  Using cached colorama-0.4.6-py2.py3-none-any.whl (25 kB)
Collecting cffi>=1.12 (from cryptography>=41.0.5->pyHanko[image-support,opentype,pkcs11,xmp])
  Using cached cffi-1.16.0-cp311-cp311-win_amd64.whl.metadata (1.5 kB)
Collecting oscrypto>=1.1.0 (from pyhanko-certvalidator<0.27,>=0.26.2->pyHanko[image-support,opentype,pkcs11,xmp])
  Using cached oscrypto-1.3.0-py2.py3-none-any.whl (194 kB)
Collecting uritools>=3.0.1 (from pyhanko-certvalidator<0.27,>=0.26.2->pyHanko[image-support,opentype,pkcs11,xmp])
  Using cached uritools-4.0.2-py3-none-any.whl.metadata (4.7 kB)
Requirement already satisfied: cached-property in c:\programdata\miniconda3\envs\myenv64bit\lib\site-packages (from python-pkcs11~=0.7.0->pyHanko[image-support,opentype,pkcs11,xmp]) (1.5.2)
Collecting typing-extensions (from qrcode>=7.3.1->pyHanko[image-support,opentype,pkcs11,xmp])
  Using cached typing_extensions-4.9.0-py3-none-any.whl.metadata (3.0 kB)
Collecting pypng (from qrcode>=7.3.1->pyHanko[image-support,opentype,pkcs11,xmp])
  Using cached pypng-0.20220715.0-py3-none-any.whl (58 kB)
Collecting charset-normalizer<4,>=2 (from requests>=2.31.0->pyHanko[image-support,opentype,pkcs11,xmp])
  Using cached charset_normalizer-3.3.2-cp311-cp311-win_amd64.whl.metadata (34 kB)
Collecting idna<4,>=2.5 (from requests>=2.31.0->pyHanko[image-support,opentype,pkcs11,xmp])
  Using cached idna-3.6-py3-none-any.whl.metadata (9.9 kB)
Collecting urllib3<3,>=1.21.1 (from requests>=2.31.0->pyHanko[image-support,opentype,pkcs11,xmp])
  Using cached urllib3-2.2.0-py3-none-any.whl.metadata (6.4 kB)
Collecting certifi>=2017.4.17 (from requests>=2.31.0->pyHanko[image-support,opentype,pkcs11,xmp])
  Using cached certifi-2023.11.17-py3-none-any.whl.metadata (2.2 kB)
Collecting tzdata (from tzlocal>=4.3->pyHanko[image-support,opentype,pkcs11,xmp])
  Using cached tzdata-2023.4-py2.py3-none-any.whl.metadata (1.4 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=41.0.5->pyHanko[image-support,opentype,pkcs11,xmp])
  Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Using cached python_barcode-0.15.1-py3-none-any.whl (212 kB)
Using cached click-8.1.7-py3-none-any.whl (97 kB)
Using cached cryptography-42.0.2-cp39-abi3-win_amd64.whl (2.9 MB)
Using cached fonttools-4.47.2-cp311-cp311-win_amd64.whl (2.2 MB)
Using cached pillow-10.2.0-cp311-cp311-win_amd64.whl (2.6 MB)
Using cached pyhanko_certvalidator-0.26.3-py3-none-any.whl (109 kB)
Using cached PyYAML-6.0.1-cp311-cp311-win_amd64.whl (144 kB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached tzlocal-5.2-py3-none-any.whl (17 kB)
Using cached uharfbuzz-0.37.3-cp311-cp311-win_amd64.whl (1.1 MB)
Using cached pyHanko-0.21.0-py3-none-any.whl (433 kB)
Using cached certifi-2023.11.17-py3-none-any.whl (162 kB)
Using cached cffi-1.16.0-cp311-cp311-win_amd64.whl (181 kB)
Using cached charset_normalizer-3.3.2-cp311-cp311-win_amd64.whl (99 kB)
Using cached idna-3.6-py3-none-any.whl (61 kB)
Using cached uritools-4.0.2-py3-none-any.whl (10 kB)
Using cached urllib3-2.2.0-py3-none-any.whl (120 kB)
Using cached typing_extensions-4.9.0-py3-none-any.whl (32 kB)
Using cached tzdata-2023.4-py2.py3-none-any.whl (346 kB)
Installing collected packages: python-barcode, pypng, urllib3, uritools, uharfbuzz, tzdata, typing-extensions, pyyaml, pycparser, Pillow, oscrypto, idna, fonttools, defusedxml, colorama, charset-normalizer, certifi, tzlocal, requests, qrcode, click, cffi, cryptography, pyhanko-certvalidator, pyHanko
Successfully installed Pillow-10.2.0 certifi-2023.11.17 cffi-1.16.0 charset-normalizer-3.3.2 click-8.1.7 colorama-0.4.6 cryptography-42.0.2 defusedxml-0.7.1 fonttools-4.47.2 idna-3.6 oscrypto-1.3.0 pyHanko-0.21.0 pycparser-2.21 pyhanko-certvalidator-0.26.3 pypng-0.20220715.0 python-barcode-0.15.1 pyyaml-6.0.1 qrcode-7.4.2 requests-2.31.0 typing-extensions-4.9.0 tzdata-2023.4 tzlocal-5.2 uharfbuzz-0.37.3 uritools-4.0.2 urllib3-2.2.0

(myenv64bit) C:\pyHanko\pyHanko>cd C:\Users\Patrik\Documents\Signatures\pades

(myenv64bit) C:\Users\Patrik\Documents\Signatures\pades>pyhanko --verbose sign addsig --field 1/40,800,150,850/Signature1 --style-name noto-qr --stamp-url "mailto:.." --timestamp-url http://timestamp.apple.com/ts01 --with-validation-info --validation-context PS-QC --use-pades pkcs11 --p11-setup PS-QC blank.pdf blank_simple.pdf
2024-02-01 15:22:52,547 - root - DEBUG - Running with --verbose
2024-02-01 15:22:52,547 - root - DEBUG - Finished reading configuration from pyhanko.yml.
2024-02-01 15:22:52,568 - cli - ERROR - Generic processing error.
Traceback (most recent call last):
  File "C:\ProgramData\miniconda3\envs\myenv64bit\Lib\site-packages\pyhanko\cli\runtime.py", line 50, in pyhanko_exception_manager
    yield
  File "C:\ProgramData\miniconda3\envs\myenv64bit\Lib\site-packages\pyhanko\cli\commands\signing\plugin.py", line 132, in _callback
    _callback_logic(plugin, infile, outfile, **kwargs)
  File "C:\ProgramData\miniconda3\envs\myenv64bit\Lib\site-packages\pyhanko\cli\commands\signing\plugin.py", line 70, in _callback_logic
    with plugin.create_signer(cli_ctx, **kwargs) as signer:
  File "C:\ProgramData\miniconda3\envs\myenv64bit\Lib\site-packages\pyhanko\sign\pkcs11.py", line 744, in __enter__
    return self._instantiate()
           ^^^^^^^^^^^^^^^^^^^
  File "C:\ProgramData\miniconda3\envs\myenv64bit\Lib\site-packages\pyhanko\sign\pkcs11.py", line 722, in _instantiate
    self._session = session = open_pkcs11_session(
                              ^^^^^^^^^^^^^^^^^^^^
  File "C:\ProgramData\miniconda3\envs\myenv64bit\Lib\site-packages\pyhanko\sign\pkcs11.py", line 357, in open_pkcs11_session
    lib = p11_lib(lib_location)
          ^^^^^^^^^^^^^^^^^^^^^
  File "C:\ProgramData\miniconda3\envs\myenv64bit\Lib\site-packages\pkcs11\__init__.py", line 32, in lib
    _lib = _pkcs11.lib(so)
           ^^^^^^^^^^^^^^^
  File "pkcs11\\_pkcs11.pyx", line 1506, in pkcs11._pkcs11.lib.__cinit__
  File "pkcs11\\_pkcs11.pyx", line 1470, in pkcs11._pkcs11.lib._load_pkcs11_lib
RuntimeError: Cannot open library at C:/Windows/SysWOW64/eop2v1czep11.dll: C:/Windows/SysWOW64/eop2v1czep11.dll is not a valid Win32 application.

Error: Generic processing error.

(myenv64bit) C:\Users\Patrik\Documents\Signatures\pades>

\ \ So it seems that the issue is probably in pkcs11 / python-pkcs11.

MatthiasValvekens commented 5 months ago

On closer inspection: that "is not a valid Win32 application" error comes from the Windows API itself, and IIRC it doesn't actually have anything to do with 32-bit vs. 64-bit. But it's been many years since I last did any serious dev work on Windows, so take that with a grain of salt.

does that DLL work with other applications (e.g. the pkcs11 integration in Acrobat)?

KingCZE commented 5 months ago

Sorry, my bad... SysWow64 is a 32-bit library, System32 is a 64-bit library. Stupid Windows and stupid me. It is correct then. I am really sorry for bothering (and I should ask KB why they have only a 32bit PKCS11 driver).

Good job, thank you for the software. I really appreciate it.