search

pyauth / python-pkcs11

PKCS#11/Cryptoki support for Python
MIT License
148 stars 70 forks source link

[BUG]: MemoryError: unable to allocate array data when sign. #180

Open haumenphai opened 1 month ago

haumenphai commented 1 month ago

Hello.

I got this error while developing a digital signing application. I downgraded to version 0.6.0 and the error was gone but the signing result was not as I expected.

/home/do/my_projects/odoo-17/venv3.10saas-odoo17/bin/python3.10 -X pycache_prefix=/home/do/.cache/JetBrains/PyCharm2024.1/cpython-cache /snap/pycharm-professional/407/plugins/python/helpers/pydev/pydevd.py --multiprocess --qt-support=auto --client 127.0.0.1 --port 36347 --file /home/do/my_projects/odoo-17/t.py 
Connected to pydev debugger (build 241.18968.29)
Traceback (most recent call last):
  File "/home/do/my_projects/odoo-17/t.py", line 71, in <module>
    pdf_signer.sign_pdf(writer, output=outf)
  File "/home/do/my_projects/odoo-17/venv3.10saas-odoo17/lib/python3.10/site-packages/pyhanko/sign/signers/pdf_signer.py", line 1524, in sign_pdf
    result = asyncio.run(
  File "/snap/pycharm-professional/407/plugins/python/helpers-pro/pydevd_asyncio/pydevd_nest_asyncio.py", line 138, in run
    return loop.run_until_complete(task)
  File "/snap/pycharm-professional/407/plugins/python/helpers-pro/pydevd_asyncio/pydevd_nest_asyncio.py", line 243, in run_until_complete
    return f.result()
  File "/usr/lib/python3.10/asyncio/futures.py", line 201, in result
    raise self._exception.with_traceback(self._exception_tb)
  File "/usr/lib/python3.10/asyncio/tasks.py", line 234, in __step
    result = coro.throw(exc)
  File "/home/do/my_projects/odoo-17/venv3.10saas-odoo17/lib/python3.10/site-packages/pyhanko/sign/signers/pdf_signer.py", line 1603, in async_sign_pdf
    post_signing_doc = await tbs_document.perform_signature(
  File "/home/do/my_projects/odoo-17/venv3.10saas-odoo17/lib/python3.10/site-packages/pyhanko/sign/signers/pdf_signer.py", line 2620, in perform_signature
    signature_cms = await signer.async_sign(
  File "/home/do/my_projects/odoo-17/venv3.10saas-odoo17/lib/python3.10/site-packages/pyhanko/sign/signers/pdf_cms.py", line 944, in async_sign
    return await self.async_sign_prescribed_attributes(
  File "/home/do/my_projects/odoo-17/venv3.10saas-odoo17/lib/python3.10/site-packages/pyhanko/sign/signers/pdf_cms.py", line 1007, in async_sign_prescribed_attributes
    signature = await self.async_sign_raw(
  File "/home/do/my_projects/odoo-17/venv3.10saas-odoo17/lib/python3.10/site-packages/pyhanko/sign/pkcs11.py", line 595, in async_sign_raw
    return await loop.run_in_executor(None, _perform_signature)
  File "/usr/lib/python3.10/asyncio/futures.py", line 285, in __await__
    yield self  # This tells Task to wait for completion.
  File "/usr/lib/python3.10/asyncio/tasks.py", line 304, in __wakeup
    future.result()
  File "/usr/lib/python3.10/asyncio/futures.py", line 201, in result
    raise self._exception.with_traceback(self._exception_tb)
  File "/usr/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/home/do/my_projects/odoo-17/venv3.10saas-odoo17/lib/python3.10/site-packages/pyhanko/sign/pkcs11.py", line 589, in _perform_signature
    signature = kh.sign(data, **spec.sign_kwargs)
  File "/home/do/my_projects/odoo-17/venv3.10saas-odoo17/lib/python3.10/site-packages/pkcs11/types.py", line 939, in sign
    return self._sign(data, **kwargs)
  File "pkcs11/_pkcs11.pyx", line 1072, in pkcs11._pkcs11.SignMixin._sign
  File "pkcs11/_pkcs11.pyx", line 1080, in pkcs11._pkcs11.SignMixin._sign
  File "pkcs11/_utils.pyx", line 11, in pkcs11._pkcs11.CK_BYTE_buffer
  File "<stringsource>", line 180, in View.MemoryView.array.__cinit__
  File "<stringsource>", line 257, in View.MemoryView._allocate_buffer
MemoryError: unable to allocate array data.

PIP:

Package                  Version
------------------------ --------------------
acme                     2.11.0
aiohttp                  3.8.4
aiosignal                1.3.1
asn1crypto               1.5.1
astor                    0.8.1
async-timeout            4.0.3
attrs                    23.2.0
Babel                    2.9.1
bcrypt                   4.1.2
beautifulsoup4           4.12.3
boto3                    1.34.96
botocore                 1.34.96
cached-property          1.5.2
cachetools               5.3.3
certifi                  2024.2.2
certvalidator            0.11.1
cffi                     1.16.0
chardet                  4.0.0
charset-normalizer       3.3.2
click                    8.1.7
cryptography             43.0.0
cssselect                1.2.0
decorator                4.4.2
defusedxml               0.7.1
docopt                   0.6.2
docutils                 0.17
ebaysdk                  2.1.5
endesive                 2.17.2
fasttext                 0.9.2
fonttools                4.53.1
freezegun                1.1.0
frozenlist               1.4.1
geoip2                   2.9.0
gevent                   21.8.0
gitdb                    4.0.11
GitPython                3.1.31
google-api-core          2.19.0
google-auth              2.23.0
google-cloud-core        2.4.1
google-cloud-storage     2.8.0
google-cloud-translate   3.11.1
google-crc32c            1.5.0
google-resumable-media   2.7.0
googleapis-common-protos 1.63.0
greenlet                 1.1.2
grpcio                   1.63.0
grpcio-status            1.62.2
idna                     2.10
isodate                  0.6.1
Jinja2                   3.0.3
jmespath                 1.0.1
josepy                   1.14.0
libsass                  0.20.1
lxml                     5.2.2
Markdown                 3.6
MarkupSafe               2.0.1
maxminddb                2.6.1
mt-940                   4.28.0
multidict                6.0.5
num2words                0.5.10
numpy                    1.26.4
ofxparse                 0.21
openupgradelib           3.6.2.dev13+g6a87c06
oscrypto                 1.3.0
paramiko                 3.0.0
passlib                  1.7.4
Pillow                   9.0.1
pip                      23.0.1
platformdirs             4.2.1
polib                    1.1.1
proto-plus               1.23.0
protobuf                 4.25.3
psutil                   5.9.0
psycopg2                 2.9.2
pyasn1                   0.6.0
pyasn1_modules           0.4.0
pybind11                 2.12.0
pycparser                2.22
pydot                    1.4.2
Pygments                 2.14.0
pyHanko                  0.25.1
pyhanko-certvalidator    0.26.3
PyKCS11                  1.5.16
PyNaCl                   1.5.0
pyOpenSSL                21.0.0
pyparsing                3.1.2
PyPDF2                   1.26.0
pyRFC3339                1.1
pyserial                 3.5
python-barcode           0.15.1
python-dateutil          2.8.1
python-ldap              3.4.0
python-pkcs11            0.7.0
python-stdnum            1.17
pytz                     2024.1
pyusb                    1.2.1
PyYAML                   6.0.1
qrcode                   7.3.1
reportlab                3.6.8
requests                 2.32.3
requests-file            2.0.0
requests-toolbelt        1.0.0
rjsmin                   1.1.0
rsa                      4.9
s3transfer               0.10.1
sentry-sdk               1.9.0
setuptools               65.5.0
six                      1.16.0
smmap                    5.0.1
soupsieve                2.5
tabulate                 0.9.0
tzlocal                  5.2
ua-parser                0.18.0
uharfbuzz                0.39.3
uritools                 4.0.3
urllib3                  1.26.5
user-agents              2.2.0
validators               0.20.0
vobject                  0.9.6.1
wbgapi                   1.0.12
Werkzeug                 2.0.2
wheel                    0.43.0
xlrd                     1.2.0
XlsxWriter               3.0.2
xlwt                     1.3.0
yarl                     1.9.4
zeep                     4.1.0
zope.event               5.0
zope.interface           6.3

My Code:

import asn1crypto.x509
from pyhanko import stamp
from pyhanko.pdf_utils import text, images, layout
from pyhanko.pdf_utils.font import opentype
from pyhanko.pdf_utils.incremental_writer import IncrementalPdfFileWriter
from pyhanko.sign import fields, signers
from pyhanko.sign.pkcs11 import PKCS11Signer

def _usb_token_get_signer():
    import pkcs11
    from cryptography import x509
    from pkcs11.constants import Attribute, ObjectClass

    lib_path = "/usr/lib/viettel-ca_v6.so"
    lib = pkcs11.lib(lib_path)
    token = lib.get_token()

    session = token.open(user_pin='12345678')
    certificates = session.get_objects({pkcs11.Attribute.CLASS: pkcs11.ObjectClass.CERTIFICATE})
    certs = [c for c in certificates]
    cert = certs[-1]
    der_cert = cert[pkcs11.Attribute.VALUE]
    x509_cert = x509.load_der_x509_certificate(der_cert)
    asn1crypto_cert = asn1crypto.x509.Certificate.load(der_cert)

    usb_signer = PKCS11Signer(session, key_id=cert[Attribute.ID], signing_cert=asn1crypto_cert)
    return usb_signer

signer = signers.SimpleSigner.load(
    '/home/do/sign_keys/private_key.pem', '/home/do/sign_keys/certificate.pem',
    ca_chain_files=None,
    key_passphrase=None
)

sign_field1 = fields.SigFieldSpec(
    sig_field_name="Signature2",
    box=(300, 500, 500, 560),
    # field_mdp_spec=fields.FieldMDPSpec(
    #     fields.FieldMDPAction.INCLUDE, fields=['SomeTextField']
    # ),
    doc_mdp_update_value=fields.MDPPerm.FILL_FORMS
)
with open('/home/do/Desktop/product.pdf', 'rb+') as doc:
    writer = IncrementalPdfFileWriter(doc)
    fields.append_signature_field(pdf_out=writer, sig_field_spec=sign_field1)
    # writer.write_in_place()

    #
    meta = signers.PdfSignatureMetadata(field_name='Signature2')
    pdf_signer = signers.PdfSigner(
        meta, signer=_usb_token_get_signer(), stamp_style=stamp.TextStampStyle(
            # the 'signer' and 'ts' parameters will be interpolated by pyHanko, if present
            border_width=1,
            background_layout=layout.SimpleBoxLayoutRule(
                x_align=layout.AxisAlignment.ALIGN_MID,
                y_align=layout.AxisAlignment.ALIGN_MID,
                margins=layout.Margins.uniform(5),
            ),
            stamp_text='Digital signed by: %(signer)s\nTime: %(ts)s',
            text_box_style=text.TextBoxStyle(
                font_size=13
            ),
            background=images.PdfImage('/home/do/Desktop/img.png'),
            background_opacity=0.3
        ),
    )
    with open('/home/do/Desktop/document-signed.pdf', 'wb') as outf:
        pdf_signer.sign_pdf(writer, output=outf)

OS: Ubuntu 20.04, 16GB RAM

haumenphai commented 1 month ago

Same issues:

import io

import pkcs11
from pkcs11 import Mechanism, Attribute
from pkcs11 import KeyType
from pypdf import PdfReader, PdfWriter
from pkcs11.types import PrivateKey
import base64

# Path to your PKCS#11 library
PKCS11_LIB = '/usr/lib/viettel-ca_v6.so'

# Load the PKCS#11 library
lib = pkcs11.lib(PKCS11_LIB)

# Open the first available slot
slot = lib.get_slots(token_present=True)[0]
token = slot.get_token()

with token.open(user_pin='12345678') as session:
    certificates = session.get_objects({pkcs11.Attribute.CLASS: pkcs11.ObjectClass.CERTIFICATE})
    certs = [c for c in certificates]
    cert = certs[-1]
    der_cert = cert[pkcs11.Attribute.VALUE]

    private_key = [key for key in session.get_objects(KeyType.RSA) if isinstance(key, PrivateKey)][1]
    signature = private_key.sign('v2', mechanism=Mechanism.RSA_PKCS)