search

coleifer / pysqlite3

SQLite3 DB-API 2.0 driver from Python 3, packaged separately, with improvements
zlib License
192 stars 53 forks source link

Lambda Docker image failing when trying to install boto3 and pysqlite3-binary #55

Closed nk9 closed 1 year ago

nk9 commented 1 year ago

I want to use a recent version of SQLite3 with Python, but the version on the Lambda images is very old. So last year I found this project, and I was using it successfully to run the Lambda container with a newer version of SQLite.

Dockerfile

FROM public.ecr.aws/lambda/python:3.10 AS base

# Set up environment
ENV LANG C.UTF-8
ENV LC_ALL C.UTF-8
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONFAULTHANDLER 1

FROM base AS python-deps

RUN pip install pipenv

COPY Pipfile* ./

RUN PIPENV_VENV_IN_PROJECT=1 pipenv install --deploy --verbose

FROM base as runtime

COPY --from=python-deps ${LAMBDA_TASK_ROOT}/.venv/lib/python*/site-packages ./
COPY app app

# Set the CMD to your handler (could also be done as a parameter override outside of the Dockerfile)
CMD [ "app.app.handler" ]

Pipfile

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
boto3 = "*"
pysqlite3-binary = { version = "*", platform_machine = "== 'x86_64'" }

[requires]
python_version = "3.10"

Things work fine when I install the binary package by itself:

bash-4.2# python3
Python 3.10.11 (main, May  8 2023, 08:58:11) [GCC 7.3.1 20180712 (Red Hat 7.3.1-15)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pysqlite3 as sqlite3
>>> print(f"{sqlite3.sqlite_version=}")
sqlite3.sqlite_version='3.40.0'

But when I include boto3, the pipenv installation fails quite spectacularly:

 > [python-deps 3/3] RUN PIPENV_VENV_IN_PROJECT=1 pipenv install --deploy --verbose:
#8 4.823 Using python: 3.10
#8 4.823 Path to python: /var/lang/bin/python3
#8 4.823 Creating a virtualenv for this project...
#8 4.824 Pipfile: /var/task/Pipfile
#8 5.017 Using /var/lang/bin/python3 (3.10.11) to create virtualenv...
#8 9.102 created virtual environment CPython3.10.11.final.0-64 in 2922ms
#8 9.102   creator CPython3Posix(dest=/var/task/.venv, clear=False, no_vcs_ignore=False, global=False)
#8 9.102   seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/root/.local/share/virtualenv)
#8 9.102     added seed packages: pip==23.1.2, setuptools==67.7.2, wheel==0.40.0
#8 9.102   activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator
#8 9.102
#8 9.133 ✔ Successfully created virtual environment!
#8 9.522 Virtualenv location: /var/task/.venv
#8 9.852 Installing dependencies from Pipfile.lock (1d44fb)...
#8 10.06 Writing supplied requirement line to temporary file: 'boto3==1.26.148 --hash=sha256:53328b99847e65ca60dbbd5a86550101ecf903dff36724b6a4e74876dd973bdf --hash=sha256:ed90522019c97db32ce1bfd1076864c57c3b65af40abc2349b2391bbac7e4865'
#8 10.09 Writing supplied requirement line to temporary file: "botocore==1.29.148 ; python_version >= '3.7' --hash=sha256:36bccc4e094faf1bfbda2241eb1c71e72a63f3c12b2adfc4c3cbd02349bc416b --hash=sha256:71c512fbd42e0ef171388b53bf82f9cb9e66283467156917b69717daeb05eba6"
#8 10.11 Writing supplied requirement line to temporary file: "jmespath==1.0.1 ; python_version >= '3.7' --hash=sha256:02e2e4cc71b5bcab88332eebf907519190dd9e6e82107fa7f83b1003a6252980 --hash=sha256:90261b206d6defd58fdd5e85f478bf633a2901798906be2ad389150c5c60edbe"
#8 10.14 Writing supplied requirement line to temporary file: "pysqlite3-binary ; platform_machine == 'x86_64'"
#8 10.16 Writing supplied requirement line to temporary file: "python-dateutil==2.8.2 ; python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3' --hash=sha256:0123cacc1627ae19ddf3c27a5de5bd67ee4586fbdd6440d9748f8abb483d3e86 --hash=sha256:961d03dc3453ebbc59dbdea9e4e11c5651520a876d0f4db161e8674aae935da9"
#8 10.19 Writing supplied requirement line to temporary file: "s3transfer==0.6.1 ; python_version >= '3.7' --hash=sha256:3c0da2d074bf35d6870ef157158641178a4204a6e689e82546083e31e0311346 --hash=sha256:640bb492711f4c0c0905e1f62b6aaeb771881935ad27884852411f8e9cacbca9"
#8 10.21 Writing supplied requirement line to temporary file: "six==1.16.0 ; python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3' --hash=sha256:1e61c37477a1626458e36f7b1d82aa5c9b094fa4802892072e49de9c60c4c926 --hash=sha256:8abb2f1d86890a2dfb989f9a77cfcfd3e47c2a354b01111771326f8aa26e0254"
#8 10.24 Writing supplied requirement line to temporary file: "urllib3==1.26.16 ; python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4, 3.5' --hash=sha256:8d36afa7616d8ab714608411b4a3b13e58f463aee519024578e062e141dce20f --hash=sha256:8f135f6502756bde6b2a9b28989df5fbe87c9970cecaa69041edcce7f0589b14"
#8 10.24 Install Phase: Standard Requirements
#8 10.24 Preparing Installation of 'boto3'
#8 10.24 Preparing Installation of 'botocore'
#8 10.24 Preparing Installation of 'jmespath'
#8 10.24 Preparing Installation of 'pysqlite3-binary'
#8 10.24 Preparing Installation of 'python-dateutil'
#8 10.24 Preparing Installation of 's3transfer'
#8 10.24 Preparing Installation of 'six'
#8 10.24 Preparing Installation of 'urllib3'
#8 10.24 $ /var/task/.venv/bin/python /var/lang/lib/python3.10/site-packages/pipenv/patched/pip/__pip-runner__.py install -i https://pypi.org/simple --no-input --upgrade --no-deps --exists-action=i -r /tmp/pipenv-8bdg3b7s-requirements/pipenv-hsujwah9-hashed-reqs.txt
#8 10.24 Using source directory: '/var/task/.venv/src'
#8 19.05 Collecting boto3==1.26.148 (from -r /tmp/pipenv-8bdg3b7s-requirements/pipenv-hsujwah9-hashed-reqs.txt (line 1))
#8 19.05
#8 19.17   Downloading boto3-1.26.148-py3-none-any.whl (135 kB)
#8 19.17
#8 19.23      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 135.6/135.6 kB 4.6 MB/s eta 0:00:00
#8 19.23
#8 24.73 Collecting botocore==1.29.148 (from -r /tmp/pipenv-8bdg3b7s-requirements/pipenv-hsujwah9-hashed-reqs.txt (line 2))
#8 24.73
#8 24.75   Downloading botocore-1.29.148-py3-none-any.whl (10.8 MB)
#8 24.75
#8 26.51      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 10.8/10.8 MB 6.1 MB/s eta 0:00:00
#8 26.51
#8 27.32 Collecting jmespath==1.0.1 (from -r /tmp/pipenv-8bdg3b7s-requirements/pipenv-hsujwah9-hashed-reqs.txt (line 3))
#8 27.32
#8 27.34   Downloading jmespath-1.0.1-py3-none-any.whl (20 kB)
#8 27.34
#8 27.49 Collecting pysqlite3-binary (from -r /tmp/pipenv-8bdg3b7s-requirements/pipenv-hsujwah9-hashed-reqs.txt (line 4))
#8 27.49
#8 27.89 ERROR: In --require-hashes mode, all requirements must have their versions pinned with ==. These do not:
#8 27.89     pysqlite3-binary from https://files.pythonhosted.org/packages/e5/af/750aa272a8282f6518a8532f31c51cc9e49815c626cf0df8cdf181367b52/pysqlite3_binary-0.5.0-cp310-cp310-manylinux_2_24_x86_64.whl (from -r /tmp/pipenv-8bdg3b7s-requirements/pipenv-hsujwah9-hashed-reqs.txt (line 4))
#8 27.89 An error occurred while installing boto3==1.26.148 --hash=sha256:53328b99847e65ca60dbbd5a86550101ecf903dff36724b6a4e74876dd973bdf --hash=sha256:ed90522019c97db32ce1bfd1076864c57c3b65af40abc2349b2391bbac7e4865! Will try again.
#8 27.89 An error occurred while installing botocore==1.29.148 ; python_version >= '3.7' --hash=sha256:36bccc4e094faf1bfbda2241eb1c71e72a63f3c12b2adfc4c3cbd02349bc416b --hash=sha256:71c512fbd42e0ef171388b53bf82f9cb9e66283467156917b69717daeb05eba6! Will try again.
#8 27.90 An error occurred while installing jmespath==1.0.1 ; python_version >= '3.7' --hash=sha256:02e2e4cc71b5bcab88332eebf907519190dd9e6e82107fa7f83b1003a6252980 --hash=sha256:90261b206d6defd58fdd5e85f478bf633a2901798906be2ad389150c5c60edbe! Will try again.
#8 27.90 An error occurred while installing pysqlite3-binary ; platform_machine == 'x86_64'! Will try again.
#8 27.90 An error occurred while installing python-dateutil==2.8.2 ; python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3' --hash=sha256:0123cacc1627ae19ddf3c27a5de5bd67ee4586fbdd6440d9748f8abb483d3e86 --hash=sha256:961d03dc3453ebbc59dbdea9e4e11c5651520a876d0f4db161e8674aae935da9! Will try again.
#8 27.90 An error occurred while installing s3transfer==0.6.1 ; python_version >= '3.7' --hash=sha256:3c0da2d074bf35d6870ef157158641178a4204a6e689e82546083e31e0311346 --hash=sha256:640bb492711f4c0c0905e1f62b6aaeb771881935ad27884852411f8e9cacbca9! Will try again.
#8 27.91 An error occurred while installing six==1.16.0 ; python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3' --hash=sha256:1e61c37477a1626458e36f7b1d82aa5c9b094fa4802892072e49de9c60c4c926 --hash=sha256:8abb2f1d86890a2dfb989f9a77cfcfd3e47c2a354b01111771326f8aa26e0254! Will try again.
#8 27.91 An error occurred while installing urllib3==1.26.16 ; python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4, 3.5' --hash=sha256:8d36afa7616d8ab714608411b4a3b13e58f463aee519024578e062e141dce20f --hash=sha256:8f135f6502756bde6b2a9b28989df5fbe87c9970cecaa69041edcce7f0589b14! Will try again.
#8 27.91 Installing initially failed dependencies...
#8 27.98 Writing supplied requirement line to temporary file: 'boto3==1.26.148 --hash=sha256:53328b99847e65ca60dbbd5a86550101ecf903dff36724b6a4e74876dd973bdf --hash=sha256:ed90522019c97db32ce1bfd1076864c57c3b65af40abc2349b2391bbac7e4865'
#8 27.98 Writing supplied requirement line to temporary file: "botocore==1.29.148 ; python_version >= '3.7' --hash=sha256:36bccc4e094faf1bfbda2241eb1c71e72a63f3c12b2adfc4c3cbd02349bc416b --hash=sha256:71c512fbd42e0ef171388b53bf82f9cb9e66283467156917b69717daeb05eba6"
#8 27.98 Writing supplied requirement line to temporary file: "jmespath==1.0.1 ; python_version >= '3.7' --hash=sha256:02e2e4cc71b5bcab88332eebf907519190dd9e6e82107fa7f83b1003a6252980 --hash=sha256:90261b206d6defd58fdd5e85f478bf633a2901798906be2ad389150c5c60edbe"
#8 27.99 Writing supplied requirement line to temporary file: "pysqlite3-binary ; platform_machine == 'x86_64'"
#8 27.99 Writing supplied requirement line to temporary file: "python-dateutil==2.8.2 ; python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3' --hash=sha256:0123cacc1627ae19ddf3c27a5de5bd67ee4586fbdd6440d9748f8abb483d3e86 --hash=sha256:961d03dc3453ebbc59dbdea9e4e11c5651520a876d0f4db161e8674aae935da9"
#8 27.99 Writing supplied requirement line to temporary file: "s3transfer==0.6.1 ; python_version >= '3.7' --hash=sha256:3c0da2d074bf35d6870ef157158641178a4204a6e689e82546083e31e0311346 --hash=sha256:640bb492711f4c0c0905e1f62b6aaeb771881935ad27884852411f8e9cacbca9"
#8 27.99 Writing supplied requirement line to temporary file: "six==1.16.0 ; python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3' --hash=sha256:1e61c37477a1626458e36f7b1d82aa5c9b094fa4802892072e49de9c60c4c926 --hash=sha256:8abb2f1d86890a2dfb989f9a77cfcfd3e47c2a354b01111771326f8aa26e0254"
#8 28.00 Writing supplied requirement line to temporary file: "urllib3==1.26.16 ; python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4, 3.5' --hash=sha256:8d36afa7616d8ab714608411b4a3b13e58f463aee519024578e062e141dce20f --hash=sha256:8f135f6502756bde6b2a9b28989df5fbe87c9970cecaa69041edcce7f0589b14"
#8 28.00 Install Phase: Standard Requirements
#8 28.00 Preparing Installation of 'boto3'
#8 28.00 Preparing Installation of 'botocore'
#8 28.00 Preparing Installation of 'jmespath'
#8 28.00 Preparing Installation of 'pysqlite3-binary'
#8 28.00 Preparing Installation of 'python-dateutil'
#8 28.00 Preparing Installation of 's3transfer'
#8 28.00 Preparing Installation of 'six'
#8 28.00 Preparing Installation of 'urllib3'
#8 28.00 $ /var/task/.venv/bin/python /var/lang/lib/python3.10/site-packages/pipenv/patched/pip/__pip-runner__.py install -i https://pypi.org/simple --no-input --upgrade --no-use-pep517 --no-deps --exists-action=i -r /tmp/pipenv-8bdg3b7s-requirements/pipenv-v8_rcwu6-hashed-reqs.txt
#8 28.00 Using source directory: '/var/task/.venv/src'
#8 35.57 Collecting boto3==1.26.148 (from -r /tmp/pipenv-8bdg3b7s-requirements/pipenv-v8_rcwu6-hashed-reqs.txt (line 1))
#8 35.57
#8 35.58   Using cached boto3-1.26.148-py3-none-any.whl (135 kB)
#8 35.58
#8 40.23 Collecting botocore==1.29.148 (from -r /tmp/pipenv-8bdg3b7s-requirements/pipenv-v8_rcwu6-hashed-reqs.txt (line 2))
#8 40.23
#8 40.27   Using cached botocore-1.29.148-py3-none-any.whl (10.8 MB)
#8 40.27
#8 41.11 Collecting jmespath==1.0.1 (from -r /tmp/pipenv-8bdg3b7s-requirements/pipenv-v8_rcwu6-hashed-reqs.txt (line 3))
#8 41.11
#8 41.12   Using cached jmespath-1.0.1-py3-none-any.whl (20 kB)
#8 41.12
#8 41.25 Collecting pysqlite3-binary (from -r /tmp/pipenv-8bdg3b7s-requirements/pipenv-v8_rcwu6-hashed-reqs.txt (line 4))
#8 41.25
#8 41.61 ERROR: In --require-hashes mode, all requirements must have their versions pinned with ==. These do not:
#8 41.61     pysqlite3-binary from https://files.pythonhosted.org/packages/e5/af/750aa272a8282f6518a8532f31c51cc9e49815c626cf0df8cdf181367b52/pysqlite3_binary-0.5.0-cp310-cp310-manylinux_2_24_x86_64.whl (from -r /tmp/pipenv-8bdg3b7s-requirements/pipenv-v8_rcwu6-hashed-reqs.txt (line 4))
#8 41.61 [pipenv.exceptions.InstallError]: ERROR: In --require-hashes mode, all requirements must have their versions pinned with ==. These do not:
#8 41.61 [pipenv.exceptions.InstallError]:     pysqlite3-binary from https://files.pythonhosted.org/packages/e5/af/750aa272a8282f6518a8532f31c51cc9e49815c626cf0df8cdf181367b52/pysqlite3_binary-0.5.0-cp310-cp310-manylinux_2_24_x86_64.whl (from -r /tmp/pipenv-8bdg3b7s-requirements/pipenv-v8_rcwu6-hashed-reqs.txt (line 4))
#8 41.61 ERROR: Couldn't install package: [Requirement(_name='boto3', vcs=None, req=NamedRequirement(name='boto3', version='==1.26.148', req=Requirement.parse('boto3==1.26.148'), extras=[], editable=False, _parsed_line=<Line (editable=False, name=boto3, path=None, uri=None, extras=(), markers=None, vcs=None, specifier===1.26.148, pyproject=None, pyproject_requires=None, pyproject_backend=None, ireq=boto3==1.26.148)>), markers=None, _specifiers='==1.26.148', index='pypi', editable=False, hashes=frozenset({'sha256:53328b99847e65ca60dbbd5a86550101ecf903dff36724b6a4e74876dd973bdf', 'sha256:ed90522019c97db32ce1bfd1076864c57c3b65af40abc2349b2391bbac7e4865'}), extras=(), abstract_dep=None, _line_instance=<Line (editable=False, name=boto3, path=None, uri=None, extras=(), markers=None, vcs=None, specifier===1.26.148, pyproject=None, pyproject_requires=None, pyproject_backend=None, ireq=boto3==1.26.148)>, _ireq=None), Requirement(_name='botocore', vcs=None, req=NamedRequirement(name='botocore', version='==1.29.148', req=Requirement.parse('botocore==1.29.148; python_version >= "3.7"'), extras=[], editable=False, _parsed_line=<Line (editable=False, name=botocore, path=None, uri=None, extras=(), markers=python_version >= '3.7', vcs=None, specifier===1.29.148, pyproject=None, pyproject_requires=None, pyproject_backend=None, ireq=botocore==1.29.148)>), markers='python_version >= "3.7"', _specifiers='==1.29.148', index=None, editable=False, hashes=frozenset({'sha256:71c512fbd42e0ef171388b53bf82f9cb9e66283467156917b69717daeb05eba6', 'sha256:36bccc4e094faf1bfbda2241eb1c71e72a63f3c12b2adfc4c3cbd02349bc416b'}), extras=(), abstract_dep=None, _line_instance=<Line (editable=False, name=botocore, path=None, uri=None, extras=(), markers=python_version >= '3.7', vcs=None, specifier===1.29.148, pyproject=None, pyproject_requires=None, pyproject_backend=None, ireq=botocore==1.29.148)>, _ireq=None), Requirement(_name='jmespath', vcs=None, req=NamedRequirement(name='jmespath', version='==1.0.1', req=Requirement.parse('jmespath==1.0.1; python_version >= "3.7"'), extras=[], editable=False, _parsed_line=<Line (editable=False, name=jmespath, path=None, uri=None, extras=(), markers=python_version >= '3.7', vcs=None, specifier===1.0.1, pyproject=None, pyproject_requires=None, pyproject_backend=None, ireq=jmespath==1.0.1)>), markers='python_version >= "3.7"', _specifiers='==1.0.1', index=None, editable=False, hashes=frozenset({'sha256:90261b206d6defd58fdd5e85f478bf633a2901798906be2ad389150c5c60edbe', 'sha256:02e2e4cc71b5bcab88332eebf907519190dd9e6e82107fa7f83b1003a6252980'}), extras=(), abstract_dep=None, _line_instance=<Line (editable=False, name=jmespath, path=None, uri=None, extras=(), markers=python_version >= '3.7', vcs=None, specifier===1.0.1, pyproject=None, pyproject_requires=None, pyproject_backend=None, ireq=jmespath==1.0.1)>, _ireq=None), Requirement(_name='pysqlite3-binary', vcs=None, req=NamedRequirement(name='pysqlite3-binary', version='', req=Requirement.parse('pysqlite3-binary; platform_machine == "x86_64"'), extras=[], editable=False, _parsed_line=<Line (editable=False, name=pysqlite3-binary, path=None, uri=None, extras=(), markers=platform_machine == 'x86_64', vcs=None, specifier=None, pyproject=None, pyproject_requires=None, pyproject_backend=None, ireq=pysqlite3-binary)>), markers='platform_machine == "x86_64"', _specifiers='', index=None, editable=False, hashes=frozenset(), extras=(), abstract_dep=None, _line_instance=<Line (editable=False, name=pysqlite3-binary, path=None, uri=None, extras=(), markers=platform_machine == 'x86_64', vcs=None, specifier=None, pyproject=None, pyproject_requires=None, pyproject_backend=None, ireq=pysqlite3-binary)>, _ireq=None), Requirement(_name='python-dateutil', vcs=None, req=NamedRequirement(name='python-dateutil', version='==2.8.2', req=Requirement.parse('python-dateutil==2.8.2; python_version >= "2.7" and python_version not in "3.0, 3.1, 3.2, 3.3"'), extras=[], editable=False, _parsed_line=<Line (editable=False, name=python-dateutil, path=None, uri=None, extras=(), markers=python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3', vcs=None, specifier===2.8.2, pyproject=None, pyproject_requires=None, pyproject_backend=None, ireq=python-dateutil==2.8.2)>), markers='python_version >= "2.7" and python_version not in "3.0, 3.1, 3.2, 3.3"', _specifiers='==2.8.2', index=None, editable=False, hashes=frozenset({'sha256:0123cacc1627ae19ddf3c27a5de5bd67ee4586fbdd6440d9748f8abb483d3e86', 'sha256:961d03dc3453ebbc59dbdea9e4e11c5651520a876d0f4db161e8674aae935da9'}), extras=(), abstract_dep=None, _line_instance=<Line (editable=False, name=python-dateutil, path=None, uri=None, extras=(), markers=python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3', vcs=None, specifier===2.8.2, pyproject=None, pyproject_requires=None, pyproject_backend=None, ireq=python-dateutil==2.8.2)>, _ireq=None), Requirement(_name='s3transfer', vcs=None, req=NamedRequirement(name='s3transfer', version='==0.6.1', req=Requirement.parse('s3transfer==0.6.1; python_version >= "3.7"'), extras=[], editable=False, _parsed_line=<Line (editable=False, name=s3transfer, path=None, uri=None, extras=(), markers=python_version >= '3.7', vcs=None, specifier===0.6.1, pyproject=None, pyproject_requires=None, pyproject_backend=None, ireq=s3transfer==0.6.1)>), markers='python_version >= "3.7"', _specifiers='==0.6.1', index=None, editable=False, hashes=frozenset({'sha256:3c0da2d074bf35d6870ef157158641178a4204a6e689e82546083e31e0311346', 'sha256:640bb492711f4c0c0905e1f62b6aaeb771881935ad27884852411f8e9cacbca9'}), extras=(), abstract_dep=None, _line_instance=<Line (editable=False, name=s3transfer, path=None, uri=None, extras=(), markers=python_version >= '3.7', vcs=None, specifier===0.6.1, pyproject=None, pyproject_requires=None, pyproject_backend=None, ireq=s3transfer==0.6.1)>, _ireq=None), Requirement(_name='six', vcs=None, req=NamedRequirement(name='six', version='==1.16.0', req=Requirement.parse('six==1.16.0; python_version >= "2.7" and python_version not in "3.0, 3.1, 3.2, 3.3"'), extras=[], editable=False, _parsed_line=<Line (editable=False, name=six, path=None, uri=None, extras=(), markers=python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3', vcs=None, specifier===1.16.0, pyproject=None, pyproject_requires=None, pyproject_backend=None, ireq=six==1.16.0)>), markers='python_version >= "2.7" and python_version not in "3.0, 3.1, 3.2, 3.3"', _specifiers='==1.16.0', index=None, editable=False, hashes=frozenset({'sha256:1e61c37477a1626458e36f7b1d82aa5c9b094fa4802892072e49de9c60c4c926', 'sha256:8abb2f1d86890a2dfb989f9a77cfcfd3e47c2a354b01111771326f8aa26e0254'}), extras=(), abstract_dep=None, _line_instance=<Line (editable=False, name=six, path=None, uri=None, extras=(), markers=python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3', vcs=None, specifier===1.16.0, pyproject=None, pyproject_requires=None, pyproject_backend=None, ireq=six==1.16.0)>, _ireq=None), Requirement(_name='urllib3', vcs=None, req=NamedRequirement(name='urllib3', version='==1.26.16', req=Requirement.parse('urllib3==1.26.16; python_version >= "2.7" and python_version not in "3.0, 3.1, 3.2, 3.3, 3.4, 3.5"'), extras=[], editable=False, _parsed_line=<Line (editable=False, name=urllib3, path=None, uri=None, extras=(), markers=python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4, 3.5', vcs=None, specifier===1.26.16, pyproject=None, pyproject_requires=None, pyproject_backend=None, ireq=urllib3==1.26.16)>), markers='python_version >= "2.7" and python_version not in "3.0, 3.1, 3.2, 3.3, 3.4, 3.5"', _specifiers='==1.26.16', index=None, editable=False, hashes=frozenset({'sha256:8d36afa7616d8ab714608411b4a3b13e58f463aee519024578e062e141dce20f', 'sha256:8f135f6502756bde6b2a9b28989df5fbe87c9970cecaa69041edcce7f0589b14'}), extras=(), abstract_dep=None, _line_instance=<Line (editable=False, name=urllib3, path=None, uri=None, extras=(), markers=python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4, 3.5', vcs=None, specifier===1.26.16, pyproject=None, pyproject_requires=None, pyproject_backend=None, ireq=urllib3==1.26.16)>, _ireq=None)]
#8 41.61  Package installation failed...
------
executor failed running [/bin/sh -c PIPENV_VENV_IN_PROJECT=1 pipenv install --deploy --verbose]: exit code: 1
error: Recipe `_build-image` failed on line 22 with exit code 1

I also tried after pinning version 0.5.0 of pysqlite3-binary, but that didn't change anything.

Do you have any idea where to start when figuring this out? This worked fine last year, but it's not working now. I would expect that boto3 is a pretty common package to run alongside this one, they should definitely work together.

coleifer commented 1 year ago

I'd suggest taking it up with pipenv, it doesn't seem to have anything whatsoever to do with pysqlite3.

The relevant error seems to be this monstrosity:

#8 27.89 ERROR: In --require-hashes mode, all requirements must have their versions pinned with ==. These do not:
#8 27.89     pysqlite3-binary from https://files.pythonhosted.org/packages/e5/af/750aa272a8282f6518a8532f31c51cc9e49815c626cf0df8cdf181367b52/pysqlite3_binary-0.5.0-cp310-cp310-manylinux_2_24_x86_64.whl (from -r /tmp/pipenv-8bdg3b7s-requirements/pipenv-hsujwah9-hashed-reqs.txt (line 4))
nk9 commented 1 year ago

So this is actually because the Pipfile was locked on an M1 Mac, but the pysqlite3-binary package doesn't have M1 wheels. So the locking just skips your package, and no hashes are included in the lock file. When it's then run in Docker on linux/amd, using pipenv install --deploy, there is nothing to download (no hashes), but the Pipfile specifies that the package is required for that platform. So the install just fails.

There are a couple issues in pipenv which discuss this: https://github.com/pypa/pipenv/issues/3902 https://github.com/pypa/pipenv/issues/5130

I also opened my own issue before I worked out what was going on: https://github.com/pypa/pipenv/issues/5723