django-redshift-backend depends directly on psycopg2. #149

[shimizukawa]

for #149.

Previously, django-redshift-backend would let the user choose whether to use psycopg2 or psycopg2-binary. django-redshift-backend relies on psycopg2, so either one is required. However, if they were unaware of the option, they could create an environment where psycopg2 was not installed. So I changed the method to change psycopg2 to a direct dependency, and if you want to select psycopg2-binary additionally, install it with pip install django-redshift-backend[psycopg2-binary].

[shimizukawa]

This PR code works fine with uv pip on uv-0.4.17, but not works with pip-24.2 .

The Problem Addressed in This PR:

• When running pip install django-redshift-backend, the psycopg2 package is not installed, leading to an ImportError exception at runtime.
• To work around this issue, users are required to explicitly run pip install psycopg2 or pip install psycopg2-binary separately.

Purpose of this PR:

1. Ensure that the psycopg2 package is installed with a standard installation: Example: pip install django-redshift-backend

2. For installations with the dev extra, install psycopg2-binary instead of psycopg2: Example: pip install django-redshift-backend[dev]

The Attempted Solution:

To achieve this, I tried using environment markers to conditionally manage dependencies by specifying psycopg2; extra != 'dev'.

note:

• https://packaging.python.org/en/latest/specifications/core-metadata/#provides-extra-multiple-use
• https://peps.python.org/pep-0508/
• https://peps.python.org/pep-0685/

However, this approach still resulted in the psycopg2 package being installed regardless.

(venv) @shimizukawa ➜ /tmp/proj $ pip -V
pip 24.2 from /tmp/proj/venv/lib/python3.8/site-packages/pip (python 3.8)

Below log says: Ignoring psycopg2 and Building wheels for collected packages: psycopg2. It looks confusing.

(venv) @shimizukawa ➜ /tmp/proj $ pip install  /workspaces/django-redshift-backend/dist/django_redshift_backend-4.1.2.dev11-py3-none-any.whl[dev]
Processing /workspaces/django-redshift-backend/dist/django_redshift_backend-4.1.2.dev11-py3-none-any.whl (from django-redshift-backend==4.1.2.dev11)
Collecting django<5.2 (from django-redshift-backend==4.1.2.dev11->django-redshift-backend==4.1.2.dev11)
  Using cached Django-4.2.16-py3-none-any.whl.metadata (4.1 kB)
Collecting psycopg2 (from django-redshift-backend==4.1.2.dev11->django-redshift-backend==4.1.2.dev11)
  Using cached psycopg2-2.9.9.tar.gz (384 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting backports.zoneinfo (from django-redshift-backend==4.1.2.dev11->django-redshift-backend==4.1.2.dev11)
  Using cached backports.zoneinfo-0.2.1-cp38-cp38-manylinux1_x86_64.whl.metadata (4.7 kB)
Ignoring psycopg2: markers 'extra != "dev"' don't match your environment
Requirement already satisfied: psycopg2-binary in ./venv/lib/python3.8/site-packages (from django-redshift-backend==4.1.2.dev11->django-redshift-backend==4.1.2.dev11) (2.9.9)
Collecting asgiref<4,>=3.6.0 (from django<5.2->django-redshift-backend==4.1.2.dev11->django-redshift-backend==4.1.2.dev11)
  Using cached asgiref-3.8.1-py3-none-any.whl.metadata (9.3 kB)
Requirement already satisfied: sqlparse>=0.3.1 in ./venv/lib/python3.8/site-packages (from django<5.2->django-redshift-backend==4.1.2.dev11->django-redshift-backend==4.1.2.dev11) (0.5.1)
Requirement already satisfied: typing-extensions>=4 in ./venv/lib/python3.8/site-packages (from asgiref<4,>=3.6.0->django<5.2->django-redshift-backend==4.1.2.dev11->django-redshift-backend==4.1.2.dev11) (4.12.2)
Using cached Django-4.2.16-py3-none-any.whl (8.0 MB)
Using cached backports.zoneinfo-0.2.1-cp38-cp38-manylinux1_x86_64.whl (74 kB)
Using cached asgiref-3.8.1-py3-none-any.whl (23 kB)
Building wheels for collected packages: psycopg2
  Building wheel for psycopg2 (pyproject.toml) ... error
  error: subprocess-exited-with-error
  × Building wheel for psycopg2 (pyproject.toml) did not run successfully.
  ERROR: Failed building wheel for psycopg2
Failed to build psycopg2
ERROR: ERROR: Failed to build installable wheels for some pyproject.toml based projects (psycopg2)

Upon further investigation, I found discussions indicating that this behavior is "undefined for extras" and that resolving the issue requires changes to PEP rather than pip itself. However, there seems to be no progress on creating the necessary PEP.

• https://github.com/pypa/pip/issues/8686
• https://discuss.python.org/t/adding-a-default-extra-require-environment/4898
• https://discuss.python.org/t/the-extra-environment-marker-and-its-operators/4976

Conclusion:

Although this approach works as expected with uv pip, it does not work with pip. Therefore, I have determined that this method cannot be adopted.