[Bug] "psycopg2" Dependency fails building Docker Images

[sridharstreaks]

Is this a new bug?

• [X] I believe this is a new bug
• [X] I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

Hi all, Building Docker image fails while installing dbt-postgres in integration with cosmos.

• On this specific step, RUN python -m venv dbt_venv && source dbt_venv/bin/activate && \ pip install --no-cache-dir dbt-postgres && deactivate (adding this in a docker file) and building using astro dev start , I'm receiving the following error 9.396 Collecting psycopg2<3.0,>=2.9 (from dbt-postgres) 9.418 Downloading psycopg2-2.9.9.tar.gz (384 kB) 9.475 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 384.9/384.9 kB 7.4 MB/s eta 0:00:00 9.542 Installing build dependencies: started 12.74 Installing build dependencies: finished with status 'done' 12.74 Getting requirements to build wheel: started 12.94 Getting requirements to build wheel: finished with status 'error' 12.95 error: subprocess-exited-with-error 12.95 12.95 × Getting requirements to build wheel did not run successfully. 12.95 │ exit code: 1 12.95 ╰─> [21 lines of output] 12.95 running egg_info 12.95 writing psycopg2.egg-info/PKG-INFO 12.95 writing dependency_links to psycopg2.egg-info/dependency_links.txt 12.95 writing top-level names to psycopg2.egg-info/top_level.txt 12.95 12.95 Error: pg_config executable not found. 12.95 12.95 pg_config is required to build psycopg2 from source. Please add the directory 12.95 containing pg_config to the $PATH or specify the full executable path with the 12.95 option: 12.95 12.95 python setup.py build_ext --pg-config /path/to/pg_config build ... 12.95 12.95 or with the pg_config option in 'setup.cfg'. 12.95 12.95 If you prefer to avoid building psycopg2 from source, please install the PyPI 12.95 'psycopg2-binary' package instead. 12.95 12.95 For further information please check the 'doc/src/install.rst' file (also at 12.95 <https://www.psycopg.org/docs/install.html>). 12.95 12.95 [end of output] Upon Searching, I guess this is doing something with Postgres and psycopg2 packages. But since that package is used in dbt we're receiving this error. I'm very noob to dbt,airflow,astro and cosmos. So can anyone help me sort this issue out?

Expected Behavior

astro dev start should be building the Airflow along with DBT Adaptor mentinoed in the Docker file. Refer Here:

Steps To Reproduce

1. Install Astro CLI
2. Create a folder and cd into it.
3. run astro dev init to create an empty airflow project
4. Add this # install dbt into a virtual environment RUN python -m venv dbt_venv && source dbt_venv/bin/activate && \ pip install --no-cache-dir <your-dbt-adapter> && deactivate in the Docker file
5. Add astronomer-cosmos in the requirements.txt file
6. run astro dev start to build the Docker Image and Spin up the Airflow container

Relevant log output

No response

Environment

- OS:Windows
- Python:3.10.9
- dbt-postgres:latest

Additional Context

No response

[jtcohen6]

Hey @sridharstreaks - I think this is a duplicate of:

• https://github.com/dbt-labs/dbt-postgres/issues/96

Please see this comment for a full explanation and discussion of the potential paths forward:

• https://github.com/dbt-labs/dbt-postgres/issues/96#issuecomment-2113020011