machawk1
commented
2 months ago For example, it references a yamz.sql, which does not exist in the repo. I believe I discussed this file with @cr625 in the past.
Open machawk1 opened 2 months ago
machawk1
commented
2 months ago For example, it references a yamz.sql, which does not exist in the repo. I believe I discussed this file with @cr625 in the past.
machawk1
commented
2 months ago An old commit https://github.com/metadata-research/yamz/tree/778566e10eadf2e2eeddd7ad859f96df11b423c8 has a yamz.sql file but it has since been deleted.
machawk1
commented
2 months ago Updated the config file reference in 286e5ba665d5aa5d8cb02ebbbe930c1e30a051b9 . Pushed straight to main, as it should not affect anything else.
Even with these changes, the container fails to build (macOS M1) when installing the requirements.
machawk1@Mat-mini yamz % docker build -t yamz .
[+] Building 15.1s (16/18)
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 82B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 35B 0.0s
=> [internal] load metadata for docker.io/library/ubuntu:focal 0.2s
=> [ 1/14] FROM docker.io/library/ubuntu:focal@sha256:874aca52f79ae5f8258faff03e10ce99ae836f6e7d2df6ecd3da5c1cad3a912b 0.0s
=> [internal] load build context 0.0s
=> => transferring context: 343B 0.0s
=> CACHED [ 2/14] RUN apt-get update && apt-get install -y tzdata python3 python3-pip git 0.0s
=> CACHED [ 3/14] RUN apt-get install -y postgresql-12 0.0s
=> CACHED [ 4/14] RUN git clone https://github.com/metadata-research/yamz.git 0.0s
=> CACHED [ 5/14] RUN chown -R postgres /yamz 0.0s
=> CACHED [ 6/14] RUN echo "host all all 0.0.0.0/0 md5" >> /etc/postgresql/12/main/pg_hba.conf 0.0s
=> CACHED [ 7/14] RUN echo "listen_addresses='*'" >> /etc/postgresql/12/main/postgresql.conf 0.0s
=> CACHED [ 8/14] RUN /etc/init.d/postgresql start && psql --command "ALTER USER postgres WITH SUPERUSER PASSWORD 'PASS';" && createdb -O postgres yamz 0.0s
=> CACHED [ 9/14] WORKDIR /yamz 0.0s
=> CACHED [10/14] COPY _config.py /yamz/config.py 0.0s
=> CACHED [11/14] COPY requirements.txt . 0.0s
=> ERROR [12/14] RUN python3 -m pip install -r requirements.txt 14.8s
------
> [12/14] RUN python3 -m pip install -r requirements.txt:
#16 0.722 Collecting alembic==1.8.1
#16 0.764 Downloading alembic-1.8.1-py3-none-any.whl (209 kB)
#16 0.831 Collecting beautifulsoup4==4.11.1
#16 0.845 Downloading beautifulsoup4-4.11.1-py3-none-any.whl (128 kB)
#16 0.884 Collecting bleach==5.0.1
#16 0.899 Downloading bleach-5.0.1-py3-none-any.whl (160 kB)
#16 0.936 Collecting blinker==1.5
#16 0.948 Downloading blinker-1.5-py2.py3-none-any.whl (12 kB)
#16 0.979 Collecting certifi==2022.9.14
#16 0.993 Downloading certifi-2022.9.14-py3-none-any.whl (162 kB)
#16 1.125 Collecting charset-normalizer==2.1.1
#16 1.137 Downloading charset_normalizer-2.1.1-py3-none-any.whl (39 kB)
#16 1.172 Collecting click==8.1.3
#16 1.185 Downloading click-8.1.3-py3-none-any.whl (96 kB)
#16 1.216 Collecting dnspython==2.2.1
#16 1.232 Downloading dnspython-2.2.1-py3-none-any.whl (269 kB)
#16 1.281 Collecting email-validator==1.3.0
#16 1.292 Downloading email_validator-1.3.0-py2.py3-none-any.whl (22 kB)
#16 1.327 Collecting Flask==2.2.2
#16 1.338 Downloading Flask-2.2.2-py3-none-any.whl (101 kB)
#16 1.371 Collecting Flask-Admin==1.6.0
#16 1.389 Downloading Flask-Admin-1.6.0.tar.gz (6.6 MB)
#16 2.042 Collecting Flask-Login==0.6.2
#16 2.053 Downloading Flask_Login-0.6.2-py3-none-any.whl (17 kB)
#16 2.076 Collecting Flask-Mail==0.9.1
#16 2.087 Downloading Flask-Mail-0.9.1.tar.gz (45 kB)
#16 2.209 Collecting Flask-Migrate==3.1.0
#16 2.225 Downloading Flask_Migrate-3.1.0-py3-none-any.whl (20 kB)
#16 2.247 Collecting Flask-PageDown==0.4.0
#16 2.259 Downloading Flask_PageDown-0.4.0-py3-none-any.whl (5.5 kB)
#16 2.282 Collecting Flask-Reuploaded==1.2.0
#16 2.294 Downloading Flask_Reuploaded-1.2.0-py3-none-any.whl (14 kB)
#16 2.324 Collecting Flask-SQLAlchemy==2.5.1
#16 2.336 Downloading Flask_SQLAlchemy-2.5.1-py2.py3-none-any.whl (17 kB)
#16 2.365 Collecting Flask-WTF==1.0.1
#16 2.377 Downloading Flask_WTF-1.0.1-py3-none-any.whl (12 kB)
#16 2.607 Collecting greenlet==1.1.3
#16 2.625 Downloading greenlet-1.1.3-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (158 kB)
#16 2.657 Collecting html5lib==1.1
#16 2.674 Downloading html5lib-1.1-py2.py3-none-any.whl (112 kB)
#16 2.706 Collecting idna==3.4
#16 2.718 Downloading idna-3.4-py3-none-any.whl (61 kB)
#16 2.775 Collecting importlib-metadata==4.12.0
#16 2.791 Downloading importlib_metadata-4.12.0-py3-none-any.whl (21 kB)
#16 2.842 Collecting importlib-resources==5.9.0
#16 2.858 Downloading importlib_resources-5.9.0-py3-none-any.whl (33 kB)
#16 2.891 Collecting itsdangerous==2.1.2
#16 2.911 Downloading itsdangerous-2.1.2-py3-none-any.whl (15 kB)
#16 2.941 Collecting Jinja2==3.1.2
#16 2.956 Downloading Jinja2-3.1.2-py3-none-any.whl (133 kB)
#16 3.273 Collecting lxml==4.9.1
#16 3.295 Downloading lxml-4.9.1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.manylinux_2_24_aarch64.whl (6.6 MB)
#16 3.540 Collecting Mako==1.2.2
#16 3.553 Downloading Mako-1.2.2-py3-none-any.whl (78 kB)
#16 3.590 Collecting Markdown==3.4.1
#16 3.609 Downloading Markdown-3.4.1-py3-none-any.whl (93 kB)
#16 3.700 Collecting MarkupSafe==2.1.1
#16 3.713 Downloading MarkupSafe-2.1.1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (26 kB)
#16 4.011 Collecting numpy==1.23.3
#16 4.028 Downloading numpy-1.23.3-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (14.0 MB)
#16 4.656 Collecting orcid==1.0.3
#16 4.669 Downloading orcid-1.0.3.tar.gz (10 kB)
#16 5.003 Collecting pandas==1.5.0
#16 5.034 Downloading pandas-1.5.0-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (11.5 MB)
#16 5.649 Collecting psycopg2-binary==2.9.3
#16 5.664 Downloading psycopg2-binary-2.9.3.tar.gz (380 kB)
#16 5.841 Collecting python-dateutil==2.8.2
#16 5.857 Downloading python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
#16 5.931 Collecting pytz==2022.2.1
#16 5.945 Downloading pytz-2022.2.1-py2.py3-none-any.whl (500 kB)
#16 6.013 Collecting rauth==0.7.3
#16 6.026 Downloading rauth-0.7.3.tar.gz (16 kB)
#16 6.160 Collecting requests==2.28.1
#16 6.171 Downloading requests-2.28.1-py3-none-any.whl (62 kB)
#16 6.326 Collecting simplejson==3.17.6
#16 6.340 Downloading simplejson-3.17.6-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (134 kB)
#16 6.370 Collecting six==1.16.0
#16 6.380 Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
#16 6.409 Collecting soupsieve==2.3.2.post1
#16 6.420 Downloading soupsieve-2.3.2.post1-py3-none-any.whl (37 kB)
#16 6.898 Collecting SQLAlchemy==1.4.41
#16 6.924 Downloading SQLAlchemy-1.4.41-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (1.6 MB)
#16 7.082 Collecting urllib3==1.26.12
#16 7.095 Downloading urllib3-1.26.12-py2.py3-none-any.whl (140 kB)
#16 7.128 Collecting webencodings==0.5.1
#16 7.138 Downloading webencodings-0.5.1-py2.py3-none-any.whl (11 kB)
#16 7.181 Collecting Werkzeug==2.2.2
#16 7.196 Downloading Werkzeug-2.2.2-py3-none-any.whl (232 kB)
#16 7.232 Collecting WTForms==3.0.1
#16 7.246 Downloading WTForms-3.0.1-py3-none-any.whl (136 kB)
#16 7.287 Collecting zipp==3.8.1
#16 7.300 Downloading zipp-3.8.1-py3-none-any.whl (5.6 kB)
#16 7.309 Building wheels for collected packages: Flask-Admin, Flask-Mail, orcid, psycopg2-binary, rauth
#16 7.310 Building wheel for Flask-Admin (setup.py): started
#16 8.539 Building wheel for Flask-Admin (setup.py): finished with status 'done'
#16 8.544 Created wheel for Flask-Admin: filename=Flask_Admin-1.6.0-py3-none-any.whl size=7496260 sha256=5358c120cc7f98b0d09f83ed0237f298ecb3a942e08e1836e503c31b0b0a2e48
#16 8.544 Stored in directory: /var/lib/postgresql/.cache/pip/wheels/29/3b/fd/4efe7f1799ddc495661193f39fc73b86af6bc0b5e69db47f2c
#16 8.545 Building wheel for Flask-Mail (setup.py): started
#16 8.650 Building wheel for Flask-Mail (setup.py): finished with status 'done'
#16 8.650 Created wheel for Flask-Mail: filename=Flask_Mail-0.9.1-py3-none-any.whl size=7568 sha256=e9ce7a225de09ffe5a8416143ccb59b0b76418a6e6f170644741a52cbb97ed46
#16 8.650 Stored in directory: /var/lib/postgresql/.cache/pip/wheels/98/bc/8c/34c329e4d7efeaf7b9886db0c76d0b23170e54de443f688e3c
#16 8.651 Building wheel for orcid (setup.py): started
#16 8.767 Building wheel for orcid (setup.py): finished with status 'done'
#16 8.768 Created wheel for orcid: filename=orcid-1.0.3-py3-none-any.whl size=10440 sha256=750f815846214985e34f2bbf2e258a3d7385b2d1899ddbf60f3592cff1d9dd49
#16 8.768 Stored in directory: /var/lib/postgresql/.cache/pip/wheels/6d/b9/f6/f0090699772788f01e8cc4cc76b69aa9f91f08a76ed2f8fca8
#16 8.769 Building wheel for psycopg2-binary (setup.py): started
#16 8.916 Building wheel for psycopg2-binary (setup.py): finished with status 'error'
#16 8.916 ERROR: Command errored out with exit status 1:
#16 8.916 command: /usr/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-kn4zfkyu/psycopg2-binary/setup.py'"'"'; __file__='"'"'/tmp/pip-install-kn4zfkyu/psycopg2-binary/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-6sw8zm1z
#16 8.916 cwd: /tmp/pip-install-kn4zfkyu/psycopg2-binary/
#16 8.916 Complete output (40 lines):
#16 8.916 running bdist_wheel
#16 8.916 running build
#16 8.916 running build_py
#16 8.916 creating build
#16 8.916 creating build/lib.linux-aarch64-3.8
#16 8.916 creating build/lib.linux-aarch64-3.8/psycopg2
#16 8.916 copying lib/_ipaddress.py -> build/lib.linux-aarch64-3.8/psycopg2
#16 8.916 copying lib/__init__.py -> build/lib.linux-aarch64-3.8/psycopg2
#16 8.916 copying lib/errorcodes.py -> build/lib.linux-aarch64-3.8/psycopg2
#16 8.916 copying lib/_json.py -> build/lib.linux-aarch64-3.8/psycopg2
#16 8.916 copying lib/_range.py -> build/lib.linux-aarch64-3.8/psycopg2
#16 8.916 copying lib/pool.py -> build/lib.linux-aarch64-3.8/psycopg2
#16 8.916 copying lib/errors.py -> build/lib.linux-aarch64-3.8/psycopg2
#16 8.916 copying lib/extras.py -> build/lib.linux-aarch64-3.8/psycopg2
#16 8.916 copying lib/sql.py -> build/lib.linux-aarch64-3.8/psycopg2
#16 8.916 copying lib/extensions.py -> build/lib.linux-aarch64-3.8/psycopg2
#16 8.916 copying lib/tz.py -> build/lib.linux-aarch64-3.8/psycopg2
#16 8.916 warning: build_py: byte-compiling is disabled, skipping.
#16 8.916
#16 8.916 running build_ext
#16 8.916 building 'psycopg2._psycopg' extension
#16 8.916 creating build/temp.linux-aarch64-3.8
#16 8.916 creating build/temp.linux-aarch64-3.8/psycopg
#16 8.916 aarch64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DPSYCOPG_VERSION=2.9.3 (dt dec pq3 ext lo64) -DPSYCOPG_DEBUG=1 -DPG_VERSION_NUM=120018 -DHAVE_LO64=1 -DPSYCOPG_DEBUG=1 -I/usr/include/python3.8 -I. -I/usr/include/postgresql -I/usr/include/postgresql/12/server -I/usr/include/libxml2 -I/usr/include/mit-krb5 -c psycopg/psycopgmodule.c -o build/temp.linux-aarch64-3.8/psycopg/psycopgmodule.o -Wdeclaration-after-statement
#16 8.916 In file included from psycopg/psycopgmodule.c:28:
#16 8.916 ./psycopg/psycopg.h:36:10: fatal error: libpq-fe.h: No such file or directory
#16 8.916 36 | #include
#16 8.916 | ^~~~~~~~~~~~
#16 8.916 compilation terminated.
#16 8.916
#16 8.916 It appears you are missing some prerequisite to build the package from source.
#16 8.916
#16 8.916 You may install a binary package by installing 'psycopg2-binary' from PyPI.
#16 8.916 If you want to install psycopg2 from source, please install the packages
#16 8.916 required for the build and try again.
#16 8.916
#16 8.916 For further information please check the 'doc/src/install.rst' file (also at
#16 8.916 ).
#16 8.916
#16 8.916 error: command 'aarch64-linux-gnu-gcc' failed with exit status 1
#16 8.916 ----------------------------------------
#16 8.916 ERROR: Failed building wheel for psycopg2-binary
#16 8.916 Running setup.py clean for psycopg2-binary
#16 8.984 Building wheel for rauth (setup.py): started
#16 9.092 Building wheel for rauth (setup.py): finished with status 'done'
#16 9.092 Created wheel for rauth: filename=rauth-0.7.3-py3-none-any.whl size=16057 sha256=678f378b0e2f024b70260779558199ea9ec40a47e4b3ed760464222b9fa23ac1
#16 9.092 Stored in directory: /var/lib/postgresql/.cache/pip/wheels/12/ba/46/fba7d3f89b3f871b076908875f0b61fa6fba64af01c7b53036
#16 9.093 Successfully built Flask-Admin Flask-Mail orcid rauth
#16 9.093 Failed to build psycopg2-binary
#16 9.201 Installing collected packages: greenlet, SQLAlchemy, zipp, importlib-resources, importlib-metadata, MarkupSafe, Mako, alembic, soupsieve, beautifulsoup4, six, webencodings, bleach, blinker, certifi, charset-normalizer, click, dnspython, idna, email-validator, Jinja2, Werkzeug, itsdangerous, Flask, WTForms, Flask-Admin, Flask-Login, Flask-Mail, Flask-SQLAlchemy, Flask-Migrate, Flask-PageDown, Flask-Reuploaded, Flask-WTF, html5lib, lxml, Markdown, numpy, urllib3, requests, simplejson, orcid, pytz, python-dateutil, pandas, psycopg2-binary, rauth
#16 9.669 WARNING: The script mako-render is installed in '/var/lib/postgresql/.local/bin' which is not on PATH.
#16 9.669 Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
#16 9.740 WARNING: The script alembic is installed in '/var/lib/postgresql/.local/bin' which is not on PATH.
#16 9.740 Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
#16 9.881 WARNING: The script normalizer is installed in '/var/lib/postgresql/.local/bin' which is not on PATH.
#16 9.881 Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
#16 10.04 WARNING: The script email_validator is installed in '/var/lib/postgresql/.local/bin' which is not on PATH.
#16 10.04 Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
#16 10.18 WARNING: The script flask is installed in '/var/lib/postgresql/.local/bin' which is not on PATH.
#16 10.18 Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
#16 10.93 WARNING: The script markdown_py is installed in '/var/lib/postgresql/.local/bin' which is not on PATH.
#16 10.93 Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
#16 11.96 WARNING: The scripts f2py, f2py3 and f2py3.8 are installed in '/var/lib/postgresql/.local/bin' which is not on PATH.
#16 11.96 Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
#16 14.21 Running setup.py install for psycopg2-binary: started
#16 14.36 Running setup.py install for psycopg2-binary: finished with status 'error'
#16 14.36 ERROR: Command errored out with exit status 1:
#16 14.36 command: /usr/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-kn4zfkyu/psycopg2-binary/setup.py'"'"'; __file__='"'"'/tmp/pip-install-kn4zfkyu/psycopg2-binary/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-_ty6vci5/install-record.txt --single-version-externally-managed --user --prefix= --compile --install-headers /var/lib/postgresql/.local/include/python3.8/psycopg2-binary
#16 14.36 cwd: /tmp/pip-install-kn4zfkyu/psycopg2-binary/
#16 14.36 Complete output (40 lines):
#16 14.36 running install
#16 14.36 running build
#16 14.36 running build_py
#16 14.36 creating build
#16 14.36 creating build/lib.linux-aarch64-3.8
#16 14.36 creating build/lib.linux-aarch64-3.8/psycopg2
#16 14.36 copying lib/_ipaddress.py -> build/lib.linux-aarch64-3.8/psycopg2
#16 14.36 copying lib/__init__.py -> build/lib.linux-aarch64-3.8/psycopg2
#16 14.36 copying lib/errorcodes.py -> build/lib.linux-aarch64-3.8/psycopg2
#16 14.36 copying lib/_json.py -> build/lib.linux-aarch64-3.8/psycopg2
#16 14.36 copying lib/_range.py -> build/lib.linux-aarch64-3.8/psycopg2
#16 14.36 copying lib/pool.py -> build/lib.linux-aarch64-3.8/psycopg2
#16 14.36 copying lib/errors.py -> build/lib.linux-aarch64-3.8/psycopg2
#16 14.36 copying lib/extras.py -> build/lib.linux-aarch64-3.8/psycopg2
#16 14.36 copying lib/sql.py -> build/lib.linux-aarch64-3.8/psycopg2
#16 14.36 copying lib/extensions.py -> build/lib.linux-aarch64-3.8/psycopg2
#16 14.36 copying lib/tz.py -> build/lib.linux-aarch64-3.8/psycopg2
#16 14.36 warning: build_py: byte-compiling is disabled, skipping.
#16 14.36
#16 14.36 running build_ext
#16 14.36 building 'psycopg2._psycopg' extension
#16 14.36 creating build/temp.linux-aarch64-3.8
#16 14.36 creating build/temp.linux-aarch64-3.8/psycopg
#16 14.36 aarch64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DPSYCOPG_VERSION=2.9.3 (dt dec pq3 ext lo64) -DPSYCOPG_DEBUG=1 -DPG_VERSION_NUM=120018 -DHAVE_LO64=1 -DPSYCOPG_DEBUG=1 -I/usr/include/python3.8 -I. -I/usr/include/postgresql -I/usr/include/postgresql/12/server -I/usr/include/libxml2 -I/usr/include/mit-krb5 -c psycopg/psycopgmodule.c -o build/temp.linux-aarch64-3.8/psycopg/psycopgmodule.o -Wdeclaration-after-statement
#16 14.36 In file included from psycopg/psycopgmodule.c:28:
#16 14.36 ./psycopg/psycopg.h:36:10: fatal error: libpq-fe.h: No such file or directory
#16 14.36 36 | #include
#16 14.36 | ^~~~~~~~~~~~
#16 14.36 compilation terminated.
#16 14.36
#16 14.36 It appears you are missing some prerequisite to build the package from source.
#16 14.36
#16 14.36 You may install a binary package by installing 'psycopg2-binary' from PyPI.
#16 14.36 If you want to install psycopg2 from source, please install the packages
#16 14.36 required for the build and try again.
#16 14.36
#16 14.36 For further information please check the 'doc/src/install.rst' file (also at
#16 14.36 ).
#16 14.36
#16 14.36 error: command 'aarch64-linux-gnu-gcc' failed with exit status 1
#16 14.36 ----------------------------------------
#16 14.50 ERROR: Command errored out with exit status 1: /usr/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-kn4zfkyu/psycopg2-binary/setup.py'"'"'; __file__='"'"'/tmp/pip-install-kn4zfkyu/psycopg2-binary/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-_ty6vci5/install-record.txt --single-version-externally-managed --user --prefix= --compile --install-headers /var/lib/postgresql/.local/include/python3.8/psycopg2-binary Check the logs for full command output.
------
executor failed running [/bin/sh -c python3 -m pip install -r requirements.txt]: exit code: 1
machawk1
commented
2 months ago psycopg2-binary installation seems to be an issue. I updated the requirements file to use the latest (v. 2.9.9) of the module and the Dockerfile builds.
cr625
commented
2 months ago The outstanding issue involves the yamz.sql file, which is a backup of the yamz terms exported from the database (as described in the readme). Without that file there is just an empty copy of the flask app that runs but doesn't do anything so we were going to decide what the docker distro would include as terms.
machawk1
commented
2 months ago @cr625 I believe it would be good for the project to supply some sample data just to allow the user to test the base functionality of the app -- even if it's not a dump from the prod db.
machawk1
commented
2 months ago Sourcing the old yamz.sql file is a little problematic. After I build the image:
docker build -t yamz .
...then run the container with port mapping (5000 is a conflict):
docker run -p 5001:5000 yamz
...click browse then a term, I get an error when accessing the term on the web:
[2024-05-22 16:28:50,784] ERROR in app: Exception on /term/alternates/user community [GET]
Traceback (most recent call last):
File "/var/lib/postgresql/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context
self.dialect.do_execute(
File "/var/lib/postgresql/.local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute
cursor.execute(statement, parameters)
psycopg2.errors.UndefinedColumn: column terms.search_vector does not exist
LINE 1: ...S terms_concept_id, terms.status AS terms_status, terms.sear...
^
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/var/lib/postgresql/.local/lib/python3.8/site-packages/flask/app.py", line 2525, in wsgi_app
response = self.full_dispatch_request()
File "/var/lib/postgresql/.local/lib/python3.8/site-packages/flask/app.py", line 1822, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/var/lib/postgresql/.local/lib/python3.8/site-packages/flask/app.py", line 1820, in full_dispatch_request
rv = self.dispatch_request()
File "/var/lib/postgresql/.local/lib/python3.8/site-packages/flask/app.py", line 1796, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
File "/yamz/app/term/views.py", line 130, in show_alternate_terms
return render_template(
File "/var/lib/postgresql/.local/lib/python3.8/site-packages/flask/templating.py", line 147, in render_template
return _render(app, template, context)
File "/var/lib/postgresql/.local/lib/python3.8/site-packages/flask/templating.py", line 130, in _render
rv = template.render(context)
File "/var/lib/postgresql/.local/lib/python3.8/site-packages/jinja2/environment.py", line 1301, in render
self.environment.handle_exception()
File "/var/lib/postgresql/.local/lib/python3.8/site-packages/jinja2/environment.py", line 936, in handle_exception
raise rewrite_traceback_stack(source=source)
File "/yamz/app/term/templates/term/display_terms.jinja", line 1, in top-level template code
{% extends './base.jinja' %}
File "/yamz/app/templates/base.jinja", line 115, in top-level template code
{% block content %}
File "/yamz/app/term/templates/term/display_terms.jinja", line 22, in block 'content'
{% for selected_term in selected_terms %}
File "/var/lib/postgresql/.local/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 2900, in __iter__
return self._iter().__iter__()
File "/var/lib/postgresql/.local/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 2907, in _iter
result = self.session.execute(
File "/var/lib/postgresql/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 1712, in execute
result = conn._execute_20(statement, params or {}, execution_options)
File "/var/lib/postgresql/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20
return meth(self, args_10style, kwargs_10style, execution_options)
File "/var/lib/postgresql/.local/lib/python3.8/site-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection
return connection._execute_clauseelement(
File "/var/lib/postgresql/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement
ret = self._execute_context(
File "/var/lib/postgresql/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context
self._handle_dbapi_exception(
File "/var/lib/postgresql/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 2124, in _handle_dbapi_exception
util.raise_(
File "/var/lib/postgresql/.local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 208, in raise_
raise exception
File "/var/lib/postgresql/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context
self.dialect.do_execute(
File "/var/lib/postgresql/.local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (psycopg2.errors.UndefinedColumn) column terms.search_vector does not exist
LINE 1: ...S terms_concept_id, terms.status AS terms_status, terms.sear...
^
[SQL: SELECT terms.class AS terms_class, terms.id AS terms_id, terms.ark_id AS terms_ark_id, terms.shoulder AS terms_shoulder, terms.naan AS terms_naan, terms.owner_id AS terms_owner_id, terms.created AS terms_created, terms.modified AS terms_modified, terms.term_string AS terms_term_string, terms.definition AS terms_definition, terms.definition_html AS terms_definition_html, terms.examples AS terms_examples, terms.examples_html AS terms_examples_html, terms.concept_id AS terms_concept_id, terms.status AS terms_status, terms.search_vector AS terms_search_vector
FROM terms
WHERE terms.term_string = %(term_string_1)s AND terms.status = %(status_1)s]
[parameters: {'term_string_1': 'user community', 'status_1': 'published'}]
(Background on this error at: https://sqlalche.me/e/14/f405)
Exception on /term/alternates/user community [GET]
Traceback (most recent call last):
File "/var/lib/postgresql/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context
self.dialect.do_execute(
File "/var/lib/postgresql/.local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute
cursor.execute(statement, parameters)
psycopg2.errors.UndefinedColumn: column terms.search_vector does not exist
LINE 1: ...S terms_concept_id, terms.status AS terms_status, terms.sear...
^
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/var/lib/postgresql/.local/lib/python3.8/site-packages/flask/app.py", line 2525, in wsgi_app
response = self.full_dispatch_request()
File "/var/lib/postgresql/.local/lib/python3.8/site-packages/flask/app.py", line 1822, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/var/lib/postgresql/.local/lib/python3.8/site-packages/flask/app.py", line 1820, in full_dispatch_request
rv = self.dispatch_request()
File "/var/lib/postgresql/.local/lib/python3.8/site-packages/flask/app.py", line 1796, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
File "/yamz/app/term/views.py", line 130, in show_alternate_terms
return render_template(
File "/var/lib/postgresql/.local/lib/python3.8/site-packages/flask/templating.py", line 147, in render_template
return _render(app, template, context)
File "/var/lib/postgresql/.local/lib/python3.8/site-packages/flask/templating.py", line 130, in _render
rv = template.render(context)
File "/var/lib/postgresql/.local/lib/python3.8/site-packages/jinja2/environment.py", line 1301, in render
self.environment.handle_exception()
File "/var/lib/postgresql/.local/lib/python3.8/site-packages/jinja2/environment.py", line 936, in handle_exception
raise rewrite_traceback_stack(source=source)
File "/yamz/app/term/templates/term/display_terms.jinja", line 1, in top-level template code
{% extends './base.jinja' %}
File "/yamz/app/templates/base.jinja", line 115, in top-level template code
{% block content %}
File "/yamz/app/term/templates/term/display_terms.jinja", line 22, in block 'content'
{% for selected_term in selected_terms %}
File "/var/lib/postgresql/.local/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 2900, in __iter__
return self._iter().__iter__()
File "/var/lib/postgresql/.local/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 2907, in _iter
result = self.session.execute(
File "/var/lib/postgresql/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 1712, in execute
result = conn._execute_20(statement, params or {}, execution_options)
File "/var/lib/postgresql/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20
return meth(self, args_10style, kwargs_10style, execution_options)
File "/var/lib/postgresql/.local/lib/python3.8/site-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection
return connection._execute_clauseelement(
File "/var/lib/postgresql/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement
ret = self._execute_context(
File "/var/lib/postgresql/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context
self._handle_dbapi_exception(
File "/var/lib/postgresql/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 2124, in _handle_dbapi_exception
util.raise_(
File "/var/lib/postgresql/.local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 208, in raise_
raise exception
File "/var/lib/postgresql/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context
self.dialect.do_execute(
File "/var/lib/postgresql/.local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (psycopg2.errors.UndefinedColumn) column terms.search_vector does not exist
LINE 1: ...S terms_concept_id, terms.status AS terms_status, terms.sear...
^
[SQL: SELECT terms.class AS terms_class, terms.id AS terms_id, terms.ark_id AS terms_ark_id, terms.shoulder AS terms_shoulder, terms.naan AS terms_naan, terms.owner_id AS terms_owner_id, terms.created AS terms_created, terms.modified AS terms_modified, terms.term_string AS terms_term_string, terms.definition AS terms_definition, terms.definition_html AS terms_definition_html, terms.examples AS terms_examples, terms.examples_html AS terms_examples_html, terms.concept_id AS terms_concept_id, terms.status AS terms_status, terms.search_vector AS terms_search_vector
FROM terms
WHERE terms.term_string = %(term_string_1)s AND terms.status = %(status_1)s]
[parameters: {'term_string_1': 'user community', 'status_1': 'published'}]
(Background on this error at: https://sqlalche.me/e/14/f405)
172.17.0.1 - - [22/May/2024 16:28:50] "GET /term/alternates/user%20community HTTP/1.1" 500 -
172.17.0.1 - - [22/May/2024 16:28:50] "GET /static/css/site.css HTTP/1.1" 304 -
172.17.0.1 - - [22/May/2024 16:28:50] "GET /static/images/yamz_purple_logo_square_small.png HTTP/1.1" 304 -
172.17.0.1 - - [22/May/2024 16:28:50] "GET /static/images/favicon.ico HTTP/1.1" 304
Perhaps this is attributed to the code expecting some different DB structure than what is provided in the old yamz.sql. A fresh, usable dump of yamz.sql would be good to have for testing. Can this be generated from production and perhaps pared down to test the fundamentals?
cr625
commented
2 months ago Perhaps this is attributed to the code expecting some different DB structure than what is provided in the old yamz.sql. A fresh, usable dump of yamz.sql would be good to have for testing. Can this be generated from production and perhaps pared down to test the fundamentals?
We did change the columns, you're right to add search_vector. We could generate the dump file from a subset. I think the simplest way is to pick a filter and export only those terms. For example, we could choose the NASA helio data as our sample data or something from material science. The other option I think would be to generate a test set but that is time consuming, I'm sure.
machawk1
commented
2 months ago @cr625 Extracting a subset from production (e.g., helio) would be a good start. If it's not to much trouble, please do so.
machawk1
commented
3 weeks ago @cr625 Can you supply the command you were using on the prod server to obtain the backup?
I do not believe I have credentials for that server. Obtaining a dump that aligns with the current main branch code is a blocker for this GitHub issue.
machawk1
commented
3 weeks ago I began stripping out a yamz.sql based on the above just to get up and running and have added that to the issue-157 branch. It would be far less laborious and more accurate to the implementation to have a modern dump that aligns with the prod db schema.
The repo has a Dockerfile. How does one use it? Let's add documentation. If it's unstable/unreliable, it would be worthwhile to mention this as well.