SwissDataScienceCenter / renku-data-services

Services that handle reading and writing data from a database
Apache License 2.0
3 stars 2 forks source link

Non-deterministic builds #259

Closed leafty closed 4 months ago

leafty commented 5 months ago

Docker image build steps do not make use of poetry.lock files which result in non-deterministic builds.

See: https://github.com/SwissDataScienceCenter/renku-data-services/blob/e37b0ea2e7df192e68f964bcc9cf0dbed30d168d/projects/renku_data_service/Dockerfile#L23-L24

Issue: the use of build-project make the image builds non-deterministic which can result in dependency breakage.

Example: The keycloak package depends on tenacity which got a broken release (8.4.0). The broken package got installed during builds on June 17 2024 when the latest version jumped from 8.3.0 to 8.4.0.

Solution: Use poetry.lock files during builds.

leafty commented 5 months ago

Reference logs: https://productionresultssa16.blob.core.windows.net/actions-results/db470ea2-a60c-4263-bd8f-dfa4ad946db0/workflow-job-run-1c1a233d-194f-5444-7647-abdfad9216db/logs/job/job-logs.txt?rsct=text%2Fplain&se=2024-06-17T13%3A07%3A07Z&sig=lKmwPJ2Il7KDOlj0KTzqC%2BSGpIhTR5rMobHhKz7jiCI%3D&ske=2024-06-17T21%3A36%3A10Z&skoid=ca7593d4-ee42-46cd-af88-8b886a2f84eb&sks=b&skt=2024-06-17T09%3A36%3A10Z&sktid=398a6654-997b-47e9-b12b-9515b896b4de&skv=2023-11-03&sp=r&spr=https&sr=b&st=2024-06-17T12%3A57%3A02Z&sv=2023-11-03

leafty commented 5 months ago

And also:

Defaulted container "data-service" out of: data-service, init-certificates (init)
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/app/env/lib/python3.12/site-packages/renku_data_services/data_api/main.py", line 17, in <module>
    from renku_data_services.app_config import Config
  File "/app/env/lib/python3.12/site-packages/renku_data_services/app_config/__init__.py", line 3, in <module>
    from renku_data_services.app_config.config import Config  # noqa: F401
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/env/lib/python3.12/site-packages/renku_data_services/app_config/config.py", line 35, in <module>
    from renku_data_services.authn.keycloak import KcUserStore, KeycloakAuthenticator
  File "/app/env/lib/python3.12/site-packages/renku_data_services/authn/keycloak.py", line 13, in <module>
    from renku_data_services.utils.core import get_ssl_context
  File "/app/env/lib/python3.12/site-packages/renku_data_services/utils/__init__.py", line 3, in <module>
    from renku_data_services.utils import core, etag
  File "/app/env/lib/python3.12/site-packages/renku_data_services/utils/core.py", line 12, in <module>
    from tenacity import retry, stop_after_attempt, stop_after_delay, wait_fixed
  File "/app/env/lib/python3.12/site-packages/tenacity/__init__.py", line 653, in <module>
    from tenacity.asyncio import AsyncRetrying  # noqa:E402,I100
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ModuleNotFoundError: No module named 'tenacity.asyncio'

(broken tenacity 8.4.0 release)