ros-infrastructure / cookbook-ros-buildfarm

Apache License 2.0
2 stars 2 forks source link

Use constraints.txt in pulp image build. #130

Closed nuclearsandwich closed 1 year ago

nuclearsandwich commented 1 year ago

When doing a deploy of repo.ros2.org today, the pulp migration stage failed to boot up with the following stacktrace:

Traceback (most recent call last):
  File "/usr/local/lib/pulp/bin/pulpcore-manager", line 11, in <module>
    sys.exit(manage())
  File "/usr/local/lib/pulp/lib/python3.6/site-packages/pulpcore/app/manage.py", line 11, in manage
    execute_from_command_line(sys.argv)
  File "/usr/lib/python3.6/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
    utility.execute()
  File "/usr/lib/python3.6/site-packages/django/core/management/__init__.py", line 325, in execute
    settings.INSTALLED_APPS
  File "/usr/lib/python3.6/site-packages/django/conf/__init__.py", line 79, in __getattr__
    self._setup(name)
  File "/usr/lib/python3.6/site-packages/django/conf/__init__.py", line 66, in _setup
    self._wrapped = Settings(settings_module)
  File "/usr/lib/python3.6/site-packages/django/conf/__init__.py", line 157, in __init__
    mod = importlib.import_module(self.SETTINGS_MODULE)
  File "/usr/lib64/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/usr/local/lib/pulp/lib/python3.6/site-packages/pulpcore/app/settings.py", line 81, in <module>
    plugin_app_config = entry_point.load()
  File "/usr/local/lib/pulp/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2464, in load
    self.require(*args, **kwargs)
  File "/usr/local/lib/pulp/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2487, in require
    items = working_set.resolve(reqs, env, installer, extras=self.extras)
  File "/usr/local/lib/pulp/lib/python3.6/site-packages/pkg_resources/__init__.py", line 782, in resolve
    raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (redis 3.5.3 (/usr/lib/python3.6/site-packages), Requirement.parse('redis>=4.0.0'), {'rq'})

Our docker build process sources many dependencies from the RHEL 8 and EPEL 8 repositories but although we pin the pip-installed packages we have not pinned any of their dependencies and it seems like a bump in the rq version (annoyingly from minor 1.13 to 1.15 although I'm not their semver dad) has introduced a conflict with the system-installed version of python3-redis. I didn't capture good notes about why jsonschema was upgraded away from the system version (it was copied over during the puppet to chef port) but I didn't take that tactic here because whacking individual packages is just going to create longer term churn and as we're dropping pulp in favor of createrepo-agent we really just need this to run for the sake of the deployment process while we're still decommissioning it.

I was able to capture the pip freeze output from the previously deployed version of pulp_image and I've copied that into the constraints.txt file that is now being considered when running pip install during the docker image build with the intent that a working configuration of python3 packages is locked in for the lifetime of the pulp workflow.

I have also grabbed a copy of the same pip freeze output for the image that won't boot a pulp process for future study and saved it here https://gist.github.com/nuclearsandwich/f423d4a679163e395e3898152b05f745 although I don't know that there will be enough of a reason to return to it.

nuclearsandwich commented 1 year ago

I'm merging this ahead of review in order to unblock the deployment of repo.ros2.org since this affects the repo host rather than Jenkins.