Lock versions of dependencies

[ohthehugemanatee]

Today I tried running the update playbook and ended up in a broken state because of docker/docker-py#3256 . I suggest pinning versions of the installed packages to avoid this kind of breakage in future.

[shaderecker]

Hi @ohthehugemanatee, I welcome the idea in general, but don't really know how it can be implemented efficiently. Currently we have one pip package here https://github.com/shaderecker/ansible-pihole/blob/master/roles/docker/tasks/main.yaml#L29 (docker) But only pinning this package alone won't do the trick since indirect dependencies are still not pinned then. Not sure what would be the best approach here, but welcome to ideas.

[bramwalet]

Hi @shaderecker,

I have stumbled upon your repository while searching for an example setting up Pi-Hole with Ansible. Great work!

I've forked your repository to make some changes of my own. For example, implementing a custom /etc/dnsmasq.d file for wildcard DNS lookups (Local DNS). Also, I've implemented synchronization using Orbital Sync.

My repository can be found here: https://github.com/bramwalet/ansible-pihole-cluster

I got a lot of inspiration watching Jeff Geerlings content and specifically this presentation: https://www.youtube.com/watch?v=kNDL13MJG6Y. I already implemented version locking of all my depencencies. I use the Renovate bot in order to check for the latest version.

Renovate bot:

• See the documentation how to set this up: https://docs.renovatebot.com/modules/platform/github/
• You can see all my listed dependencies in the Dependency Dashboard: https://github.com/bramwalet/ansible-pihole-cluster/issues/1
• You can see the bot in action on one of the closed Pull requests: https://github.com/bramwalet/ansible-pihole-cluster/pull/8

How to specify versions:

• An example of specifying versions of collections can be found here: https://github.com/bramwalet/ansible-pihole-cluster/blob/master/requirements.yml
• An example of specifying versions of the docker containers I am using can be found here: https://github.com/bramwalet/ansible-pihole-cluster/blob/135dc20f019c6877f3fbad8c2b7b3b8831a96ef0/roles/pihole/tasks/main.yml#L52 (Please note that this does not work when you are using Ansible variables in order to define the used Docker image somewhere else as a variable)

In order for this to work efficiently, I am implementing integration tests using Ansible Molecule. I have a single node setup working currently (See the GitHub Actions tab on my Repo) and working on a clustered version. One issue I'm stumbling upon is the use of ansible_host for the IP address within the Orbital Sync task.

I am also refactoring the playbooks. It looks like your playbooks are written to be performed in a specific sequence (bootstrap-pihole.yaml, keepalived to enable HA, then periodically sync.yaml and update-pihole.yaml). I am working towards a playbook setup where I define one playbook to cover the entire installation of the Pi including all necessary software (based upon your settings, so HA=true or false), and one playbook (which imports the master playbook) which skips certain roles or tasks (using Ansible tags) in order to update all necessary software (this will skip for example the bootstrap role).

Let me know what you think of my setup!

[shaderecker]

Hi @bramwalet, Thank you for sharing this, it is really cool stuff that you are building there! I'm looking forward to seeing how the clustered version will work.

Thank you also for the suggestion with renovatebot. I am wondering if this would also help with this specific problem that @ohthehugemanatee found (pinning the version of the docker pip package and its indirect dependencies)?

[bramwalet]

Hi @bramwalet, Thank you for sharing this, it is really cool stuff that you are building there! I'm looking forward to seeing how the clustered version will work.

Thanks! I will keep you updated. I committed my work in progress in a separate branch.

I am wondering if this would also help with this specific problem that @ohthehugemanatee found (pinning the version of the docker pip package and its indirect dependencies)?

Ansible collection community.docker has fixed this problem in version 3.10.2, see the release notes: https://docs.ansible.com/ansible/latest/collections/community/docker/changelog.html#v3-10-2

I've tested this:

1. I created a separate branch
2. Triggered a CI run (Lint + Molecule Test) with version 3.10.4 (latest): https://github.com/bramwalet/ansible-pihole-cluster/actions/runs/9711370574
3. I then downgraded community.docker plugin to 3.10.1: https://github.com/bramwalet/ansible-pihole-cluster/commit/90da264c215705d4d2139df61ec65f7d171328d4
4. Triggered a CI run (Lint + Molecule test), see the result here: https://github.com/bramwalet/ansible-pihole-cluster/actions/runs/9711466701 It then breaks because of the requests problem mentioned in this issue.

However, while running the test against version 3.10.0 of the collection, the build also breaks. It looks like community.docker hasn't fixed their dependency versions (requests) either, so then it won't fix the problem.

[bramwalet]

I realized community.docker requires requests to be installed, it can be done manually. So in my molecule/prepare.yml and prepare steps for Github Actions, I used to install this using the latest version. When I specify a version in a requirements.txt file, to the older version 2.31.0, things won't break.

See changes and more changes and build result.

[shaderecker]

Ok cool, nice catch! Thank you for testing this so thoroughly.