ansible-collections / community.general

Ansible Community General Collection
https://galaxy.ansible.com/ui/repo/published/community/general/
GNU General Public License v3.0
814 stars 1.49k forks source link

Unable to to specify a specific package version in Ansible for apt_rpm module #8217

Open nikita197 opened 5 months ago

nikita197 commented 5 months ago

Summary

When I try to install specific package version via Ansible, I got error:

TASK [Install my package] ***************************************************************************************************************************************************************************************************
fatal: [test]: FAILED! => changed=false 
  msg: '''apt-get -y install python3-module-yaml=6.0.1'' failed: '

my playbook example:

---
- hosts: all
  become: true

  tasks:
  - name: Install my package
    package:
     name: python3-module-yaml=6.0.1
     state: present
     update_cache: true

Target host is AltLinux server v10.2

As I know, in https://github.com/ansible-collections/community.general/issues/7414 in file apt_rpm.py added function that upgrade package if newer version exists. But it is not good for me, because I'm getting inconsistent state of packages (For example, I don't want to upgrade the k8s cluster if I just run playbook).

Issue Type

Feature idea

Component Name

apt_rpm

Ansible Version

$ ansible --version
ansible [core 2.15.5]
  config file = /ansible/host_group/altlinux_kubernetes/ansible.cfg
  configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3.11/site-packages/ansible
  ansible collection location = /ansible/host_group/altlinux_kubernetes/collections
  executable location = /usr/bin/ansible
  python version = 3.11.6 (main, Oct  4 2023, 06:22:18) [GCC 12.2.1 20220924] (/usr/bin/python3)
  jinja version = 3.1.2
  libyaml = True

Community.general Version

$ ansible-galaxy collection list community.general
# /usr/lib/python3.11/site-packages/ansible_collections
Collection        Version
----------------- -------
community.general 7.5.0  

Configuration

$ ansible-config dump --only-changed
CACHE_PLUGIN(/ansible/host_group/altlinux_kubernetes/ansible.cfg) = jsonfile
CACHE_PLUGIN_CONNECTION(/ansible/host_group/altlinux_kubernetes/ansible.cfg) = /tmp
COLLECTIONS_PATHS(/ansible/host_group/altlinux_kubernetes/ansible.cfg) = ['/ansible/host_group/altlinux_kubernetes/collections']
CONFIG_FILE() = /ansible/host_group/altlinux_kubernetes/ansible.cfg
DEFAULT_GATHERING(/ansible/host_group/altlinux_kubernetes/ansible.cfg) = smart
DEFAULT_REMOTE_USER(/ansible/host_group/altlinux_kubernetes/ansible.cfg) = sysop
DEFAULT_ROLES_PATH(/ansible/host_group/altlinux_kubernetes/ansible.cfg) = ['/ansible/host_group/altlinux_kubernetes/roles']
DEFAULT_STDOUT_CALLBACK(/ansible/host_group/altlinux_kubernetes/ansible.cfg) = yaml
HOST_KEY_CHECKING(/ansible/host_group/altlinux_kubernetes/ansible.cfg) = False
INVENTORY_ENABLED(/ansible/host_group/altlinux_kubernetes/ansible.cfg) = ['ini']
INVENTORY_IGNORE_EXTS(/ansible/host_group/altlinux_kubernetes/ansible.cfg) = ['~', '.orig', '.bak', '.ini', '.cfg', '.retry', '.pyc', '.pyo', '.creds']
INVENTORY_IGNORE_PATTERNS(/ansible/host_group/altlinux_kubernetes/ansible.cfg) = ['artifacts', 'credentials']

OS / Environment

Alpine Linux v3.18

Steps to Reproduce

just run

ansible-playbook playbook.yml

Expected Results

Specific package version installed on target AltLinux host. Moreover, I can install a package of a specific version on the target host via apt-get:

[root@host-107 budnov]# apt-get install python3-module-yaml=6.0.1
Чтение списков пакетов... Завершено
Построение дерева зависимостей... Завершено
Selected version python3-module-yaml#6.0.1-alt1.1:p10+340092.554.16.1@1709018089 for python3-module-yaml=6.0.1
Следующие дополнительные пакеты будут установлены:
  libyaml2 python3-module-yaml
Следующие НОВЫЕ пакеты будут установлены:
  libyaml2 python3-module-yaml
0 будет обновлено, 2 новых установлено, 0 пакетов будет удалено и 31 не будет обновлено.
Необходимо получить 217kB архивов.
После распаковки потребуется дополнительно 906kB дискового пространства.
Продолжить? [Y/n] y
Получено: 1 http://ftp.altlinux.org p10/branch/x86_64/classic libyaml2 0.2.5-alt1:sisyphus+278391.100.1.1@1626094928 [49,9kB]
Получено: 2 http://ftp.altlinux.org p10/branch/x86_64/classic python3-module-yaml 6.0.1-alt1.1:p10+340092.554.16.1@1709018089 [167kB]
Получено 217kB за 0s (610kB/s).
Совершаем изменения...
Подготовка...                                                                             #################################################################################################### [100%]
Обновление / установка...
1: libyaml2-0.2.5-alt1                                                                    #################################################################################################### [ 50%]
2: python3-module-yaml-6.0.1-alt1.1                                                       #################################################################################################### [100%]
Завершено.

Actual Results

docker-desktop:/ansible/host_group/altlinux_kubernetes# ansible-playbook playbook.yml -i inventory.ini --vault-pass-file vault -vvvv
ansible-playbook [core 2.15.5]
  config file = /ansible/host_group/altlinux_kubernetes/ansible.cfg
  configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3.11/site-packages/ansible
  ansible collection location = /ansible/host_group/altlinux_kubernetes/collections
  executable location = /usr/bin/ansible-playbook
  python version = 3.11.6 (main, Oct  4 2023, 06:22:18) [GCC 12.2.1 20220924] (/usr/bin/python3)
  jinja version = 3.1.2
  libyaml = True
Using /ansible/host_group/altlinux_kubernetes/ansible.cfg as config file
setting up inventory plugins
Loading collection ansible.builtin from 
Parsed /ansible/host_group/altlinux_kubernetes/inventory.ini inventory source with ini plugin
redirecting (type: callback) ansible.builtin.yaml to community.general.yaml
Loading collection community.general from /usr/lib/python3.11/site-packages/ansible_collections/community/general
redirecting (type: callback) ansible.builtin.yaml to community.general.yaml
Loading callback plugin community.general.yaml of type stdout, v2.0 from /usr/lib/python3.11/site-packages/ansible_collections/community/general/plugins/callback/yaml.py
Skipping callback 'default', as we already have a stdout callback.
Skipping callback 'minimal', as we already have a stdout callback.
Skipping callback 'oneline', as we already have a stdout callback.

PLAYBOOK: playbook.yml ******************************************************************************************************************************************************************************************************
Positional arguments: playbook.yml
verbosity: 4
remote_user: sysop
connection: smart
timeout: 10
become_method: sudo
tags: ('all',)
inventory: ('/ansible/host_group/altlinux_kubernetes/inventory.ini',)
vault_password_files: ('/ansible/host_group/altlinux_kubernetes/vault',)
forks: 5
1 plays in playbook.yml

PLAY [all] ******************************************************************************************************************************************************************************************************************
Trying secret FileVaultSecret(filename='/ansible/host_group/altlinux_kubernetes/vault') for vault_id=default

TASK [Install my package] ***************************************************************************************************************************************************************************************************
task path: /ansible/host_group/altlinux_kubernetes/playbook.yml:6
redirecting (type: modules) ansible.builtin.apt_rpm to community.general.apt_rpm
redirecting (type: modules) ansible.builtin.apt_rpm to community.general.apt_rpm
Running apt_rpm
redirecting (type: modules) ansible.builtin.apt_rpm to community.general.apt_rpm
Using module file /usr/lib/python3.11/site-packages/ansible_collections/community/general/plugins/modules/apt_rpm.py
Pipelining is enabled.
<192.168.0.107> ESTABLISH SSH CONNECTION FOR USER: budnov
<192.168.0.107> SSH: EXEC sshpass -d12 ssh -vvv -o ControlMaster=auto -o ControlPersist=1800s -o StrictHostKeyChecking=no -o 'User="budnov"' -o ConnectTimeout=10 -o 'ControlPath="/root/.ansible/cp/f8a796ea78"' 192.168.0.107 '/bin/sh -c '"'"'sudo -H -S -p "[sudo via ansible, key=lawkbdlayeizmthnlwvbciscebcyfgnw] password:" -u root /bin/sh -c '"'"'"'"'"'"'"'"'echo BECOME-SUCCESS-lawkbdlayeizmthnlwvbciscebcyfgnw ; /usr/bin/python3.9'"'"'"'"'"'"'"'"' && sleep 0'"'"''
Escalation succeeded
<192.168.0.107> (1, b'\n{"failed": true, "msg": "\'apt-get -y install python3-module-yaml=6.0.1\' failed: ", "invocation": {"module_args": {"name": "python3-module-yaml=6.0.1", "state": "present", "update_cache": true, "package": ["python3-module-yaml=6.0.1"], "clean": false, "dist_upgrade": false, "update_kernel": false}}}\n', b"OpenSSH_9.3p2, OpenSSL 3.1.4 24 Oct 2023\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug2: resolve_canonicalize: hostname 192.168.0.107 is address\r\ndebug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> '/root/.ssh/known_hosts'\r\ndebug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' -> '/root/.ssh/known_hosts2'\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 32\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 1\r\n")
<192.168.0.107> Failed to connect to the host via ssh: OpenSSH_9.3p2, OpenSSL 3.1.4 24 Oct 2023
debug1: Reading configuration data /etc/ssh/ssh_config
debug2: resolve_canonicalize: hostname 192.168.0.107 is address
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> '/root/.ssh/known_hosts'
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' -> '/root/.ssh/known_hosts2'
debug1: auto-mux: Trying existing master
debug2: fd 3 setting O_NONBLOCK
debug2: mux_client_hello_exchange: master version 4
debug3: mux_client_forwards: request forwardings: 0 local, 0 remote
debug3: mux_client_request_session: entering
debug3: mux_client_request_alive: entering
debug3: mux_client_request_alive: done pid = 32
debug3: mux_client_request_session: session request sent
debug1: mux_client_request_session: master session id: 2
debug3: mux_client_read_packet: read header failed: Broken pipe
debug2: Received exit status from master 1
fatal: [test]: FAILED! => changed=false 
  invocation:
    module_args:
      clean: false
      dist_upgrade: false
      name: python3-module-yaml=6.0.1
      package:
      - python3-module-yaml=6.0.1
      state: present
      update_cache: true
      update_kernel: false
  msg: '''apt-get -y install python3-module-yaml=6.0.1'' failed: '

PLAY RECAP ******************************************************************************************************************************************************************************************************************
test                       : ok=0    changed=0    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0   

Code of Conduct

ansibullbot commented 5 months ago

Files identified in the description:

If these files are incorrect, please update the component name section of the description or use the !component bot command.

click here for bot help

ansibullbot commented 5 months ago

cc @evgkrsk click here for bot help

felixfontein commented 5 months ago

The module documents nowhere that you can specify versions, so this is not a bug, but a feature request.

nikita197 commented 5 months ago

The module documents nowhere that you can specify versions, so this is not a bug, but a feature request.

It seems like it is. Сan anyone add this functionality? I think I'm not the only one who has this need.

shenxianpeng commented 5 months ago

I'm confused your example used package, from the package doc it supports specific versions. so it seems package bug if it does not work.

felixfontein commented 5 months ago

The module documents nowhere that you can specify versions, so this is not a bug, but a feature request.

It seems like it is. Сan anyone add this functionality? I think I'm not the only one who has this need.

Yes, this is a community collection, so it's owned by the community :)

I'm confused your example used package, from the package doc it supports specific versions. so it seems package bug if it does not work.

The package module isn't a module, but an action which passes its parameters through to the selected package manager module. Which features are actually supported depends on the specific module used. The important part in the documentation is Syntax varies with package manager. (It should better say "package manager module" though, since it in particular depends on that one. The module might support more or less things than the actual package manager.) The apt_rpm module doesn't support features that many other package manager modules support, like specifying versions.

nikita197 commented 5 months ago

I'm confused your example used package, from the package doc it supports specific versions. so it seems package bug if it does not work.

I tried also apt_rpm module, but the same problem:

---
- hosts: all
  become: true

  tasks:
  - name: Install my package
    become: yes
    apt_rpm:
      name: bird=1.6.8
      state: present
      update_cache: yes
...

output:

docker-desktop:/ansible/host_group/postgres# ansible-playbook playbook.yml -i inventory.ini --vault-pass-file vault_pass 
[DEPRECATION WARNING]: ansible.posix.skippy has been deprecated. See the plugin documentation for more details. This feature will be removed from ansible.posix in a release after 2022-06-01. Deprecation 
warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.

PLAY [all] *****************************************************************************************************************************************************************************************************

TASK [Install squid packages] **********************************************************************************************************************************************************************************
fatal: [postgres-pro.lab.adc.spb]: FAILED! => {"changed": false, "msg": "'apt-get -y install bird=1.6.8' failed: "}

PLAY RECAP *****************************************************************************************************************************************************************************************************
postgres-pro.lab.adc.spb   : ok=0    changed=0    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0   

docker-desktop:/ansible/host_group/postgres# 

On official AltLinux FTP package server (http://ftp.altlinux.org/pub/distributions/ALTLinux/p10/branch/x86_64/RPMS.classic/) required package present: image

nikita197 commented 5 months ago

The module documents nowhere that you can specify versions, so this is not a bug, but a feature request.

It seems like it is. Сan anyone add this functionality? I think I'm not the only one who has this need.

Yes, this is a community collection, so it's owned by the community :)

I'm confused your example used package, from the package doc it supports specific versions. so it seems package bug if it does not work.

The package module isn't a module, but an action which passes its parameters through to the selected package manager module. Which features are actually supported depends on the specific module used. The important part in the documentation is Syntax varies with package manager. (It should better say "package manager module" though, since it in particular depends on that one. The module might support more or less things than the actual package manager.) The apt_rpm module doesn't support features that many other package manager modules support, like specifying versions.

Yes, you are right. Also I noticed that "present" state for apt_rpm module will update package if a newer version exists, but this is very strange behavior. On the contrary, I want to fix the package version and always be sure that I know the final state of the operating system

Viktor3434 commented 5 months ago

I have the same problem

Viktor3434 commented 5 months ago

Previously, this could be fixed as follows

def fix_package_name(name):
    package_name = name.replace('=', '-')
    return package_name

def query_package(module, name):
    # rpm -q returns 0 if the package is installed,
    # 1 if it is not installed
    rc, out, err = module.run_command("%s -q %s" % (RPM_PATH, name))
    rc, out, err = module.run_command("%s -q %s" % (RPM_PATH, fix_package_name(name)))
    if rc == 0:
        return True
    else:
def query_package_provides(module, name):
    # rpm -q returns 0 if the package is installed,
    # 1 if it is not installed
    rc, out, err = module.run_command("%s -q --provides %s" % (RPM_PATH, name))
    rc, out, err = module.run_command("%s -q --provides %s" % (RPM_PATH, fix_package_name(name)))
    return rc == 0
Viktor3434 commented 5 months ago

now we have a rather strange default behavior that is not well suited for a production environment

https://github.com/ansible-collections/community.general/blob/main/plugins/modules/apt_rpm.py#L99 https://github.com/ansible-collections/community.general/blob/main/plugins/modules/apt_rpm.py#L170

when we specify "present" we get "latest"

felixfontein commented 5 months ago

I agree that this behavior is odd, but the module has been claiming this behavior from the beginning.

How about:

  1. Add two new states latest and installed, where the former upgrades and the latter does not.
  2. Add a deprecation warning that state=present will change its meaning in the future (basically change from latest to installed).
  3. Then in a couple of versions, change the behavior for state=present.

Then users can soon start specifying the exact behavior they want, but they can also chose to stick to the old behavior. And they can get rid of the deprecation warning by adjusting state to one of the two new values.

And eventually installed will be an alias of present, as its already the case for multiple other package managing modules.

(This doesn't fix the problem that you might want to specify a package version, but at least it stops the strange behavior.)

nikita197 commented 5 months ago

I agree that this behavior is odd, but the module has been claiming this behavior from the beginning.

How about:

  1. Add two new states latest and installed, where the former upgrades and the latter does not.
  2. Add a deprecation warning that state=present will change its meaning in the future (basically change from latest to installed).
  3. Then in a couple of versions, change the behavior for state=present.

Then users can soon start specifying the exact behavior they want, but they can also chose to stick to the old behavior. And they can get rid of the deprecation warning by adjusting state to one of the two new values.

And eventually installed will be an alias of present, as its already the case for multiple other package managing modules.

(This doesn't fix the problem that you might want to specify a package version, but at least it stops the strange behavior.)

The problem with automatically updating packages to the latest version when it is present state is minor. Now it is extremely important for me to find a way to fix package versions

felixfontein commented 5 months ago

I created #8247 to add new states latest and present_not_latest (installed was already there, and equivalent to present).

nikita197 commented 5 months ago

@felixfontein thank you very much for your contribution! Also please create a new PR to add a package version fixation

felixfontein commented 4 months ago

@nikita197 I likely won't implement that, but that shouldn't stop anyone else from trying it.

felixfontein commented 4 months ago

(If someone wants to work on this, please note that #8264 does some general refactoring of the run_command calls in this module.)

felixfontein commented 4 months ago

8285 deprecates the current behavior of state=installed and state=present so it can change to what is less surprising in community.general 11.0.0.