search

ansible-collections / ansible.posix

Ansible Collection for Posix
Other
160 stars 153 forks source link

synchronize should handle exotic quotation for rsync's --rsh param correctly #589

Open samweisgamdschie opened 15 hours ago

samweisgamdschie commented 15 hours ago
SUMMARY

According to the man page of rsync (e.g. https://ss64.com/bash/rsync_options.html) --rsh=COMMAND has an exotic way to handle quotation. E.g. quotation of the param list like

--rsh='/usr/bin/ssh -oProxyCommand="ssh -i -W %h:%p -oProxyCommand=\"ssh -W jumphost2:22 ansible@jumphost1\" ansible@jumphost2"'

should instead be escaped like

--rsh='/usr/bin/ssh -oProxyCommand="ssh -i -W %h:%p -oProxyCommand=""ssh -W jumphost2:22 ansible@jumphost1"" ansible@jumphost2"'

To be precise: an escaped single or double-quote should written "" instead of \", with single quotes accordingly.

Because we use such strings successfully as ansible_ssh_args for jumping we also need to have rsync handle that correctly.

ISSUE TYPE
COMPONENT NAME

synchronize

ANSIBLE VERSION
bash-5.1$ ansible --version
ansible [core 2.15.12]
  config file = None
  configured module search path = ['/runner/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python3.9/site-packages/ansible
  ansible collection location = /runner/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/local/bin/ansible
  python version = 3.9.18 (main, Jan 24 2024, 00:00:00) [GCC 11.4.1 20231218 (Red Hat 11.4.1-3)] (/usr/bin/python3)
  jinja version = 3.1.4
  libyaml = True

This is the Ansible distribution in container image ansible/awx-ee:24.5.0

COLLECTION VERSION
bash-5.1$ ansible-galaxy collection list

# /usr/share/ansible/collections/ansible_collections
Collection              Version
----------------------- -------
amazon.aws              8.0.0  
ansible.posix           1.5.4  
ansible.windows         2.3.0  
awx.awx                 24.4.0 
azure.azcollection      2.4.0  
community.vmware        4.4.0  
google.cloud            1.3.0  
kubernetes.core         4.0.0  
kubevirt.core           1.4.0  
openstack.cloud         2.2.0  
ovirt.ovirt             3.2.0  
redhatinsights.insights 1.2.2  
theforeman.foreman      4.0.0  
bash-5.1$
CONFIGURATION
bash-5.1$ ansible-config dump --only-changed
CONFIG_FILE() = None
bash-5.1$
OS / ENVIRONMENT

Container: ansible/awx-ee:24.5.0 Executed within AWX 24.5.0 With awx-operator 2.18.0

STEPS TO REPRODUCE

Use group vars like:

{
  "ansible_private_key_file": "{{ lookup(\"env\",\"JH3_SSH_PRIVATE_KEY\") }}",
  "ansible_ssh_common_args": "-o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ProxyCommand=\"ssh -i {{ lookup(\"env\",\"JH3_SSH_PRIVATE_KEY\") }} -W %h:%p -oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null -oProxyCommand=\\\"ssh -i {{ lookup(\"env\",\"JH1_SSH_PRIVATE_KEY\") }} -W {{ jh3_ip }}:{{ jh3_ssh_port }} -oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null {{ jh1_ssh_user }}@{{ jh1_ip }}\\\" {{ jh3_ssh_user }}@{{ jh3_ip }}\"",
  "ansible_ssh_user": "{{ jh3_ssh_user }}"
}

With a dest_host1 it works perfectly jumping to it via ssh and all tasks used like

  - name: List repos enabled in Spacewalk
    shell: /usr/sbin/spacewalk-channel -l
    register: repolist

but if we use synchronize like

- name: "Synchronize {{ item.app_relative_path }} repo from local Ansible host to {{ username_home }}/{{ app_dest }}/{{ item.app_relative_path }} on remote host"
  synchronize:
    src: "{{ app_src }}"
    dest: "{{ username_home }}/{{ app_dest }}/"
    recursive: true
    delete: true
    checksum: true
    rsync_opts:
      - "--prune-empty-dirs"
      - "--itemize-changes"
      - "--no-owner"
      - "--no-group"
      - "--no-times"
  become: true
  become_user: "{{ username_nix_user }}"
  notify:
    - Ensure correct permissions are set in etc
EXPECTED RESULTS
<<CHANGED>>.d...p..... path_to_be_synced/
<<CHANGED>>.d...p..... path_to_be_synced/local/
<<CHANGED>>.f...p..... path_to_be_synced/local/app.conf
<<CHANGED>>.f...p..... path_to_be_synced/local/inputs.conf
<<CHANGED>>.f...p..... path_to_be_synced/local/props.conf
<<CHANGED>>.f...p..... path_to_be_synced/local/savedsearches.conf

This is what I get, when I replace the \" with "" in the command. After that, the --rsh parameter looks like the second described in the SUMMARY.

ACTUAL RESULTS

with the same config for dest_host1, we get the following AWX output in json format:

{
  "rc": 255,
  "cmd": "/usr/bin/rsync --delay-updates -F --compress --delete-after --checksum --archive --rsh='/usr/bin/ssh -S none -i /runner/env/tmpocuvvus0 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -C -o ControlMaster=auto -o ControlPersist=60s -o ServerAliveInterval=120 -oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null -oProxyCommand=\"ssh -i /runner/env/tmpocuvvus0 -W %h:%p -oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null -oProxyCommand=\\\"ssh -i /runner/env/tmp8x61_zwv -W jumphost3.fqdn:22 -oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null ansible@jumphost1.fqdn\\\" ansible@jumphost3.fqdn\"' --rsync-path='sudo -u username rsync' --prune-empty-dirs --itemize-changes --no-owner --no-group --no-times --out-format='<<CHANGED>>%i %n%L' /opt/source/path ansible@dest_host1.fqdn:/opt/dest/path/",
  "msg": "usage: ssh [-46AaCfGgKkMNnqsTtVvXxYy] [-B bind_interface]\n           [-b bind_address] [-c cipher_spec] [-D [bind_address:]port]\n           [-E log_file] [-e escape_char] [-F configfile] [-I pkcs11]\n           [-i identity_file] [-J [user@]host[:port]] [-L address]\n           [-l login_name] [-m mac_spec] [-O ctl_cmd] [-o option] [-p port]\n           [-Q query_option] [-R address] [-S ctl_path] [-W host:port]\n           [-w local_tun[:remote_tun]] destination [command]\nkex_exchange_identification: Connection closed by remote host\r\nConnection closed by UNKNOWN port 65535\r\nrsync: connection unexpectedly closed (0 bytes received so far) [sender]\nrsync error: unexplained error (code 255) at io.c(228) [sender=3.2.3]\n",
  "invocation": {
    "module_args": {
      "src": "/opt/source/path",
      "dest": "ansible@dest_host1.fqdn:/opt/dest/path/",
      "recursive": true,
      "delete": true,
      "checksum": true,
      "rsync_opts": [
        "--prune-empty-dirs",
        "--itemize-changes",
        "--no-owner",
        "--no-group",
        "--no-times"
      ],
      "ssh_args": "-C -o ControlMaster=auto -o ControlPersist=60s -o ServerAliveInterval=120 -oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null -oProxyCommand=\"ssh -i /runner/env/tmpocuvvus0 -W %h:%p -oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null -oProxyCommand=\\\"ssh -i /runner/env/tmp8x61_zwv -W jumphost3.fqdn:22 -oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null ansible@jumphost1.fqdn\\\" ansible@jumphost3.fqdn\"",
      "_local_rsync_path": "rsync",
      "private_key": "/runner/env/tmpocuvvus0",
      "_local_rsync_password": null,
      "rsync_path": "sudo -u username rsync",
      "_substitute_controller": false,
      "archive": true,
      "compress": true,
      "existing_only": false,
      "dirs": false,
      "copy_links": false,
      "set_remote_user": true,
      "rsync_timeout": 0,
      "ssh_connection_multiplexing": false,
      "partial": false,
      "verify_host": false,
      "delay_updates": true,
      "mode": "push",
      "dest_port": null,
      "links": null,
      "perms": null,
      "times": null,
      "owner": null,
      "group": null,
      "link_dest": null
    }
  },
  "_ansible_no_log": false,
  "changed": false
}

The very same output I get, when I reproduce it in our dev environment. On commandline it says:

usage: ssh [-46AaCfGgKkMNnqsTtVvXxYy] [-B bind_interface]
           [-b bind_address] [-c cipher_spec] [-D [bind_address:]port]
           [-E log_file] [-e escape_char] [-F configfile] [-I pkcs11]
           [-i identity_file] [-J [user@]host[:port]] [-L address]
           [-l login_name] [-m mac_spec] [-O ctl_cmd] [-o option] [-p port]
           [-Q query_option] [-R address] [-S ctl_path] [-W host:port]
           [-w local_tun[:remote_tun]] destination [command]
kex_exchange_identification: Connection closed by remote host
Connection closed by UNKNOWN port 65535
rsync: connection unexpectedly closed (0 bytes received so far) [sender]
rsync error: unexplained error (code 255) at io.c(228) [sender=3.2.3]
samweisgamdschie commented 15 hours ago

One more thing: sad but true, our (actually) longest ansible_ssh_common_args setup looks like this:

{
  "ansible_private_key_file": "{{ lookup(\"env\",\"JH6_SSH_PRIVATE_KEY\") }}",
  "ansible_ssh_common_args": "-oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null -oProxyCommand=\"ssh -i {{ lookup(\"env\",\"JH6_SSH_PRIVATE_KEY\") }} -W %h:%p -oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null -oProxyCommand=\\\"ssh -i {{ lookup(\"env\",\"JH4_SSH_PRIVATE_KEY\") }} -W {{ jh6_ip }}:{{ jh6_ssh_port }} -oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null -oProxyCommand=\\\\\\\"ssh -i {{ lookup(\"env\",\"JH1_SSH_PRIVATE_KEY\") }} -W {{ jh4_ip }}:{{ jh4_ssh_port }} -oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null {{ jh1_ssh_user }}@{{ jh1_ip }}\\\\\\\" {{ jh4_ssh_user }}@{{ jh4_ip }}\\\" {{ jh6_ssh_user }}@{{ jh6_ip }}\"",
  "ansible_ssh_user": "{{ jh6_ssh_user }}"
}

Which means, we need to handle at least triple quotes: \\\\\\\" in JSON, means \\\" in YAML, means """ in rsync's --rsh command parameter.

Welcome again in the escape hell, D'OH!