ansible-collections / ansible.posix

Ansible Collection for Posix
Other
158 stars 153 forks source link

Less verbose synchronize module output #341

Open bluikko opened 2 years ago

bluikko commented 2 years ago
SUMMARY

Currently the synchronize module output is very verbose.
On level -vv the same rsync output, including line for each transferred item, is printed 3 times:

ISSUE TYPE
COMPONENT NAME

synchronize

ADDITIONAL INFORMATION

If several thousand -- or even tens of thousands, by no means an unreasonable possibility -- files are being synchronized it seems excessive, even if I had requested verbosity with -vv.
A couple of different things could be possible:

Number of files: 4,497 (reg: 4,437, dir: 60)
Number of created files: 4,496 (reg: 4,437, dir: 59)
Number of deleted files: 0
Number of regular files transferred: 4,437
Total file size: 37,086,484,092 bytes
Total transferred file size: 37,086,484,092 bytes
Literal data: 37,086,484,092 bytes
Matched data: 0 bytes
File list size: 65,534
File list generation time: 0.074 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 37,095,880,814
Total bytes received: 84,830

sent 37,095,880,814 bytes  received 84,830 bytes  68,003,603.38 bytes/sec
total size is 37,086,484,092  speedup is 1.00

It does not seem useful to print the same information 3 times when the content could be thousands or tens of thousands of lines.
Turning the itemized output off with a suitable rsync_opt switch makes the task not detect whether it was changed. So obviously some output is needed for making the decision whether the task was changed, failed, etc. Possibly the necessary information could be parsed out of rsync summary data block instead of the long itemized output?

Example output from synchronize:

TASK [Copy files] *************************************************************************************************************************************************************
task path: /ansible/testcopy.yml:50
.d..tp..... ./
>f+++++++++ testfile0000
[...]
changed: [host.example.com -> host.example.com] => changed=true
  cmd: /usr/bin/rsync --archive --out-format='<<CHANGED>>%i %n%L' /source/ /destination
  msg: |-
    .d..tp..... ./
    >f+++++++++ testfile0000
[...]
  rc: 0
  stdout_lines:
  - .d..tp..... ./
  - '>f+++++++++ testfile0000'
[...]

Each of the three [...] above is to replace the full list of thousands of files/directories.

vbotka commented 2 months ago

The main problem is that you can't override the option

--out-format='<<CHANGED>>%i %n%L'

You can control the output without the option --out-format and with the option --info. For example, given the simple tree below

shell> tree /tmp/ansible/
/tmp/ansible/
├── dest
└── src
    ├── a
    ├── b
    └── c

You can ask for statistics and no files

shell> rsync -a --info=name0,stats2 /tmp/ansible/src/ /tmp/ansible/dest

Number of files: 4 (reg: 3, dir: 1)
Number of created files: 3 (reg: 3)
Number of deleted files: 0
Number of regular files transferred: 3
Total file size: 0 bytes
Total transferred file size: 0 bytes
Literal data: 0 bytes
Matched data: 0 bytes
File list size: 0
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 227
Total bytes received: 76

sent 227 bytes  received 76 bytes  606.00 bytes/sec
total size is 0  speedup is 0.00

The below command shows nothing at all

shell> rsync -a --info=name0 /tmp/ansible/src/ /tmp/ansible/dest

Unfortunately, this doesn't work with the module synchronize. You can set rsync_opts

    - synchronize:
        src: /tmp/ansible/src/
        dest: /tmp/ansible/dest
        rsync_opts:
          - --info=name0,stats2
          - --out-format=""
      register: changed_files

, but you can't override --out-format='<>%i %n%L'

  changed_files:
    changed: true
    cmd: /usr/bin/rsync --delay-updates -F --compress --archive --info=name0,stats2 --out-format="" --out-format='<<CHANGED>>%i %n%L' /tmp/ansible/src/ /tmp/ansible/dest
    failed: false
    msg: |-
      .d..t...... ./
      >f+++++++++ a
      >f+++++++++ b
      >f+++++++++ c

      Number of files: 4 (reg: 3, dir: 1)
      Number of created files: 3 (reg: 3)
      Number of deleted files: 0
      Number of regular files transferred: 3
      Total file size: 0 bytes
      Total transferred file size: 0 bytes
      Literal data: 0 bytes
      Matched data: 0 bytes
      File list size: 0
      File list generation time: 0.001 seconds
      File list transfer time: 0.000 seconds
      Total bytes sent: 232
      Total bytes received: 80

      sent 232 bytes  received 80 bytes  624.00 bytes/sec
      total size is 0  speedup is 0.00
    rc: 0
    stdout_lines:
    - .d..t...... ./
    - '>f+++++++++ a'
    - '>f+++++++++ b'
    - '>f+++++++++ c'
    - 'Number of files: 4 (reg: 3, dir: 1)'
    - 'Number of created files: 3 (reg: 3)'
    - 'Number of deleted files: 0'
    - 'Number of regular files transferred: 3'
    - 'Total file size: 0 bytes'
    - 'Total transferred file size: 0 bytes'
    - 'Literal data: 0 bytes'
    - 'Matched data: 0 bytes'
    - 'File list size: 0'
    - 'File list generation time: 0.001 seconds'
    - 'File list transfer time: 0.000 seconds'
    - 'Total bytes sent: 232'
    - 'Total bytes received: 80'
    - sent 232 bytes  received 80 bytes  624.00 bytes/sec
    - total size is 0  speedup is 0.00

This should be considered a bug. Fixing this bug would solve this feature request.

No triage is needed.