CA-394409: plugins: Fix multiple file patterns for the same directory

[bernhardkaindl]

CA-394409: Fix collecting logfiles of auto-cert-kit and tapdisk-logs:

• Fix multiple file patterns for the same directory:

  • Append the directory specifications to a list instead of overwriting the previous directory spec.

    if path in directory_specifications:
        directory_specifications[path].append((cap, pattern, negate))
    else:
        directory_specifications[path] = [(cap, pattern, negate)]

    Then, when we loop over the directory_specifications, loop over the list of tree_output_entries:

    for directory, tree_output_entries in directory_specs.items():
    # Multiple tree_output calls may have appended multiple output entries:
    for directory_items in tree_output_entries:
        # Unpack the stored capability, pattern and negate flag of each row:
        capability, pattern, negate = directory_items

    When ignoring shifting the indendation to loop over the tree_output_entries, the change is small: The output of git log -w -p -n1 xen-bugtool of the commit is:

    
    --- a/xen-bugtool
    +++ b/xen-bugtool
    @@ -602,7 +602,10 @@ def file_output(cap, path_list):

  def tree_output(cap, path, pattern = None, negate = False): if cap in entries:

• directory_specifications[path] = (cap, pattern, negate)

• if path in directory_specifications:

• directory_specifications[path].append((cap, pattern, negate))

• else:

• directory_specifications[path] = [(cap, pattern, negate)]

  def traverse_directory_specifications(directory_specs, requested_capabilities): @@ -611,7 +614,9 @@ def traverse_directory_specifications(directory_specs, requested_capabilities): :param directory_specs: Directories to lookup with cap, pattern, and negate. :param requested_capabilities: The list of requested capabilities. """

• for directory, directory_items in directory_specs.items():

• for directory, tree_output_entries in directory_specs.items():

• Multiple tree_output calls may have appended multiple output entries:

• for directory_items in tree_output_entries:

  Unpack the stored capability, pattern and negate flag of each row:

       capability, pattern, negate = directory_items
       # If the capability is in the requested inventory entries, check it:

• Ensure that this works as expected by extending the tests accordingly.

Change:

Fix this issue to collect logs from tapdisk and auto-cert-kit as expected.

Details:

• At least three plugins collect files in /var/log using different file patterns: Auto_Cert_Kit, tapdisk-logs and xcp-rrdd-plugins.
• After merging CP-41238, only one pattern or directory tree collection was supported: Previous patterns were overwritten by plugins loaded after plugins loaded earlier.
  • Directory tree patterns added by plugins loaded earlier were overwritten by plugins loaded later.
  • As auto-cert-kit is not installed by default, this issue could only be noticed by missing tapdisk logs.

Manual Verification Testing

curl -Lo xen-bugtool.CP-49944 https://github.com/xenserver-next/status-report/raw/CP-49944-multiple-patterns-for-one-directory/xen-bugtool
chmod +x xen-bugtool.CP-49944

DoD:

# mkdir /var/log/blktap
# touch /var/log/blktap/logfile.log
XENRT_BUGTOOL_BASENAME=bugtool ./xen-bugtool.CP-49944 -y --entries=tapdisk-logs,xcp-rrdd-plugins
# tar tf /var/opt/xen/bug-report/bugtool.tar.bz2|grep -e 'log$' -e blktap
bugtool/var/log/blktap/logfile.log
bugtool/var/log/xcp-rrdd-plugins.log

and likewise, for when auto-cert-kit is installed for:

# yum install auto-cert-kit 
# touch bugtool/var/log/auto-cert-kit{-plugin}.log
# XENRT_BUGTOOL_BASENAME=bugtool ./xen-bugtool.CP-49944 -y --entries=Auto_Cert_Kit
# tar tf /var/opt/xen/bug-report/bugtool.tar.bz2|grep -e 'log$'
bugtool/var/log/auto-cert-kit.log
bugtool/var/log/auto-cert-kit-plugin.log

[coveralls]

coverage: 93.137% (+1.3%) from 91.837% when pulling 3a316bde029ee84a72e8297cff323b1e455b69a8 on xenserver-next:CP-49944-multiple-patterns-for-one-directory into 1c8d5443a002ed1eb3c5470b181f5a26916a0d54 on xenserver:master.