added ansible reporting

[starnold-redhat]

Added hcs-collector code to take advantage of the crhc ansible usage download feature, and report on number of machines managed by ansible

[waldirio]

Hello @starnold-redhat

Thank you for this PR.

I'll wait for the change on this one in order to test/merge this request.

Thank you! Waldirio

[waldirio]

Hello @starnold-redhat

Let me share what I got. I have already some previous data on my local filesystem and when syncing the latest version and processing, I got the issue below

$ ./hcs-collector.py process
process data with a tag of none

## RHEL On-Demand

Max Concurrent RHEL On-Demand, referrent to ..: 2022-06
On-Demand, Physical Node .....................: 0
On-Demand, Virtual Node ......................: 0
Virtual Data Center, Virtual Node ............: 0
Unknown ......................................: 0

## RHEL Add-ons

Traceback (most recent call last):
  File "./hcs-collector.py", line 10, in <module>
    parse.main()
  File "/home/wpinheir/.venv/hcs-collector/lib64/python3.7/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/home/wpinheir/.venv/hcs-collector/lib64/python3.7/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/home/wpinheir/.venv/hcs-collector/lib64/python3.7/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/wpinheir/.venv/hcs-collector/lib64/python3.7/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/wpinheir/.venv/hcs-collector/lib64/python3.7/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/home/wpinheir/deleteme/hcs-collector/parse/parse.py", line 65, in process
    execution.process_data(tag)
  File "/home/wpinheir/deleteme/hcs-collector/execution/execution.py", line 117, in process_data
    generate_report(path_to_csv_dir, csv_files_list, path_to_json_dir, json_files_list, tag)
  File "/home/wpinheir/deleteme/hcs-collector/execution/execution.py", line 131, in generate_report
    process_rhel_addons.ondemand_rhel_related_products(path_to_csv_dir, csv_files_list, tag)
  File "/home/wpinheir/deleteme/hcs-collector/execution/process_rhel_addons.py", line 46, in ondemand_rhel_related_products
    if stage_rhel_ha_physical > rhel_ha_physical:
UnboundLocalError: local variable 'stage_rhel_ha_physical' referenced before assignment

As we can see, the app is trying to process the info from 2022-06.

When moving all the old stuff and keeping only the latest package, we can see that the report works.

Here we can see creating the new dir and also collecting the new stuff

$ ./hcs-collector.py collect
collect data
initial directory setup
dir /home/wpinheir/hcs_data/2023/08 it's not there yet, creating ...
dir /home/wpinheir/hcs_data/2023/08/JSON it's not there yet, creating ...
dir /home/wpinheir/hcs_data/2023/08/CSV it's not there yet, creating ...
/home/wpinheir/demo/crhc
Cleaning the current cache files
removing the file /tmp/inventory.json
removing the file /tmp/swatch.json
removing the file /tmp/match_inv_sw.csv
removing the file /tmp/patch.json
removing the file /tmp/vulnerability.json
removing the file /tmp/advisor.json
Downloading and creating the new match info
dumping the inventory information to '/tmp/inventory.json', this can take some time to finish
dumping the subscription information to '/tmp/swatch.json'
dumping the patch information to '/tmp/patch.json'
dumping the vulnerability information to '/tmp/vulnerability.json'
dumping the advisor information to '/tmp/advisor.json'
File /tmp/crhc_data.zip created.
File /tmp/inventory.json already in place, using it.
File /tmp/swatch.json already in place, using it.
File /tmp/match_inv_sw.csv created
File /tmp/issue_summary.log created
appending inventory tags to json
appending inventory tags to csv
Downloading the ansible info

Note. I can see above the tags task, which is adding the info that you are looking for. Below, we can see the next run.

$ ./hcs-collector.py process
process data with a tag of none

## RHEL On-Demand

Max Concurrent RHEL On-Demand, referrent to ..: 2023-08
On-Demand, Physical Node .....................: 591
On-Demand, Virtual Node ......................: 3930
Virtual Data Center, Virtual Node ............: 136
Unknown ......................................: 0

## RHEL Add-ons

On-Demand, High Availability, Physical Node...: 13
On-Demand, High Availability, Virtual Node....: 205
On-Demand, Directory Server Node .............: 56

## Virtualization

On-Demand, Virtualization Sockets ............: 513

## Middleware

On-Demand, JBoss EAP Cores ...................: 126
On-Demand, JWS Cores .........................: 0

## RHEL Virtual Data Center

Virtual Data Center, Hypervisor ..............: 54
Virtual Data Center, Hypervisor Sockets ......: 105

## Ansible Managed Hosts

On-Demand, Ansible Managed Hosts .............: 1382

@starnold-redhat could you please update the code in order that, if the customer has already the data, the script will not fail when parsing the old stuff, and will work with the new ones? This will keep the compatibility and will not break/stop the current customer process.

Thank you! Waldirio

[starnold-redhat]

Good catch - thanks. I'll check it out.

[starnold-redhat]

Hi @waldirio can you try again. I've fixed a few issues with variables not being set when there are no files. And also added in a bit more defensive programming around getting the files from directories. I can't reproduce the error now - but if you could check again on your side - that would be awesome.

[waldirio]

Hello @starnold-redhat

Perfect, It looks good to me.

Thank you! Waldirio