hunter integration and refactor

shashank-boyapally commented 5 months ago

Type of change

[x] Refactor
[x] New feature
[ ] Bug fix
[ ] Optimization
[ ] Documentation Update

Description

Hunter is being integrated so that change points can be detected. Refactored code to increase readability. Below is the sample output

(venv) sboyapal orion (cli) >> orion --hunter-analyze
2024-02-01 15:51:47,374 - Orion - INFO - The test aws-small-scale-cluster-density-v2 has started
2024-02-01 15:51:47,375 - Matcher - INFO - Executing query against index=perf_scale_ci
2024-02-01 15:51:47,745 - Matcher - INFO - Executing query against index=ripsaw-kube-burner*
2024-02-01 15:51:47,925 - Orion - INFO - Collecting podReadyLatency
2024-02-01 15:51:47,925 - Matcher - INFO - Executing query against index=ripsaw-kube-burner
2024-02-01 15:51:48,687 - Orion - INFO - Collecting apiserverCPU
2024-02-01 15:51:48,688 - Matcher - INFO - Executing query against index=ripsaw-kube-burner
2024-02-01 15:51:50,838 - Orion - INFO - Collecting ovnCPU
2024-02-01 15:51:50,838 - Matcher - INFO - Executing query against index=ripsaw-kube-burner
2024-02-01 15:51:52,510 - Orion - INFO - Collecting etcdCPU
2024-02-01 15:51:52,510 - Matcher - INFO - Executing query against index=ripsaw-kube-burner
2024-02-01 15:51:55,177 - Orion - INFO - Collecting etcdDisck
2024-02-01 15:51:55,177 - Matcher - INFO - Executing query against index=ripsaw-kube-burner
time                       uuid                                    P99    apiserverCPU_cpu_avg    ovnCPU_cpu_avg    etcdCPU_cpu_avg    etcdDisck_duration_avg
-------------------------  ------------------------------------  -----  ----------------------  ----------------  -----------------  ------------------------
2023-10-16 19:25:28 +0000  e624b10f-148b-4871-acb5-e8a71cf1b605  13000                 10.175            2.42737            7.63694                 0.0115776
2023-10-18 15:11:43 +0000  325dd073-b680-4e44-951f-26506f401be0  13000                 10.4876           2.49018            8.12781                 0.0107434
2023-10-24 10:52:22 +0000  8cfc4fcf-46c7-4b9f-a464-cd100bfb84bf  13000                 10.3932           2.44498            7.60968                 0.0116801
                                                                                                ················                                               
                                                                                                          +13.8%                                               
                                                                                                ················                                               
2023-10-31 10:30:02 +0000  3c9e2789-59d4-4d6f-9f07-5c400cf25c93  13000                 11.0129           2.73347            8.83458                 0.0112971
2023-11-03 04:08:14 +0000  649228fb-9ff4-458a-8a8b-b9ecb521ca1b  13000                 10.8605           2.82005            8.34146                 0.0101805
2023-11-07 10:26:08 +0000  d8e0c965-4548-48d4-8a7a-c065b93ee766  15000                 10.5964           2.6356             7.48679                 0.0124896
2023-11-15 12:36:38 +0000  a990f7f9-1faf-41f7-b9fb-3b83f91b6e52  13000                 10.8222           2.81836            8.56302                 0.0112316
2023-11-22 12:53:45 +0000  e6c219dc-a9aa-4cf9-854a-fbd846c8ad22  13000                 10.8555           2.84967            8.20129                 0.0111251
2023-11-29 12:32:04 +0000  adc95590-791d-423b-9d09-8190178edbf6  13000                 11.4115           2.8615             8.67125                 0.0127318
                                                                                                                                     ························  
                                                                                                                                                       +39.3%  
                                                                                                                                     ························  
2023-12-06 12:34:57 +0000  5b06c830-be55-4c84-b4da-92720b55c304  13000                 11.05             2.85996            8.69638                 0.0134358
2023-12-20 12:37:17 +0000  c5f95336-27d4-4615-94c1-d618e975a36b  13000                 10.9253           2.77954            8.20148                 0.0137848
2023-12-27 12:36:59 +0000  bd5e6cad-6bc2-472d-948a-b1fbb350e80a  13000                 11.1967           2.77645            8.63235                 0.0154012
2024-01-03 12:33:09 +0000  b91c7156-0b86-4c79-81eb-60dfb7288070  13000                 10.7863           2.80557            8.54677                 0.0138396
                                                                                                ················  ·················                            
                                                                                                          +21.7%             +13.8%                            
                                                                                                ················  ·················                            
2024-01-10 12:38:16 +0000  12a1750e-a7eb-48ed-bc00-35078d2bbad8  13000                 10.9379           3.10667            8.86329                 0.0154639
2024-01-10 22:59:25 +0000  c6b4aaa3-cd2c-4f04-af20-3d10c52449a2  13000                 11.4162           3.1338             9.29143                 0.0128269
                                                                        ······················                                                                 
                                                                                        +22.7%                                                                 
                                                                        ······················                                                                 
2024-01-11 17:30:26 +0000  c2963cec-e800-423f-92b1-3e6f692a55cb  13000                 16.1465           3.80976           10.1149                  0.0149901
2024-01-15 13:01:36 +0000  4f73a079-0088-4027-af0d-7c1b5b429fdd  13000                 15.0479           3.93557           10.0966                  0.0193588
2024-01-15 20:30:57 +0000  90189fbf-7181-4129-8ca5-3cc8d656b595  13000                 10.8181           3.05989            8.48998                 0.015606
2024-01-17 12:35:28 +0000  bff92096-3a8e-4cb4-805b-55638be4aa23  14000                 11.3023           3.03366            9.04686                 0.0124663
2024-01-22 12:57:43 +0000  f8778f71-08b3-42af-8e71-6ce1b8335c8d  13000                 16.0607           3.93435           10.5545                  0.0231879
2024-01-24 12:48:41 +0000  d70a8864-b251-417a-b43b-d85a54604c0f  13000                 10.7205           3.05               8.23406                 0.015117
2024-01-29 16:41:22 +0000  28175f25-7f08-4b86-b754-cca0a59f35ea  13000                 15.3787           3.9335            10.4592                  0.0213644
2024-01-31 12:41:55 +0000  33feb742-30be-45c4-9849-52072e52aaee  13000                 11.1809           3.0081             9.01858                 0.0164261

Related Tickets & Documents

Related Issue #4
Closes #

Checklist before requesting a review

[ ] I have performed a self-review of my code.
[ ] If it is a core feature, I have added thorough tests.

Testing

Please describe the System Under Test.
Please provide detailed steps to perform tests related to this code change.
How were the fix/results from this change verified? Please provide relevant screenshots or results.

jtaleric commented 5 months ago

I tried running this in a fresh venv --

(test) jtaleric at polaris in ~/code/orion on hunter-integration▲ $ orion --help
Traceback (most recent call last):
  File "/home/jtaleric/code/orion/test/bin/orion", line 5, in <module>
    from orion import orion
  File "/home/jtaleric/code/orion/test/lib64/python3.12/site-packages/orion.py", line 14, in <module>
    from utils.orion_funcs import run_hunter_analyze, get_metadata, \
  File "/home/jtaleric/code/orion/test/lib64/python3.12/site-packages/utils/orion_funcs.py", line 13, in <module>
    from hunter.report import Report, ReportType
ModuleNotFoundError: No module named 'hunter'

paigerube14 commented 5 months ago

Similarly to joes issue, I hit this issue when trying to do pip install -r requirements.txt

% python --version
Python 3.10.7

INFO: pip is looking at multiple versions of astroid to determine which version is compatible with other requirements. This could take a while.
ERROR: Cannot install -r requirements.txt (line 1) and typing-extensions==3.10.0.2 because these package versions have conflicting dependencies.

The conflict is caused by:
    The user requested typing-extensions==3.10.0.2
    astroid 3.0.2 depends on typing-extensions>=4.0.0; python_version < "3.11"

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts

shashank-boyapally commented 5 months ago

I tried running this in a fresh venv --

(test) jtaleric at polaris in ~/code/orion on hunter-integration▲ $ orion --help
Traceback (most recent call last):
  File "/home/jtaleric/code/orion/test/bin/orion", line 5, in <module>
    from orion import orion
  File "/home/jtaleric/code/orion/test/lib64/python3.12/site-packages/orion.py", line 14, in <module>
    from utils.orion_funcs import run_hunter_analyze, get_metadata, \
  File "/home/jtaleric/code/orion/test/lib64/python3.12/site-packages/utils/orion_funcs.py", line 13, in <module>
    from hunter.report import Report, ReportType
ModuleNotFoundError: No module named 'hunter'

Just ran orion in a new venv it is working well on my end. Can you please check if the hunter is installed using pip list, I added hunter in the new requirements.txt

(newvenv) sboyapal orion (hunter-integration) >> orion --hunter-analyze
2024-02-05 13:23:06,835 - Orion - INFO - The test aws-small-scale-cluster-density-v2 has started
2024-02-05 13:23:06,835 - Matcher - INFO - Executing query against index=perf_scale_ci
2024-02-05 13:23:06,973 - Matcher - INFO - Executing query against index=ripsaw-kube-burner*
2024-02-05 13:23:08,718 - Orion - INFO - Collecting podReadyLatency
2024-02-05 13:23:08,718 - Matcher - INFO - Executing query against index=ripsaw-kube-burner
2024-02-05 13:23:08,890 - Orion - INFO - Collecting apiserverCPU
2024-02-05 13:23:08,890 - Matcher - INFO - Executing query against index=ripsaw-kube-burner
2024-02-05 13:23:11,217 - Orion - INFO - Collecting ovnCPU
2024-02-05 13:23:11,217 - Matcher - INFO - Executing query against index=ripsaw-kube-burner
2024-02-05 13:23:13,315 - Orion - INFO - Collecting etcdCPU
2024-02-05 13:23:13,315 - Matcher - INFO - Executing query against index=ripsaw-kube-burner
2024-02-05 13:23:15,807 - Orion - INFO - Collecting etcdDisck
2024-02-05 13:23:15,807 - Matcher - INFO - Executing query against index=ripsaw-kube-burner
time                       uuid                                    P99    apiserverCPU_cpu_avg    ovnCPU_cpu_avg    etcdCPU_cpu_avg    etcdDisck_duration_avg
-------------------------  ------------------------------------  -----  ----------------------  ----------------  -----------------  ------------------------
2023-10-16 19:25:28 +0000  e624b10f-148b-4871-acb5-e8a71cf1b605  13000                 10.175            2.42737            7.63694                 0.0115776
2023-10-18 15:11:43 +0000  325dd073-b680-4e44-951f-26506f401be0  13000                 10.4876           2.49018            8.12781                 0.0107434
2023-10-24 10:52:22 +0000  8cfc4fcf-46c7-4b9f-a464-cd100bfb84bf  13000                 10.3932           2.44498            7.60968                 0.0116801
                                                                                                ················                                               
                                                                                                          +13.8%                                               
                                                                                                ················                                               
2023-10-31 10:30:02 +0000  3c9e2789-59d4-4d6f-9f07-5c400cf25c93  13000                 11.0129           2.73347            8.83458                 0.0112971
2023-11-03 04:08:14 +0000  649228fb-9ff4-458a-8a8b-b9ecb521ca1b  13000                 10.8605           2.82005            8.34146                 0.0101805
2023-11-07 10:26:08 +0000  d8e0c965-4548-48d4-8a7a-c065b93ee766  15000                 10.5964           2.6356             7.48679                 0.0124896
2023-11-15 12:36:38 +0000  a990f7f9-1faf-41f7-b9fb-3b83f91b6e52  13000                 10.8222           2.81836            8.56302                 0.0112316
2023-11-22 12:53:45 +0000  e6c219dc-a9aa-4cf9-854a-fbd846c8ad22  13000                 10.8555           2.84967            8.20129                 0.0111251
2023-11-29 12:32:04 +0000  adc95590-791d-423b-9d09-8190178edbf6  13000                 11.4115           2.8615             8.67125                 0.0127318
                                                                                                                                     ························  
                                                                                                                                                       +28.5%  
                                                                                                                                     ························  
2023-12-06 12:34:57 +0000  5b06c830-be55-4c84-b4da-92720b55c304  13000                 11.05             2.85996            8.69638                 0.0134358
2023-12-20 12:37:17 +0000  c5f95336-27d4-4615-94c1-d618e975a36b  13000                 10.9253           2.77954            8.20148                 0.0137848
2023-12-27 12:36:59 +0000  bd5e6cad-6bc2-472d-948a-b1fbb350e80a  13000                 11.1967           2.77645            8.63235                 0.0154012
2024-01-03 12:33:09 +0000  b91c7156-0b86-4c79-81eb-60dfb7288070  13000                 10.7863           2.80557            8.54677                 0.0138396
                                                                                                ················                                               
                                                                                                          +23.9%                                               
                                                                                                ················                                               
2024-01-10 12:38:16 +0000  12a1750e-a7eb-48ed-bc00-35078d2bbad8  13000                 10.9379           3.10667            8.86329                 0.0154639
                                                                                                                  ·················                            
                                                                                                                             +15.2%                            
                                                                                                                  ·················                            
2024-01-10 22:59:25 +0000  c6b4aaa3-cd2c-4f04-af20-3d10c52449a2  13000                 11.4162           3.1338             9.29143                 0.0128269
                                                                        ······················                                                                 
                                                                                        +25.9%                                                                 
                                                                        ······················                                                                 
2024-01-11 17:30:26 +0000  c2963cec-e800-423f-92b1-3e6f692a55cb  13000                 16.1465           3.80976           10.1149                  0.0149901
2024-01-15 13:01:36 +0000  4f73a079-0088-4027-af0d-7c1b5b429fdd  13000                 15.0479           3.93557           10.0966                  0.0193588
2024-01-15 20:30:57 +0000  90189fbf-7181-4129-8ca5-3cc8d656b595  13000                 10.8181           3.05989            8.48998                 0.015606
2024-01-17 12:35:28 +0000  bff92096-3a8e-4cb4-805b-55638be4aa23  14000                 11.3023           3.03366            9.04686                 0.0124663
2024-01-22 12:57:43 +0000  f8778f71-08b3-42af-8e71-6ce1b8335c8d  13000                 16.0607           3.93435           10.5545                  0.0231879
2024-01-24 12:48:41 +0000  d70a8864-b251-417a-b43b-d85a54604c0f  13000                 10.7205           3.05               8.23406                 0.015117
2024-01-29 16:41:22 +0000  28175f25-7f08-4b86-b754-cca0a59f35ea  13000                 15.3787           3.9335            10.4592                  0.0213644
2024-01-31 12:41:55 +0000  33feb742-30be-45c4-9849-52072e52aaee  13000                 11.1809           3.0081             9.01858                 0.0164261
2024-02-05 12:57:47 +0000  ab482da1-a857-4d08-948a-897d9c670b64  13000                 16.3836           4.08402           10.4616                  0.0217523

jtaleric commented 5 months ago

  Using cached numpy-1.24.0.tar.gz (10.9 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [33 lines of output]
      Traceback (most recent call last):
        File "/home/jtaleric/code/orion/test/lib64/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/home/jtaleric/code/orion/test/lib64/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/home/jtaleric/code/orion/test/lib64/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 112, in get_requires_for_build_wheel
          backend = _build_backend()
                    ^^^^^^^^^^^^^^^^
        File "/home/jtaleric/code/orion/test/lib64/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 77, in _build_backend
          obj = import_module(mod_path)
                ^^^^^^^^^^^^^^^^^^^^^^^
        File "/usr/lib64/python3.12/importlib/__init__.py", line 90, in import_module
          return _bootstrap._gcd_import(name[level:], package, level)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
        File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
        File "<frozen importlib._bootstrap>", line 1310, in _find_and_load_unlocked
        File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
        File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
        File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
        File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
        File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
        File "<frozen importlib._bootstrap_external>", line 994, in exec_module
        File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
        File "/tmp/pip-build-env-rg1jzvtv/overlay/lib/python3.12/site-packages/setuptools/__init__.py", line 16, in <module>
          import setuptools.version
        File "/tmp/pip-build-env-rg1jzvtv/overlay/lib/python3.12/site-packages/setuptools/version.py", line 1, in <module>
          import pkg_resources
        File "/tmp/pip-build-env-rg1jzvtv/overlay/lib/python3.12/site-packages/pkg_resources/__init__.py", line 2172, in <module>
          register_finder(pkgutil.ImpImporter, find_on_path)
                          ^^^^^^^^^^^^^^^^^^^
      AttributeError: module 'pkgutil' has no attribute 'ImpImporter'. Did you mean: 'zipimporter'?
      [end of output]

Trying pip install -r requirements.txt I am running into this. with Python 3.12.1

paigerube14 commented 5 months ago

Think we should actually add in the installation of the requirements in each of these python version workflows so we can catch these failures before hand I updated this workflow in this pr, think we should try getting this other pr/requirement versions working before adding more in. Thoughts? CC: @jtaleric

jtaleric commented 5 months ago

Just to update what I did to make this work

16214  [2024-02-05 14:49:11] python3.11 -m venv test
16215  [2024-02-05 14:49:20] source test/bin/activate
16216  [2024-02-05 14:49:31] python3.11 setup.py install
16217  [2024-02-05 14:49:42] pip install -r requirements.txt 
16218  [2024-02-05 14:50:21] orion --help

jtaleric commented 5 months ago

Can we add to the readme on python versions to use and versions to avoid and why @shashank-boyapally ?

vishnuchalla commented 5 months ago

sorry for being a man in the middle here. I have seen many python repos struggling with dependency management quite a bit when there is only a requirements.txt. In order to get rid of these issues we need to have a dependency management tool integrated to the project. Here is one of them: https://pdm-project.org/latest/, which satisfies all the below asks

It should decide how we will manage our list of deps It should decide how we will lock/pin the set of deps It should decide how/when we will update deps to new versions It must work with RH's container building infrastructure It should be consistent when rebuilding It should be ideally compatible with Dependabot, so new updates can be fired up automatically (with all the CI-tooling involved)

cc: @shashank-boyapally

shashank-boyapally commented 5 months ago

Hi @vishnuchalla, thank you for adding this idea. This would relieve us from dependency management. We should add this in a future PR.

jtaleric commented 5 months ago

Lets home in on Python 3.11 -- not sure we need to worry with older versions of python.

shashank-boyapally commented 5 months ago

made changes accordingly to support only Python 3.11. Thank you for the insights @jtaleric @paigerube14

cloud-bulldozer / orion