Ouranosinc / PAVICS-e2e-workflow-tests

Test user-level workflow.
Apache License 2.0
0 stars 2 forks source link

stress-test.ipynb: wrong max_time calculation: ValueError: max() arg is an empty sequence #102

Open tlvu opened 2 years ago

tlvu commented 2 years ago

I have a feeling this return max(self.times) should have been something like return self.max_time (a constant set in __init__()), @ahandan what do you think?

12:28:26  Input In [2], in StressTestResult.max_time(self)
12:28:26       95 @property
12:28:26       96 def max_time(self):
12:28:26  ---> 97     return max(self.times)

Full Jenkins error:

12:28:26  _____________________ notebooks/stress-tests.ipynb::Cell 3 _____________________
12:28:26  Notebook cell execution failed
12:28:26  Cell 3: Cell execution caused an exception
12:28:26  
12:28:26  Input:
12:28:26  # NBVAL_IGNORE_OUTPUT
12:28:26  
12:28:26  # Executing WPS test on multiple threads
12:28:26  wps_urls = [
12:28:26      f"{TWITCHER_URL}/{bird}/wps?service=wps&request=getcapabilities"
12:28:26      for bird in TEST_WPS_BIRDS
12:28:26  ]
12:28:26  
12:28:26  for execution, url in enumerate(wps_urls):
12:28:26      run_threads(target_test = stress_test,
12:28:26                  url = url,
12:28:26                  execution_id = execution,
12:28:26                  n_threads = TEST_N_THREADS_WPS,
12:28:26                  runs_per_threads = TEST_RUNS_WPS)
12:28:26  
12:28:26  Traceback:
12:28:26  
12:28:26  ---------------------------------------------------------------------------
12:28:26  ValueError                                Traceback (most recent call last)
12:28:26  Input In [4], in <module>
12:28:26        4 wps_urls = [
12:28:26        5     f"{TWITCHER_URL}/{bird}/wps?service=wps&request=getcapabilities"
12:28:26        6     for bird in TEST_WPS_BIRDS
12:28:26        7 ]
12:28:26        9 for execution, url in enumerate(wps_urls):
12:28:26  ---> 10     run_threads(target_test = stress_test,
12:28:26       11                 url = url,
12:28:26       12                 execution_id = execution,
12:28:26       13                 n_threads = TEST_N_THREADS_WPS,
12:28:26       14                 runs_per_threads = TEST_RUNS_WPS)
12:28:26  
12:28:26  Input In [3], in run_threads(target_test, url, execution_id, n_threads, runs_per_threads)
12:28:26       34 for thread in threads:
12:28:26       35     thread.join()
12:28:26       37 assert (
12:28:26       38     stress_test_progression.failed_threads_count == 0
12:28:26  ---> 39 ), stress_test_progression.fail_report()
12:28:26       41 print(f"All threads passed {target_test.__name__} function.")
12:28:26  
12:28:26  Input In [2], in StressTestProgression.fail_report(self)
12:28:26       46     i+= 1
12:28:26       48 report_error_message += f"\n\n\n Failed Test"
12:28:26  ---> 49 report_error_message += f"\n {failed_result}"
12:28:26       51 report_error_message += f"\n\n === Exceptions ==="
12:28:26       52 for i, exception in enumerate(failed_result.request_exceptions):
12:28:26  
12:28:26  Input In [2], in StressTestResult.__str__(self)
12:28:26      124     summary = [
12:28:26      125         f"Detected {self.sum_err_code} erroneous HTTP codes not equal to expected {self.code}."
12:28:26      126     ]
12:28:26      127 elif self.status == -2:
12:28:26      128     summary = [
12:28:26      129         f"Detected regression with long request time.",
12:28:26  --> 130         f"Expected max-avg-time: ({self.max_avg_time:.3f}s <= {self.max_time:.3f}s)."
12:28:26      131     ]
12:28:26      132 elif self.status == -3:
12:28:26      133     summary = [
12:28:26      134         f"Maximum number of timeout ({self.timeout_abort}s) requests exceeded ({self.timeout_count}).",
12:28:26      135         "Test was aborted to avoid further delays."
12:28:26      136     ]
12:28:26  
12:28:26  Input In [2], in StressTestResult.max_time(self)
12:28:26       95 @property
12:28:26       96 def max_time(self):
12:28:26  ---> 97     return max(self.times)
12:28:26  
12:28:26  ValueError: max() arg is an empty sequence