Dyalog / ullu

Testing primitives of DyalogAPL
MIT License
1 stars 0 forks source link

BUG: time counter reports incorrect time to run all the tests #46

Closed sloorush closed 8 months ago

sloorush commented 8 months ago
          Known issue: time counter reports incorrect time to run all the tests

Originally posted by @sloorush in https://github.com/Dyalog/ullu/issues/45#issuecomment-1865916900

sloorush commented 8 months ago

False alarm: The counter does not report incorrect time but it reports the CPU time instead of the elapsed time. I am not sure why CPU time and elapsed time are so different. I think running the tests multiple times causes too much time spent on IO things?

Log:

      ]performance.runtime 'unittest.RunTests tests.unique 0 0 5'

* Benchmarking "unittest.RunTests tests.unique 0 0 5"

Options:

Version information:
 Dyalog  18.2.45405 64-bit Unicode, BuildID 8ebc0eba 
 OS      Windows 10 or Windows Server 2016 64-bit    
 Link    3.0.19                                      
 SALT    2.9                                         
 UCMD    2.51                                        
 .NET                                                
 WS      18.2                                        
Repetition: 1 with Option:
      ⎕RL is set to: 1956353843

 #.tests.unique.test_unique 

Repetition 1 tests completed:  9488 ran, 0 failed test, 9488 successes

Repetition: 2 with Option:
      ⎕RL is set to: 1488808292

 #.tests.unique.test_unique 

Repetition 2 tests completed:  9488 ran, 0 failed test, 9488 successes

Repetition: 3 with Option:
      ⎕RL is set to: 406691272

 #.tests.unique.test_unique 

Repetition 3 tests completed:  9488 ran, 0 failed test, 9488 successes

Repetition: 4 with Option:
      ⎕RL is set to: 834366396

 #.tests.unique.test_unique 

Repetition 4 tests completed:  9488 ran, 0 failed test, 9488 successes

Repetition: 5 with Option:
      ⎕RL is set to: 1955270430

 #.tests.unique.test_unique 

Repetition 5 tests completed:  9488 ran, 0 failed test, 9488 successes

All tests completed:  5  repetitions, total tests: 47440 ran, 0 failed test, 47440 successes in 7.031 s

              (ms) 
 CPU (avg):   7031 
 Elapsed:    12198