randombit / botan

Cryptography Toolkit
https://botan.randombit.net
BSD 2-Clause "Simplified" License
2.59k stars 569 forks source link

Systematically time all test suites #4417

Open randombit opened 3 weeks ago

randombit commented 3 weeks ago

Some but not all of the tests report how long they took. Fix this.

Also the time reported should be normalized to always print seconds, instead of in ms if fast but in seconds if > 1 s.

KaganCanSit commented 1 week ago

Hello, @randombit @reneme

I have made the necessary changes for this bug report, but I think it would be useful to evaluate the situation I encountered and get more information in general.

First, I made adjustments to print duration values in seconds with a precision of 6 digits.

Next, I went through the test output and added calls to tests that did not have time measurement calls. This change required me to update quite a lot of files. Here, I thought about adjusting the time in the constructor and destructor of the result object, or evaluating different ideas, but I didn't want to break the existing code structure, considering the situation with the fields where the object is held in the vector.

After these changes, I noticed that when I ran the tests, the duration values ​​were not always printed due to the "if (m_ns_taken) > 0" condition of the "std::string Test::Result::result_string()" function in "test.cpp".

I tried to make these changes quickly (over the weekend), but I will review them again. At this point, I would like your guidance on the time measurement and the if condition I mentioned above.

I would love to work on this bug report if you can. (I didn't want to create a merge request for now.)

Commits: https://github.com/KaganCanSit/botan/commit/0182be711278c067e3f459d1c35672e57bd946cf https://github.com/KaganCanSit/botan/commit/8bc89882cf944142efb44cee14442e13f61a3b47