search

softwareconstruction240 / autograder

Autograder for BYU's CS 240 Chess project
https://cs240.click
2 stars 2 forks source link

Test 160 proper history verification #336

Closed frozenfrank closed 1 month ago

frozenfrank commented 2 months ago

This includes unit tests for #233 (#160).

These unit tests pass and demonstrate that the code works. They would have revealed other bugs had we figured out these tests sooner.

Performance Testing

This system requires O(n*m) time to perform it's evaluations. For students with at most 200 commits with 100 patches each, the system will definitely finish in less than one second.

Interesting Charts

Visualizes the time required to evaluate different combinations of commits with patches. The red dot at (200, 200) represents the general maximum bound of any student repository.

Screenshot 2024-05-14 at 4 09 41 PM

Shows that the time complexity of the system is O(n * m) where n is the number of commits and m is the average number of patches in each commit. This could also be expressed as O(p) where p is the total number of patches in the selected history.

Screenshot 2024-05-14 at 4 15 16 PM

Shows the time results of each phase of the performance testing (repo initialization, commit generation, and history evaluation). The history evaluation takes significantly longer to run than simply generating the commits, and it's interesting to see the relationship between them charted.

Screenshot 2024-05-14 at 4 10 13 PM

The data and analysis is available to view on Google Sheets, and it has been shared with the TA account.

19mdavenport commented 2 months ago

Is there anything in particular you're stuck on, or just the whole process?

frozenfrank commented 1 month ago

@19mdavenport I'm now marking this as ready for review. Through much difficulty and many failed attempts, I finally figured out how to dynamically generate and evaluate the repos to demonstrate that the system behaves as intended.

I'll add in a few more tests using the now-established pattern, and I'll clean up some of the code a bit, but at this point it's nearly ready.

It passes the tests and provides tools that will be helpful for manual debugging in the future.

This PR also contains a few improvements and optimizations to the actual git verification system as well.

Please let me know what adjusts you would like. Overall, I feel the method of testing has been well designed and flexibly implemented.