commaai / panda

code powering the comma.ai panda
MIT License
1.52k stars 766 forks source link

Enable cache for MISRA mutation test #1799

Closed bongbui321 closed 8 months ago

bongbui321 commented 8 months ago

Now mutation test can cache from ~45s to ~10s, replace tempDirectory() so that we can name the directory ourselves

9.21s call     panda/tests/misra/test_mutation.py::test_misra_mutation[None-None-False]
8.92s call     panda/tests/misra/test_mutation.py::test_misra_mutation[board/stm32h7/llfdcan.h-s/return ret;/if (true) { return ret; } else { return false; }/g-True]
8.63s call     panda/tests/misra/test_mutation.py::test_misra_mutation[board/safety/safety_toyota.h-s/is_lkas_msg =.*;/is_lkas_msg = addr == 1 || addr == 2;/g-True]
8.48s call     panda/tests/misra/test_mutation.py::test_misra_mutation[board/stm32fx/llbxcan.h-s/1U/1/g-True]

(6 durations < 0.005s hidden.  Use -vv to show these durations.)
=================================================================== 4 passed in 9.97s ====================================================================
adeebshihadeh commented 8 months ago

How is this supposed to work? From what I understand, cppcheck will only use the cache if nothing changed?

bongbui321 commented 8 months ago

Are you not sure of how this work or you don't like it? I changed the way how we created tempdir so that the directory used for each mutation is the same everytime, and each mutation has its own cache The reason each mutation was not caching before was because we use tempDirectory() which generates random directory name everytime, which doesn't match with the last time it was cached. Now each mutation will have constant directory name and has its own cache

adeebshihadeh commented 8 months ago

Ah, so this just caches each patch between consecutive runs?