search

fraunhoferhhi / vvenc

VVenC, the Fraunhofer Versatile Video Encoder
https://www.hhi.fraunhofer.de/en/departments/vca/technologies-and-solutions/h266-vvc.html
BSD 3-Clause Clear License
968 stars 173 forks source link

Unit testing of SIMD kernels #451

Closed georges-arm closed 1 week ago

georges-arm commented 1 month ago

When writing AArch64 Neon and SVE implementations for some of the MCTF kernels I noticed that it was quite easy to find cases (e.g. tail loops) which do not appear to be covered by the existing encoding tests run by CTest.

It doesn't appear that there are any existing unit tests for common kernels, so I was wondering if there was appetite to add such test cases to the library? I'd be happy to write test cases for some of the kernels that I have been looking at if you could let me know a preference in terms of test framework library or how best to integrate it with the existing test code?

Thanks!

adamjw24 commented 4 weeks ago

So, we don't.

When developing SIMD, I would just run a standard simulation and assume that if there is no differences in all of the encodings, it is correct. You need to know if your test utilizes your kernel or create a set of test conditions that do.

Is that something you could work with?

adamjw24 commented 4 weeks ago

If you want to write a simple test-suite... I would be reluctant to add external dependencies to the repo. It would need to have a license that allows us to just copy it into our repo, like we did with simde and the json library.

If you do decide to do something, lets discuss it before you put in the work.

georges-arm commented 3 weeks ago

If you want to write a simple test-suite... I would be reluctant to add external dependencies to the repo. It would need to have a license that allows us to just copy it into our repo, like we did with simde and the json library.

If you do decide to do something, lets discuss it before you put in the work.

Thanks for the detail.

I've put up #459 as a quick proof of concept of something that does not need external dependencies and can just run standalone. If you think something like this is suitable I'd be happy to go ahead and continue adding test cases as I write new kernels, since this by itself already helped a lot when developing my recent patch to add new Neon kernels for the TCoeffOps class.

Thanks!

adamjw24 commented 2 weeks ago

I will have a look as soon as I'm back from JVET next week. Sorry for the slow responsiveness atm.