Testing of RNP - Githubissues

rnpgp / rnp

RNP: high performance C++ OpenPGP library used by Mozilla Thunderbird

https://www.rnpgp.org

Other

199 stars 55 forks source link

Testing of RNP #345

Open MohitKAgnihotri opened 7 years ago

MohitKAgnihotri commented 7 years ago

I would like to discuss the strategy for testing RNP to improve it and allow faster feedback for the developer.

Currently, I am facing issue with defining the approach to derive new test cases which will help us the most given the active development.

My first thoughts were

Module based testing mostly driven by use cases and then focusing on the coverage for the module.

Please provide your opinions and suggestions to better the testing approach.

MohitKAgnihotri commented 7 years ago

@ronaldtse @catap @ni4: Please share your opinions.

catap commented 7 years ago

I think we should also ask @dewyatt ;)

Anyway the first idea about testing is including tests to check compatibility with different versions of GnuPG

MohitKAgnihotri commented 7 years ago

@dewyatt : Please share your inputs on how can we better our testing.

ni4 commented 7 years ago

Ideally we should have 'static tests', 'dynamic tests' and 'self tests'. First ones are about processing of already existing OpenPGP data : i.e. all kinds of keys, signed files, encrypted files, with different encryption algorithms, hash algorithms, key algorithms, key versions and so on. Dynamic ones are about generating stuff with GnuPG, and then processing it with RNP, and vice versa. This is what I'm doing with Python now, but due to lack of time didn't put to pull request state yet. And self tests are what we have in cmocka - do some operation via lib and then test it for correctness via the lib. I.e. encrypt/decrypt.

ronaldtse commented 7 years ago

Agree with @catap and @ni4.

dewyatt commented 7 years ago

(I haven't had much time to think lately, so I don't have much valuable input - sorry!)

I do like the above ideas. (Side note: 'static tests' has an official definition in software testing that involves not running any code, but I think you mean 'tests on static data' maybe?)

I think it would also be good to add more generated test data (keyrings, etc) at every release, so we can test for regressions like #335.

ni4 commented 7 years ago

@dewyatt yeah, let's then call it 'tests on static data'.

ni4 commented 5 years ago

@ronaldtse @dewyatt Should we close it while we seem to have tests discussed here already?