saveourtool / save-cli

Universal test framework for cli tools [mainly for code analyzers and compilers]
MIT License
42 stars 4 forks source link

Support for real-time test reporting #426

Open orchestr7 opened 2 years ago

orchestr7 commented 2 years ago

Right now save-cli is writing test report after all logic is already done. We keep all results in the memory and only after all tests are processed save provides them to a reporter that is writing results.

We have two variants here: 1) in real time write results to the file (one test - one file) 2) append results to some pipe/file in a real time

That will be useful for save-cloud

petertrr commented 2 years ago

We keep all results in the memory and only after all tests are processed save provides them to a reporter that is writing results.

That's not true. Reporter already has a number of 'lifecycle listeners', e.g. onEvent(event: TestResult). PlainTextReporter is writing all data in real time, as does JSON reporter. However, for JSON reporter report becomes properly formatted only after the execution is finished, because JSON is a structured text format, unlike plain text.