Shinmera / parachute

An extensible and cross-compatible testing framework.
https://shinmera.github.io/parachute
zlib License
97 stars 9 forks source link

xunit xml reporting #39

Open tadas-subonis opened 2 years ago

tadas-subonis commented 2 years ago

How can I output my test results in xunit/junit XML format? This would help me integrate with CI easier.

Shinmera commented 2 years ago

You'll have to write a serialiser yourself.

tadas-subonis commented 2 years ago

Do you have any examples how to do that?

On Wed, Feb 9, 2022 at 12:58 PM Nicolas Hafner @.***> wrote:

You'll have to write a serialiser yourself.

— Reply to this email directly, view it on GitHub https://github.com/Shinmera/parachute/issues/39#issuecomment-1033684062, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAB4LMUYVACWJTS2QRPWNWLU2JJE5ANCNFSM5N5IIL4A . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you authored the thread.Message ID: @.***>

--

Kind Regards, Tadas Šubonis

Shinmera commented 2 years ago

No. test returns a report which has a results list. Transforming that into whatever format you need should be trivial, though.

daewok commented 2 years ago

I've got a quickly hacked together thing at: https://gitlab.common-lisp.net/cl-tar/cl-tar-file/-/blob/ee44e6e30a2074849af51199f99813141454e89c/test/report.lisp

An example of GitLab CI reading the results and displaying them: https://gitlab.common-lisp.net/cl-tar/cl-tar-file/-/pipelines/6095/test_report

Still a work in progress, but I plan to split it into its own system when it's a little more polished.

One thing I learned while doing it is there doesn't seem to be a single JUnit format. I wrote that to target GitLab. It might need some massaging to target other CI platforms (for instance, GitLab allows nested testsuite tags).

selwynsimsek commented 2 years ago

I've got a quickly hacked together thing at: https://gitlab.common-lisp.net/cl-tar/cl-tar-file/-/blob/ee44e6e30a2074849af51199f99813141454e89c/test/report.lisp

An example of GitLab CI reading the results and displaying them: https://gitlab.common-lisp.net/cl-tar/cl-tar-file/-/pipelines/6095/test_report

Still a work in progress, but I plan to split it into its own system when it's a little more polished.

One thing I learned while doing it is there doesn't seem to be a single JUnit format. I wrote that to target GitLab. It might need some massaging to target other CI platforms (for instance, GitLab allows nested testsuite tags).

We are interested in adding this functionality to Parachute as an extension. I've managed to get your code working locally to produce JUnit test reports.

How would you prefer to handle this? We could polish up your hack and include it, but it would have to be under the 'zlib' licence, like the rest of Parachute.

Shinmera commented 1 year ago

@daewok Ping