How to get Fitnesse test statistics in my fixture

[Ted201606]

I'm trying to get the test statistics counted by Fitnesse (see below) in my fixture and send them in my email report: Test Pages: 2 right, 1 wrong, 0 ignored, 0 exceptions Assertions: 25 right, 38 wrong, 86 ignored, 0 exceptions (2.569 seconds)

and also trying to get the test start time so that I can attach below URL in the email report to show the test history. http://localhost:8080/Test?pageHistory&resultDate=20160602163022

Are there any public APIs I can call from my fixture code to get these data? Please kindly advise, thanks!

[amolenaar]

Hi,

Why do you want to do this from within a fixture? Fixtures are typically used to interact with your System Under Test (SUT).

It might be simpler to create a script/application that generates the report after the tests have run. You can use the classes in fitnesse.reporting.history for that.

Regards,

Arjan

On 02 Jun 2016, at 11:00, Ted201606 notifications@github.com wrote:

I'm trying to get the test statistics counted by Fitnesse (see below) in my fixture and send them in my email report: Test Pages: 2 right, 1 wrong, 0 ignored, 0 exceptions Assertions: 25 right, 38 wrong, 86 ignored, 0 exceptions (2.569 seconds)

and also trying to get the test start time so that I can attach below URL in the email report to show the test history. http://localhost:8080/Test?pageHistory&resultDate=20160602163022

Are there any public APIs I can call from my fixture code to get these data? Please kindly advise, thanks!

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

[Ted201606]

Thanks Arjan! I want that info in my fixture because I need to trigger the email report inside of a fitnesse test page before the test end, and the email report needs to have these info. I don't want to create another application/script to send the email. Any way I can achieve my goal?

[woodybrood]

This has come up before. The fixtures don't have access to that data. They run inside slim and they don't have access to the whole page. Test result counts and run time are all tracked in the server, as I recall. So, I don't think there is any way to do this.

The best choice is to have the way you run the tests capture that data and send it.

Why does it have to be in the test? That could actually result in a lot of noise (from debugging and such). Better to have it done through you CI process.

On June 2, 2016 at 8:43:02 AM, Ted201606 (notifications@github.com) wrote:

Thanks Arjan! I want that info in my fixture because I need to trigger the email report inside of a fitnesse test page before the test end, and the email report needs to have these info. I don't want to create another application/script to send the email. Any way I can achieve my goal?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/unclebob/fitnesse/issues/932#issuecomment-223295184, or mute the thread https://github.com/notifications/unsubscribe/AAIJPh2d4xTgoZ3ZbQ2toob6Pec3nVBQks5qHt3mgaJpZM4IsV6c .

[Ted201606]

Thank you! My case is different from normal automated testing on SUT, instead, I'm using JDBC slim to test database tables. I need to calculate some statistics like row counts, column counts and etc during my testing, and summarize them into a email report at the end of the test. And to unify the AT platform I don't want to create other standalone script or applications to run the test.

Jenkins is an option, but I found it's not easy to generate the HTML report format and get the contents I want. Do you happen to have any guides on this?

So I looked into the source code of Fitnesse, and found a way to get the date string of test history page. I persisted the start time into a file at testStarted function in fitnesse.reporting.history.TestXmlFormatter.java . Then read the file and get the start time during testing. It may not be a correct way to do that but meets my initial requirement. And with this way I still can't get other statistics like pass, failed, ignore counts, because I don't know when and how to persist these data.

[amolenaar]

The problem here is is that you're crossing the barrier between the test runner (FitNesse) and your system (SUT).

The information you want to acquire lives in the realm of the runner. FitNesse has a couple of plug points where you can add your own custom functionality. In your case, that might be a custom formatter or a custom table type (depending on the information you require).

@fhoeben has some code that can give you some inspiration: https://github.com/fhoeben/hsac-fitnesse-plugin.

[fhoeben]

I'm not sure what data you need exactly but besides the plugin I created for some custom table types, that @amolenaar refers to, I believe another approach might be more suitable: I created my own subclass of FitNesse's own jUnit test runner. This is a very nice way, for me, to get full access to the test life cycle and ensure reporting the way I want it.

Some key benefits:

• A full HTML report of the test run is created by the default runner (which you could include in your email). You can find a sample in my project.
• FitNesse and Slim are run in a single Java process by default, allowing you access to information from FitNesse and your fixtures in a single place.
• You get full access to the life cycle: before tests are started, each test start/finish/failure, run completion. This gives you full control to record extra information and send an email when a run completes.
• A jUnit test run is easy to set up as a build job in a server like Jenkins: with its features to schedule/trigger runs and keep history.

The only thing you describe I do not believe you can access is the number of tests ignored.

[six42]

Hi Ted,

How do you plan to send the email with the results?

[Ted201606]

@amolenaar @fhoeben Thanks very much for your help on this topic, that's exactly what I want! I will look into fhoeben's project for the details and see how can I introduce it to my project. Thanks again!

[Ted201606]

Hi @six42 I will follow fhoeben's suggestion as above :) Thanks for your great shared jdbcslim project, I customized it a little bit to support data comparison which is quite helpful to my project.

[six42]

Great that you found a solution for your problem.

Please send me an email with details about the customisation you did to jdbcSlim. I am always interested in ideas how it can be improved.

[amolenaar]

Can I close this issue?