Inconsistent code coverage formats

rp123 commented 10 years ago

We have been able to get cobertura code coverage for robolectic, using the steps you mentioned.

For UI Automation, we are using MonkeyRunner. We have Emma code coverage setup for MonkeyRunner.

Now we would like to aggregate code coverage, but are running into format issue.

Questions: Is it possible to get Emma code coverage for robolectic? Or Is it possible to get Cobertura code coverage for MonkeyRunner?

Goal is to have the code coverage reports in the same format so we can merge it.

Thanks!

rp123 commented 10 years ago

@stephanenicolas could you please a quick look at this and make a suggestion? Thanks!

stephanenicolas commented 10 years ago

Hi rp123,

No, you can't make emma work with Robolectric. At least, I couldn't and I remember I tried hard. But I think that the way Robolectric manipulates bytecode is not compatible with the way Emma does it.

For Cobertura and monkey runner, I really can't say. To my mind, what is gonna bug is that cobertura must provide a offline mode for this to work. Apparently, it can run this way : http://rwatsh.blogspot.fr/2008/03/emma-vs-cobertura.html So yes, it should work, but cobertura is a bit difficult to handle as it as its own lifecycle.

On the hand, I think all of this could be easier with Jacoco. It has offline instrumentation working, and seems to work with all technologies.

Really sorry for the delay, I didn't have much time up to now to work on QAT again. Stéphane

2013/10/7 rp123 notifications@github.com

@stephanenicolas https://github.com/stephanenicolas could you please a quick look at this and make a suggestion? Thanks!

— Reply to this email directly or view it on GitHubhttps://github.com/stephanenicolas/Quality-Tools-for-Android/issues/14#issuecomment-25779337 .

Stéphane NICOLAS, OCTO Technology Développeur & Consultant Android / Java .......................................................... 50, Avenue des Champs-Elysées 75008 Paris +33 (0)6.26.32.34.09 www.octo.com - mobilite.octo.com blog.octo.com - www.usievents.com ...........................................................

rp123 commented 10 years ago

Thanks Stephane for your detailed response. I'll give Jacoco a try based on your recommendation. Will let you know how it goes.

stephanenicolas commented 10 years ago

We would be pleased to have a pull request for that. In that case, open a new ticket please.

I think there is something to dig here. This could turn into a measure of completion of monkey testing : having covered enough code...

stephanenicolas commented 10 years ago

That's for maven, right ? Otherwise with gradle, just forget for now.

2013/10/7 rp123 notifications@github.com

Thanks Stephane for your detailed response. I'll give Jacoco a try based on your recommendation. Will let you know how it goes.

— Reply to this email directly or view it on GitHubhttps://github.com/stephanenicolas/Quality-Tools-for-Android/issues/14#issuecomment-25846874 .

Stéphane NICOLAS, OCTO Technology Développeur & Consultant Android / Java .......................................................... 50, Avenue des Champs-Elysées 75008 Paris +33 (0)6.26.32.34.09 www.octo.com - mobilite.octo.com blog.octo.com - www.usievents.com ...........................................................

rp123 commented 10 years ago

Yes, this was with Maven. I tried the config file from (https://github.com/stephanenicolas/Quality-Tools-for-Android/blob/master/android-sample-robolectric-tests/pom.xml and https://github.com/stephanenicolas/Quality-Tools-for-Android/blob/master/pom.xml), but it seems to be using sonar. I am able to generate jacoco.exec file, but was looking for a way to generate HTML report outside of Sonar.

I also noticed that the plugin had runtime, I was wondering if this was still correct in case of offline instrumentation?

Is there something different I need to do for offline instrumentation?

stephanenicolas commented 10 years ago

Hi rp123,

This config works offline ! ;) I think there is a report goal to the jacoco plugin that you can use to generate a report.

It should work either in post test phase or in reporting phase. Probably the latter. There should be no problem to get it running with the exec file you got already.

S. Le 8 oct. 2013 20:13, "rp123" notifications@github.com a écrit :

Yes, this was with Maven. I tried the config file from ( https://github.com/stephanenicolas/Quality-Tools-for-Android/blob/master/android-sample-robolectric-tests/pom.xmland https://github.com/stephanenicolas/Quality-Tools-for-Android/blob/master/pom.xml), but it seems to be using sonar. I am able to generate jacoco.exec file, but was looking for a way to generate HTML report outside of Sonar.

I also noticed that the plugin had runtime, I was wondering if this was still correct in case of offline instrumentation?

Is there something different I need to do for offline instrumentation?

— Reply to this email directly or view it on GitHubhttps://github.com/stephanenicolas/Quality-Tools-for-Android/issues/14#issuecomment-25914037 .

stephanenicolas / Quality-Tools-for-Android

Inconsistent code coverage formats #14