uds-se / droidmate

DroidMate-2: A Platform for Android Test Generation
http://www.boxmate.org/
GNU General Public License v3.0
28 stars 14 forks source link

ISSUE -4 INCONSISTENT COVERAGE RESULTS #4

Closed mmilella94 closed 5 years ago

mmilella94 commented 5 years ago

I was trying to extract coverage information from the output of the tool.

I see that in the folder named “model” there is a file containing a set of statements (coverage.txt). Each statement is associated to a counter but its purpose is not clear to me. This counter is labeled as “time” but it is not clear to me what it represents. Is it the number of times the statement has been covered? Is the time passed (millis) between the beginning of the testing session and the moment in which the statement has been covered?

Also, I have seen that the folder “coverage” contains a set of files, each one containing the uuid of some statements. Are these all the statements that have been explored for a session? In this case how does droidmate understand when a session is finished (a new file of statement is needed)? I have tried to figure out which is the relationship between these files and the file inside the folder coverage but i wasn’t able to.

I need to know how to parse these data because i want to measure the code coverage of a testing session and extract the list of covered statements.

JeannedArk commented 5 years ago

Hi @mmilella94,

thank you for your interest.

I see that in the folder named “model” there is a file containing a set of statements (coverage.txt). Each statement is associated to a counter but its purpose is not clear to me. This counter is labeled as “time” but it is not clear to me what it represents. Is it the number of times the statement has been covered? Is the time passed (millis) between the beginning of the testing session and the moment in which the statement has been covered?

You almost got it right. The coverage.txt file contains a sorted list of statements associated with their first corresponding occurence. The time states the duration between the beginning of the session and the moment in which the statement has been explored in seconds. I will make that clearer. Thank you for pointing that out.

Also, I have seen that the folder “coverage” contains a set of files, each one containing the uuid of some statements. Are these all the statements that have been explored for a session?

Yes.

In this case how does droidmate understand when a session is finished (a new file of statement is needed)?

Each file does not mark a single session. Each file only contains a set of encountered statements from the former point in time at which the statements have been transferred (of course an exception for the first time). If you want to gain a deeper understanding, here is the implementation: https://github.com/uds-se/droidmate/blob/ebb7545889a514ca371fc1df54dc9eb95cbe7013/project/pcComponents/core/src/main/kotlin/org/droidmate/exploration/modelFeatures/StatementCoverageMF.kt#L174

For the code coverage have a look at our wiki: https://github.com/uds-se/droidmate/wiki/Getting-Statement-Coverage

I hope I could clear things up. Please, feel free to ask, if something is still unclear.

mmilella94 commented 5 years ago

Hi @JeannedArk , thank you very much for the answer and for the specific info for the coverage function.