Incomplete JSON files getting generated for coverage results

[pjg005]

I have complete setup of JSCover with our Selenium Webdriver. Initially when I tried running 10-20 test cases, the coverage shown was perfect and I was able to merge those reports as well. Now, we have about 4000 test cases to be executed. When I executed these testcases, json files were generated but they were incomplete and ended abruptly. e.g. it ended at nu instead of null. Is there any size limitation for json files been generated by JSCover?

Also, I had observed that when we run the test cases on Selenium using chrome driver, it considers every chrome driver instance as a fresh one and no previous data is stored in --local-storage of the browser. Is this observation correct? Because due to this, I have scripted the webdriver to provide JSON per test case script.

[tntim96]

it ended at nu instead of null

How big is the resulting file in bytes?

it considers every chrome driver instance as a fresh one and no previous data is stored in --local-storage

If I understand correctly, that's not the correct behaviour. I assume you're using JSCover's local-storage option which adds the following code:

if (typeof(_$jscoverage) === "undefined" && (typeof(Storage) !== "undefined") && typeof(localStorage["jscover"]) !== "undefined")
    _$jscoverage = jscoverage_parseCoverageJSON(localStorage["jscover"]);
if (typeof(jscoverbeforeunload) === "undefined") {
    jscoverbeforeunload = (window.onbeforeunload) ? window.onbeforeunload : function () {};
    window.onbeforeunload = function () {
        jscoverbeforeunload();
        if ((typeof(_$jscoverage) !== "undefined") && (typeof(Storage) !== "undefined"))
            localStorage["jscover"] = jscoverage_serializeCoverageToJSON();
    };
}

Is there any code that could interfere with the window.onbeforeunload function?

[pjg005]

The json file is varying from 1000 to 4500 kbs. The max file size I have observed is of 4768 kbs.

I have used the function jscoverage_serializeCoverageToJSON() in Webdriver for json generation. I am not explicitly calling '--local-storage' anywhere. So I believe its the browser's storage itself. Anyways, there is no code that would interfere with the window.onbeforeunload function. We are using chrome with limited desired capabilities defined and driver.close()/driver.quit() function after every test script.

I would require per test case coverage which I would merge later on for overall application coverage. Which storage option would be the best in this case?

[alonsocucei]

Hi, Probably the same issue, but not sure. I'm using Qunit to run around 80 tests, which use more than 100 files, in the report I detected 43 files with incorrect percentage number. I know they are incorrect because running just the tests that use those files show greater numbers (the ones I expect). I'm using version 1.0.19 The arguments I'm passing are: -fs --local-storage --no-instrument-reg=....

I agree with @poojagangurde, it seems to be a size limitation. If it's a different issue do you know if it's solved in 1.0.24?

Thanks.

[tntim96]

I've got some time to look at this now, so I'll try to reproduce this. Any extra hints to narrow the problem would be helpful. Just some thoughts to check:

1. Is there any error in jscover.log?
2. Does the JVM have enough memory?
3. Does it matter which browser you use?
4. Can you run the same test but run jscoverage_report to generate the jscoverage.json file? Is it truncated?

@poojagangurde

I would require per test case coverage which I would merge later on for overall application coverage. Which storage option would be the best in this case?

Unless you're noticing a problem you may not need anything. You may want to use --isolate-browsers as in https://github.com/tntim96/JSCover/issues/197 depending on your usage.

@alonsocucei

I'm using version 1.0.19...do you know if it's solved in 1.0.24?

No - I haven't fixed anything like that in recent revisions. What size JSON file are you getting? Are you also calling the function jscoverage_serializeCoverageToJSON in Webdriver?

[alonsocucei]

I cannot reproduce the issue again, it was happening with a previous version of the project, but a teammate changed the order of the tests and it seems that was the problem. However, I'll give you the info I have available in case it's needed in the future:

1.- No there wasn't any error. 2.- I'm executing it from a gradle task, so I don't really know if gradle modifies the JVM memory. 3.- I'm using a virtual machine where I can't install any extra software, I open the page with firefox 4.- As I'm not able to reproduce what was happening I don't think it makes sense to try this now.

JSON file is 238 K

Thanks.

[pjg005]

1. There are no error in jscover.log file.
2. I have allocated 4096 kbs as off heap memory to the tomcat server where instrumented application is hosted.
3. Our application gives best support on chrome. There are UI issues on other browsers. So, I haven't tested that on other browsers. I am using Chrome version 51.
4. I used jscoveragereport but it displayed error 'Error: SyntaxError: Unexpected token ILLEGAL’_

I would try the test scripts with --isolate-browsers and update the status.

Thanks in advance.

[tntim96]

I should have stated that jscoverage_report requires you to be running in server mode. Are you?

Anyway, I was able to generate a jscoverage.json file of 641,716 bytes with jscoverage_report, but when I save is from jscoverage_serializeCoverageToJSON(), it was only 532,480 bytes - and corrupt as you described.

This looks like a limitation of JavascriptExecutor.

I'm open to any elegant way of solving this. I'll have to think about this, but if you've got any suggestions, let me know.

[tntim96]

When debugging, all the JSON appeared to be there, but when saved with Apache Commons IOUtils, the data appears to be lost unless I called IOUtils.closeQuietly(output);, where output is the output stream you're writing to.

Can you check you're closing the stream properly?

[pjg005]

Hello,

I hadn't used the IOUtils.closeQuietly(output); in our code. I tried this and ran 5 test scripts. The output is now getting generated properly and json files are not getting truncated. I guess that has solved the issue.

Let me execute all the test scripts today and update you.

Thanks.

[pjg005]

Adding IOUtils.closeQuietly(output); in the code solved the problem. Proper json files are getting generated for all test cases and coverage shown is perfect.

Thanks!!!