amplify-appsync-simulator - Coverage reports

[chucre]

As we start relying on automated test execution to deploy our application we want to know how well tested are our applications. One of the most common metrics is code coverage. Since AppSync relies on VLT and a very good amount of code is in VLT it will be nice, even mandatory so some companies, have a report for that coverage.

Describe the solution you'd like I like to have a Istanbul-like report so I can generate other reports (html, lcov, etc, xml).

The Istanbul format requires a full path, but since we may not have it, I think it's acceptable the internal format that VelocityTemplate uses to store the path (e.g: Mutation.unblockPatient.request.vtl).

[nikhname]

@chucre, could you give some details about the type of coverage data you would like to see?

[chucre]

Hi @nikhname,

I started a plugin that tries to add the coverage in the serverless-appsync-simulator which uses amplify-appsync-simulator.

The plugin serverless-appsync-simulator-coverage hijack thegetReferences, setValue, getMacro, getBlock methods of the class Compile from amplify-velocity-template and add a few other more.

You can check an example here.

The result is that it's generate a file coverage.json and we can then generate reports like this:

and

I'm able to do this by manipulating the asts tree in the Compile instance. I did a few changes that don't impact the render process.

But It doesn't fill all aspects of the coverage. For instance, it doesn't count correct the branch path, since I don't have a clear place to count it.

So the result in the report is something like this:

You can clearly see that the else was the path (check the hit line), but my plugin missed it.

Ideally the Compile class should add a counter in the asts tree and the plugin just need to format it correctly.