Codecoverage reports must include .cls or .trigger extensions

[AllanOricil]

Summary

Because the keys in the coverage report do not include the file extension (.cls or .trigger), when a sandbox has a class and a trigger with the same name, only the code coverage of the class, or the trigger, is added to the code coverage report. For instance, I can create a class called Lead.cls and a trigger called Lead.trigger, which would result in a coverage.json having a single entry for Lead as shown below:

{
   "no-map/Lead":{
      "fnMap":{},
      "branchMap":{},
      "path":"no-map/Lead",
      "f":{ },
      "b":{},
      "s":{
         "1":1,
         "2":1
      },
      "statementMap":{
         "1":{
            "start":{
               "line":1,
               "column":0
            },
            "end":{
               "line":1,
               "column":0
            }
         }
      }
   }
}

cobertura.xml format has the same problem.

Steps To Reproduce:

1 - Create a class called Lead 2 - Create a trigger called Lead 3 - Write some apex test that covers both class and trigger 4 - run sfdx deploy validation command, but configure this command to return code coverage as a json.

use the following parameters as described in this doc: https://developer.salesforce.com/docs/atlas.en-us.sfdx_cli_reference.meta/sfdx_cli_reference/cli_reference_force_source.htm#cli_reference_force_source_deploy

[--resultsdir RESULTSDIR]
[--coverageformatters COVERAGEFORMATTERS]
[--junit]

5 - open the coverage report and verify it contains a single entry called no-map/Lead

Expected result

Code coverage report must include .cls or .trigger extensions

no-map/Lead.cls no-map/Lead.trigger

Actual result

Code coverage report does not include .cls or .trigger extensions

no-map/Lead

does this code coverage report belong to a class or a trigger? we can't determine it

System Information

bash

[github-actions[bot]]

Thank you for filing this issue. We appreciate your feedback and will review the issue as soon as possible. Remember, however, that GitHub isn't a mechanism for receiving support under any agreement or SLA. If you require immediate assistance, contact Salesforce Customer Support.

[peternhale]

Could you provide the exact command you are using and the results of sfdx plugins —verbose

[AllanOricil]

@peternhale

My CI first enqueues a deploy validation job

sfdx force:source:deploy -u $orgAlias -x dist/$orgAlias/package/package.xml -l RunLocalTests -w 0 -g -c --json --junit --coverageformatters=cobertura,html,json,text --resultsdir=./dist/$orgAlias/tests

then it extracts the job id from the previous command output, and start fetching the job result with

sfdx force:source:deploy:report -u $orgAlias -i $jobId -w 60 --json --junit --coverageformatters=cobertura,html,json,text --resultsdir=./dist/$orgAlias/tests

{
  "cliVersion": "sfdx-cli/[7](https://github.com/VodafoneIS/sf-metadata/actions/runs/3487804969/jobs/5836174029#step:3:8).176.1",
  "architecture": "linux-x64",
  "nodeVersion": "node-v14.20.1",
  "pluginVersions": [
    "@oclif/plugin-autocomplete 1.3.5 (core)",
    "@oclif/plugin-commands 2.2.1 (core)",
    "@oclif/plugin-help 5.1.17 (core)",
    "@oclif/plugin-not-found 2.3.7 (core)",
    "@oclif/plugin-plugins 2.1.6 (core)",
    "@oclif/plugin-update 3.0.6 (core)",
    "@oclif/plugin-version 1.1.3 (core)",
    "@oclif/plugin-warn-if-update-available 2.0.12 (core)",
    "@oclif/plugin-which 2.1.1 (core)",
    "alias 2.1.7 (core)",
    "apex 1.3.0 (core)",
    "auth 2.2.19 (core)",
    "community 2.0.14 (core)",
    "config 1.4.22 (core)",
    "custom-metadata 2.0.3 (core)",
    "data 2.1.7 (core)",
    "generator 2.0.9 (core)",
    "info 2.2.4 (core)",
    "limits 2.0.10 (core)",
    "org 2.2.11 (core)",
    "packaging 1.10.1 (core)",
    "schema 2.1.12 (core)",
    "signups 1.2.11 (core)",
    "source 2.3.0 (core)",
    "telemetry 2.0.2 (core)",
    "templates 55.1.0 (core)",
    "trust 2.0.4 (core)",
    "user 2.1.16 (core)",
    "@salesforce/sfdx-plugin-lwc-test 1.0.1 (core)",
    "salesforce-alm 54.[8](https://github.com/VodafoneIS/sf-metadata/actions/runs/3487804969/jobs/5836174029#step:3:9).3 (core)"
  ],
  "osVersion": "Linux 5.15.0-[10](https://github.com/VodafoneIS/sf-metadata/actions/runs/3487804969/jobs/5836174029#step:3:11)[22](https://github.com/VodafoneIS/sf-metadata/actions/runs/3487804969/jobs/5836174029#step:3:23)-azure",
  "shell": "bash",
  "rootPath": "/usr/local/lib/node_modules/sfdx-cli"
}

[peternhale]

The functionally for code coverage report s via deploy only supports summary level data. The class/trigger details are not currently available via metadata API. Please use the apex test run/report command to determine class/trigger coverage.

[AllanOricil]

@peternhale this would increase CI workflows usage time by a lot! I would have to rerun all apex class tests in every environment (dev, staging prod) at least once just to get the correct code coverage I need? It does not make sense.

[AllanOricil]

@peternhale @mshanemc

I believe this is fixable. I looked in the code and found where the Coverage Report is created.

https://github.com/forcedotcom/salesforcedx-apex/blob/main/src/reporters/coverageReporter.ts#L126-L138

Line 136 has a variable called file which has the file extension.

If Im right, I can open a PR with the fix.

edited: it is extremelly weird not having the file extension in this variable record.ApexClassOrTrigger.Name How is this possible?

[cromwellryan]

@AllanOricil thanks for taking a look. Any chance you can drop a pr for your suggestion? If you do, we can run this against our internal integration suite and see the impact. I can’t guarantee it’ll move forward quickly–there are some other things we have in min–but it’ll get us closer.

[AllanOricil]

Sure. I will investigate this 😀

[peternhale]

@AllanOricil I am going to close this issue. Please file a PR when you have found a suitable solution.

[AllanOricil]

@peternhale @cromwellryan I found two possible solutions:

1 - use ApexClassOrTriggerId prefix to determine if it is a class or a trigger. Apex classes start with 01p while triggers start with 01q

2 - use records[0].ApexClassOrTrigger.attributes.url to determine if it is a class or trigger. Might not work, because deployResult.response?.details?.runTestResult?.codeCoverage does not have attributes.url. Which means the following method wouldn't be able to properly convert to the format sfdx needs.

but Im still considering it as a solution because instead of transforming the deploy result codecoverage object, passing through the above function, I can simply run an extra api call to fetch all ApexCodeCoverageAggregate for a given deploy result id. If this works, Im expecting a json like this one https://github.com/forcedotcom/salesforcedx-apex/blob/f8ae861e6abb2a40af22cc2c0011c2ec7d7027c4/test/reporters/coverageReporter.test.ts#L14-L221

I will continue my experiments tomorrow. I believe I will have a PR by the end of this week.

[mcarvin8]

Is there any solution for this yet @AllanOricil ?

This would affect this plugin I just made which deals with converting the code coverage JSON into a XML that SonarQube accepts - https://github.com/mcarvin8/apex-code-coverage-transformer

I ran into this issue a while back where the JSON created automatically by the Salesforce CLI wasn't being accepted by SonarQube so I had to convert the JSON into an XML (did this in Python originally but I just converted it into a plugin).

Fortunately, my team does not have any classes or triggers with the same file name (😆 ) but it's definitely something that should be fixed for the whole community.

[AllanOricil]

@peternhale is taking care of it

[AllanOricil]

just realized that almost no one will have this problem. I don't know why I thought about testing it. I guess I like to find bugs.