search

sebastianbergmann / phpcov

TextUI frontend for php-code-coverage
BSD 3-Clause "New" or "Revised" License
223 stars 58 forks source link

PHPUnit vs PHPCov coverage stats #88

Closed thewholelifetolearn closed 4 years ago

thewholelifetolearn commented 5 years ago

To speed-up my tests, I decided to give a try to phpcov merge.
Until now, I was doing a PHPUnit test coverage on the entire application and now I split the tests depending on the directories. It comes out that the result isn't the same. It happens to be that even the number of methods and lines aren't equal!
It seems that PHPUnit counts only named functions, against PHPCov that counts named and anonymous functions. Regarding the lines, I have no clue for the difference.

I'm using PHPUnit 7.3.5 (installed via Composer) and PHPCov 5.0.0 (installed via Phar).

Here are the outputs:
PHPUnit:

Code Coverage Report:        
  2019-02-26 09:53:53        

 Summary:                    
  Classes: 43.84% (32/73)    
  Methods: 75.66% (373/493)  
  Lines:   83.55% (2478/2966)

\AppBundle\Command::AppBundle\Command\AgentReinitPwdCommand
  Methods:  80.00% ( 4/ 5)   Lines:  95.95% ( 71/ 74)
\AppBundle\Command::AppBundle\Command\DatabaseVersionCommand
  Methods:  66.67% ( 2/ 3)   Lines:  43.75% (  7/ 16)
\AppBundle\Command::AppBundle\Command\HarpeLeaderUnitCommand
  Methods:  66.67% ( 2/ 3)   Lines:  22.92% ( 11/ 48)
\AppBundle\Command::AppBundle\Command\HarpePersonneFullCommand
  Methods:  25.00% ( 2/ 8)   Lines:  28.21% ( 11/ 39)
\AppBundle\Command::AppBundle\Command\HarpeStructureFullCommand
  Methods:  25.00% ( 2/ 8)   Lines:  18.03% ( 11/ 61)
\AppBundle\Command::AppBundle\Command\ImportAffaireCommand
  Methods: 100.00% ( 3/ 3)   Lines: 100.00% ( 49/ 49)
\AppBundle\Command::AppBundle\Command\ImportCommand
  Methods:  66.67% ( 2/ 3)   Lines:  65.67% ( 44/ 67)
\AppBundle\Controller::AppBundle\Controller\AffaireController
  Methods: 100.00% ( 7/ 7)   Lines: 100.00% ( 56/ 56)
\AppBundle\Controller::AppBundle\Controller\AgentController
  Methods:  71.43% ( 5/ 7)   Lines:  96.20% ( 76/ 79)
\AppBundle\Controller::AppBundle\Controller\AttributController
  Methods:  83.33% ( 5/ 6)   Lines:  80.77% ( 42/ 52)
\AppBundle\Controller::AppBundle\Controller\ContractController
  Methods:  85.71% ( 6/ 7)   Lines:  97.06% ( 66/ 68)
\AppBundle\Controller::AppBundle\Controller\DashboardController
  Methods:  71.43% ( 5/ 7)   Lines:  99.19% (245/247)
\AppBundle\Controller::AppBundle\Controller\ImportController
  Methods:   0.00% ( 0/ 3)   Lines:  21.74% ( 10/ 46)
\AppBundle\Controller::AppBundle\Controller\IndicatorController
  Methods: 100.00% ( 7/ 7)   Lines: 100.00% ( 61/ 61)
\AppBundle\Controller::AppBundle\Controller\TeamController
  Methods: 100.00% ( 8/ 8)   Lines: 100.00% ( 85/ 85)
\AppBundle\Controller::AppBundle\Controller\UnitController
  Methods: 100.00% ( 8/ 8)   Lines: 100.00% ( 88/ 88)
\AppBundle\Entity::AppBundle\Entity\Activity
  Methods:  35.71% ( 5/14)   Lines:  50.00% ( 13/ 26)
\AppBundle\Entity::AppBundle\Entity\Affaire
  Methods:  76.47% (13/17)   Lines:  84.21% ( 32/ 38)
\AppBundle\Entity::AppBundle\Entity\Agent
  Methods:  93.33% (42/45)   Lines:  97.27% (178/183)
\AppBundle\Entity::AppBundle\Entity\AgentTeam
  Methods:  75.00% ( 3/ 4)   Lines:  87.50% (  7/  8)
\AppBundle\Entity::AppBundle\Entity\AgentUnit
  Methods:  75.00% ( 3/ 4)   Lines:  87.50% (  7/  8)
\AppBundle\Entity::AppBundle\Entity\Attribut
  Methods:  82.61% (19/23)   Lines:  85.11% ( 40/ 47)
\AppBundle\Entity::AppBundle\Entity\Contract
  Methods:  63.64% (21/33)   Lines:  77.78% ( 70/ 90)
\AppBundle\Entity::AppBundle\Entity\Import
  Methods:  55.56% (10/18)   Lines:  66.67% ( 20/ 30)
\AppBundle\Entity::AppBundle\Entity\Indicator
  Methods:  84.62% (11/13)   Lines:  82.14% ( 23/ 28)
\AppBundle\Entity::AppBundle\Entity\LeaderTeam
  Methods:  75.00% ( 3/ 4)   Lines:  87.50% (  7/  8)
\AppBundle\Entity::AppBundle\Entity\LeaderUnit
  Methods:  75.00% ( 3/ 4)   Lines:  87.50% (  7/  8)
\AppBundle\Entity::AppBundle\Entity\Team
  Methods:  88.46% (23/26)   Lines:  82.09% ( 55/ 67)
\AppBundle\Entity::AppBundle\Entity\Unit
  Methods:  83.33% (20/24)   Lines:  84.00% ( 42/ 50)
\AppBundle\Form::AppBundle\Form\AddAgentToTeamType
  Methods: 100.00% ( 2/ 2)   Lines: 100.00% ( 13/ 13)
\AppBundle\Form::AppBundle\Form\AffaireEditType
  Methods: 100.00% ( 1/ 1)   Lines: 100.00% (  3/  3)
\AppBundle\Form::AppBundle\Form\AffaireType
  Methods: 100.00% ( 3/ 3)   Lines: 100.00% (  9/  9)
\AppBundle\Form::AppBundle\Form\AgentType
  Methods: 100.00% ( 4/ 4)   Lines: 100.00% ( 16/ 16)
\AppBundle\Form::AppBundle\Form\AttributEditInIndicatorType
  Methods: 100.00% ( 3/ 3)   Lines: 100.00% (  7/  7)
\AppBundle\Form::AppBundle\Form\AttributInIndicatorType
  Methods: 100.00% ( 3/ 3)   Lines: 100.00% (  9/  9)
\AppBundle\Form::AppBundle\Form\AttributType
  Methods: 100.00% ( 3/ 3)   Lines: 100.00% ( 12/ 12)
\AppBundle\Form::AppBundle\Form\ContractEditType
  Methods: 100.00% ( 1/ 1)   Lines: 100.00% (  5/  5)
\AppBundle\Form::AppBundle\Form\ContractType
  Methods: 100.00% ( 3/ 3)   Lines: 100.00% ( 26/ 26)
\AppBundle\Form::AppBundle\Form\DashboardInstantType
  Methods: 100.00% ( 3/ 3)   Lines: 100.00% ( 37/ 37)
\AppBundle\Form::AppBundle\Form\DashboardType
  Methods: 100.00% ( 3/ 3)   Lines: 100.00% ( 41/ 41)
\AppBundle\Form::AppBundle\Form\ImportAffaireType
  Methods: 100.00% ( 3/ 3)   Lines: 100.00% (  9/  9)
\AppBundle\Form::AppBundle\Form\IndicateurEditType
  Methods: 100.00% ( 1/ 1)   Lines: 100.00% (  7/  7)
\AppBundle\Form::AppBundle\Form\IndicateurType
  Methods: 100.00% ( 3/ 3)   Lines: 100.00% ( 11/ 11)
\AppBundle\Form::AppBundle\Form\TeamEditType
  Methods: 100.00% ( 1/ 1)   Lines: 100.00% (  4/  4)
\AppBundle\Form::AppBundle\Form\TeamType
  Methods: 100.00% ( 3/ 3)   Lines: 100.00% ( 27/ 27)
\AppBundle\Form::AppBundle\Form\UniteEditType
  Methods: 100.00% ( 1/ 1)   Lines: 100.00% (  4/  4)
\AppBundle\Form::AppBundle\Form\UniteType
  Methods: 100.00% ( 3/ 3)   Lines: 100.00% ( 16/ 16)
\AppBundle\Form\EventListener::AppBundle\Form\EventListener\AgentListFieldSubscriber
  Methods: 100.00% ( 2/ 2)   Lines: 100.00% ( 14/ 14)
\AppBundle\Form\EventListener::AppBundle\Form\EventListener\ParentAttributeListFieldSubscriber
  Methods:  50.00% ( 1/ 2)   Lines:  92.31% ( 12/ 13)
\AppBundle\Form\EventListener::AppBundle\Form\EventListener\RoleListFieldSubscriber
  Methods: 100.00% ( 3/ 3)   Lines: 100.00% ( 13/ 13)
\AppBundle\Manager::AppBundle\Manager\HarpePersonneManager
  Methods:  87.50% (14/16)   Lines:  91.37% (127/139)
\AppBundle\Manager::AppBundle\Manager\TeamManager
  Methods:  75.00% ( 6/ 8)   Lines:  78.57% ( 22/ 28)
\AppBundle\Manager::AppBundle\Manager\UnitManager
  Methods:  66.67% ( 6/ 9)   Lines:  68.75% ( 22/ 32)
\AppBundle\Menu::AppBundle\Menu\MenuBuilder
  Methods:  50.00% ( 1/ 2)   Lines:  97.30% ( 36/ 37)
\AppBundle\Model::AppBundle\Model\AgentAssociation
  Methods: 100.00% ( 6/ 6)   Lines: 100.00% ( 13/ 13)
\AppBundle\Repository::AppBundle\Repository\ActivityRepository
  Methods: 100.00% ( 8/ 8)   Lines: 100.00% ( 81/ 81)
\AppBundle\Repository::AppBundle\Repository\AffaireRepository
  Methods: 100.00% ( 3/ 3)   Lines: 100.00% ( 26/ 26)
\AppBundle\Repository::AppBundle\Repository\AgentRepository
  Methods: 100.00% ( 8/ 8)   Lines: 100.00% ( 89/ 89)
\AppBundle\Repository::AppBundle\Repository\AttributRepository
  Methods:  80.00% ( 4/ 5)   Lines:  96.72% ( 59/ 61)
\AppBundle\Repository::AppBundle\Repository\ContractRepository
  Methods:  71.43% ( 5/ 7)   Lines:  82.50% ( 99/120)
\AppBundle\Repository::AppBundle\Repository\ImportRepository
  Methods:  25.00% ( 1/ 4)   Lines:  32.43% ( 12/ 37)
\AppBundle\Repository::AppBundle\Repository\IndicatorRepository
  Methods:  80.00% ( 4/ 5)   Lines:  91.43% ( 32/ 35)
\AppBundle\Repository::AppBundle\Repository\ParamRepository
  Methods: 100.00% ( 1/ 1)   Lines: 100.00% (  8/  8)
\AppBundle\Repository::AppBundle\Repository\TeamRepository
  Methods: 100.00% ( 5/ 5)   Lines: 100.00% ( 43/ 43)
\AppBundle\Repository::AppBundle\Repository\UnitRepository
  Methods:  66.67% ( 6/ 9)   Lines:  57.41% ( 62/108)
\AppBundle\Utils::AppBundle\Utils\DateUtils
  Methods: 100.00% ( 1/ 1)   Lines: 100.00% ( 18/ 18)

PHPCov:

Code Coverage Report:        
  2019-02-26 13:18:54        

 Summary:                    
  Classes: 43.84% (32/73)    
  Methods: 76.60% (396/517)  
  Lines:   83.55% (2478/2966)

\AppBundle\Command::AppBundle\Command\AgentReinitPwdCommand
  Methods:  80.00% ( 4/ 5)   Lines:  95.95% ( 71/ 74)
\AppBundle\Command::AppBundle\Command\DatabaseVersionCommand
  Methods:  66.67% ( 2/ 3)   Lines:  43.75% (  7/ 16)
\AppBundle\Command::AppBundle\Command\HarpeLeaderUnitCommand
  Methods:  66.67% ( 2/ 3)   Lines:  22.92% ( 11/ 48)
\AppBundle\Command::AppBundle\Command\HarpePersonneFullCommand
  Methods:  25.00% ( 2/ 8)   Lines:  28.21% ( 11/ 39)
\AppBundle\Command::AppBundle\Command\HarpeStructureFullCommand
  Methods:  25.00% ( 2/ 8)   Lines:  18.03% ( 11/ 61)
\AppBundle\Command::AppBundle\Command\ImportAffaireCommand
  Methods: 100.00% ( 4/ 4)   Lines: 100.00% ( 34/ 34)
\AppBundle\Command::AppBundle\Command\ImportCommand
  Methods:  60.00% ( 3/ 5)   Lines:  57.14% ( 28/ 49)
\AppBundle\Controller::AppBundle\Controller\AffaireController
  Methods: 100.00% ( 7/ 7)   Lines: 100.00% ( 56/ 56)
\AppBundle\Controller::AppBundle\Controller\AgentController
  Methods:  71.43% ( 5/ 7)   Lines:  96.20% ( 76/ 79)
\AppBundle\Controller::AppBundle\Controller\AttributController
  Methods:  83.33% ( 5/ 6)   Lines:  80.77% ( 42/ 52)
\AppBundle\Controller::AppBundle\Controller\ContractController
  Methods:  85.71% ( 6/ 7)   Lines:  97.06% ( 66/ 68)
\AppBundle\Controller::AppBundle\Controller\DashboardController
  Methods:  77.78% ( 7/ 9)   Lines:  99.19% (244/246)
\AppBundle\Controller::AppBundle\Controller\ImportController
  Methods:   0.00% ( 0/ 3)   Lines:  21.74% ( 10/ 46)
\AppBundle\Controller::AppBundle\Controller\IndicatorController
  Methods: 100.00% ( 7/ 7)   Lines: 100.00% ( 61/ 61)
\AppBundle\Controller::AppBundle\Controller\TeamController
  Methods: 100.00% ( 8/ 8)   Lines: 100.00% ( 85/ 85)
\AppBundle\Controller::AppBundle\Controller\UnitController
  Methods: 100.00% ( 8/ 8)   Lines: 100.00% ( 88/ 88)
\AppBundle\Entity::AppBundle\Entity\Activity
  Methods:  35.71% ( 5/14)   Lines:  50.00% ( 13/ 26)
\AppBundle\Entity::AppBundle\Entity\Affaire
  Methods:  76.47% (13/17)   Lines:  84.21% ( 32/ 38)
\AppBundle\Entity::AppBundle\Entity\Agent
  Methods:  93.33% (42/45)   Lines:  97.27% (178/183)
\AppBundle\Entity::AppBundle\Entity\AgentTeam
  Methods:  75.00% ( 3/ 4)   Lines:  87.50% (  7/  8)
\AppBundle\Entity::AppBundle\Entity\AgentUnit
  Methods:  75.00% ( 3/ 4)   Lines:  87.50% (  7/  8)
\AppBundle\Entity::AppBundle\Entity\Attribut
  Methods:  82.61% (19/23)   Lines:  85.11% ( 40/ 47)
\AppBundle\Entity::AppBundle\Entity\Contract
  Methods:  65.71% (23/35)   Lines:  76.19% ( 64/ 84)
\AppBundle\Entity::AppBundle\Entity\Import
  Methods:  55.56% (10/18)   Lines:  66.67% ( 20/ 30)
\AppBundle\Entity::AppBundle\Entity\Indicator
  Methods:  84.62% (11/13)   Lines:  82.14% ( 23/ 28)
\AppBundle\Entity::AppBundle\Entity\LeaderTeam
  Methods:  75.00% ( 3/ 4)   Lines:  87.50% (  7/  8)
\AppBundle\Entity::AppBundle\Entity\LeaderUnit
  Methods:  75.00% ( 3/ 4)   Lines:  87.50% (  7/  8)
\AppBundle\Entity::AppBundle\Entity\Team
  Methods:  88.46% (23/26)   Lines:  82.09% ( 55/ 67)
\AppBundle\Entity::AppBundle\Entity\Unit
  Methods:  83.33% (20/24)   Lines:  84.00% ( 42/ 50)
\AppBundle\Form::AppBundle\Form\AddAgentToTeamType
  Methods: 100.00% ( 3/ 3)   Lines: 100.00% (  6/  6)
\AppBundle\Form::AppBundle\Form\AffaireEditType
  Methods: 100.00% ( 1/ 1)   Lines: 100.00% (  3/  3)
\AppBundle\Form::AppBundle\Form\AffaireType
  Methods: 100.00% ( 3/ 3)   Lines: 100.00% (  9/  9)
\AppBundle\Form::AppBundle\Form\AgentType
  Methods: 100.00% ( 4/ 4)   Lines: 100.00% ( 16/ 16)
\AppBundle\Form::AppBundle\Form\AttributEditInIndicatorType
  Methods: 100.00% ( 3/ 3)   Lines: 100.00% (  7/  7)
\AppBundle\Form::AppBundle\Form\AttributInIndicatorType
  Methods: 100.00% ( 3/ 3)   Lines: 100.00% (  9/  9)
\AppBundle\Form::AppBundle\Form\AttributType
  Methods: 100.00% ( 3/ 3)   Lines: 100.00% ( 12/ 12)
\AppBundle\Form::AppBundle\Form\ContractEditType
  Methods: 100.00% ( 1/ 1)   Lines: 100.00% (  5/  5)
\AppBundle\Form::AppBundle\Form\ContractType
  Methods: 100.00% ( 7/ 7)   Lines: 100.00% ( 18/ 18)
\AppBundle\Form::AppBundle\Form\DashboardInstantType
  Methods: 100.00% ( 3/ 3)   Lines: 100.00% ( 37/ 37)
\AppBundle\Form::AppBundle\Form\DashboardType
  Methods: 100.00% ( 3/ 3)   Lines: 100.00% ( 41/ 41)
\AppBundle\Form::AppBundle\Form\ImportAffaireType
  Methods: 100.00% ( 3/ 3)   Lines: 100.00% (  9/  9)
\AppBundle\Form::AppBundle\Form\IndicateurEditType
  Methods: 100.00% ( 1/ 1)   Lines: 100.00% (  7/  7)
\AppBundle\Form::AppBundle\Form\IndicateurType
  Methods: 100.00% ( 3/ 3)   Lines: 100.00% ( 11/ 11)
\AppBundle\Form::AppBundle\Form\TeamEditType
  Methods: 100.00% ( 1/ 1)   Lines: 100.00% (  4/  4)
\AppBundle\Form::AppBundle\Form\TeamType
  Methods: 100.00% ( 7/ 7)   Lines: 100.00% ( 21/ 21)
\AppBundle\Form::AppBundle\Form\UniteEditType
  Methods: 100.00% ( 1/ 1)   Lines: 100.00% (  4/  4)
\AppBundle\Form::AppBundle\Form\UniteType
  Methods: 100.00% ( 5/ 5)   Lines: 100.00% ( 12/ 12)
\AppBundle\Form\EventListener::AppBundle\Form\EventListener\AgentListFieldSubscriber
  Methods: 100.00% ( 5/ 5)   Lines: 100.00% ( 11/ 11)
\AppBundle\Form\EventListener::AppBundle\Form\EventListener\ParentAttributeListFieldSubscriber
  Methods:  80.00% ( 4/ 5)   Lines:  90.00% (  9/ 10)
\AppBundle\Form\EventListener::AppBundle\Form\EventListener\RoleListFieldSubscriber
  Methods: 100.00% ( 3/ 3)   Lines: 100.00% ( 13/ 13)
\AppBundle\Manager::AppBundle\Manager\HarpePersonneManager
  Methods:  87.50% (14/16)   Lines:  91.37% (127/139)
\AppBundle\Manager::AppBundle\Manager\TeamManager
  Methods:  75.00% ( 6/ 8)   Lines:  78.57% ( 22/ 28)
\AppBundle\Manager::AppBundle\Manager\UnitManager
  Methods:  66.67% ( 6/ 9)   Lines:  68.75% ( 22/ 32)
\AppBundle\Menu::AppBundle\Menu\MenuBuilder
  Methods:  50.00% ( 1/ 2)   Lines:  97.30% ( 36/ 37)
\AppBundle\Model::AppBundle\Model\AgentAssociation
  Methods: 100.00% ( 6/ 6)   Lines: 100.00% ( 13/ 13)
\AppBundle\Repository::AppBundle\Repository\ActivityRepository
  Methods: 100.00% ( 8/ 8)   Lines: 100.00% ( 81/ 81)
\AppBundle\Repository::AppBundle\Repository\AffaireRepository
  Methods: 100.00% ( 3/ 3)   Lines: 100.00% ( 26/ 26)
\AppBundle\Repository::AppBundle\Repository\AgentRepository
  Methods: 100.00% ( 8/ 8)   Lines: 100.00% ( 89/ 89)
\AppBundle\Repository::AppBundle\Repository\AttributRepository
  Methods:  80.00% ( 4/ 5)   Lines:  96.72% ( 59/ 61)
\AppBundle\Repository::AppBundle\Repository\ContractRepository
  Methods:  71.43% ( 5/ 7)   Lines:  82.50% ( 99/120)
\AppBundle\Repository::AppBundle\Repository\ImportRepository
  Methods:  25.00% ( 1/ 4)   Lines:  32.43% ( 12/ 37)
\AppBundle\Repository::AppBundle\Repository\IndicatorRepository
  Methods:  80.00% ( 4/ 5)   Lines:  91.43% ( 32/ 35)
\AppBundle\Repository::AppBundle\Repository\ParamRepository
  Methods: 100.00% ( 1/ 1)   Lines: 100.00% (  8/  8)
\AppBundle\Repository::AppBundle\Repository\TeamRepository
  Methods: 100.00% ( 5/ 5)   Lines: 100.00% ( 43/ 43)
\AppBundle\Repository::AppBundle\Repository\UnitRepository
  Methods:  66.67% ( 6/ 9)   Lines:  57.41% ( 62/108)
\AppBundle\Utils::AppBundle\Utils\DateUtils
  Methods: 100.00% ( 1/ 1)   Lines: 100.00% ( 18/ 18)
sebastianbergmann commented 5 years ago

The text report is useless, IMO, and there is no point to have any meaningful discussion about data accuracy based on it.

thewholelifetolearn commented 5 years ago

Some application use this output to display the coverage percentage (for example badges in GitLab). Plus, this output summarizes well the information.
In any case, if the output is not reliable, the option shouldn't be available.

stale[bot] commented 5 years ago

This issue has been automatically marked as stale because it has not had activity within the last 60 days. It will be closed after 7 days if no further activity occurs. Thank you for your contributions.

thewholelifetolearn commented 5 years ago

This issue should not be closed! It is important to rely on such statistics to measure code coverage. It is even more important as both tools rely on PHP_CodeCoverage to gather the information and that both are maintained by the same organization.
As I said before, all outputs should give the same result and be reliable.

sebastianbergmann commented 5 years ago

Thank you for your report.

Please provide a minimal, self-contained, reproducing test case that shows the problem you are reporting.

Without such a minimal, self-contained, reproducing test case I will not be able to investigate this issue.

thewholelifetolearn commented 5 years ago

I'm at the moment in holidays. I will try to submit a test case before end of May.

stale[bot] commented 5 years ago

This issue has been automatically marked as stale because it has not had activity within the last 60 days. It will be closed after 7 days if no further activity occurs. Thank you for your contributions.

thewholelifetolearn commented 5 years ago

I'm sorry i didn't propose a test case yet. I will propose one this month. I should have more spare time to do that.

sebastianbergmann commented 4 years ago

No feedback, closing.