search

SlatherOrg / slather

Generate test coverage reports for Xcode projects & hook it into CI.
MIT License
1.55k stars 233 forks source link

Slather can't find Coverage.profdata if project is setup to use Xcode test plan #556

Closed adincebic closed 2 months ago

adincebic commented 2 months ago

Steps to reproduce

  1. Make sure that your project uses test plans in Xcode
  2. Run the tests
  3. Run slather either via fastlane or directly

Expected

Slather should do its thing and produce the desired output.

Actual

Slather displays an error

`Error Slathering... /Users/user/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/slather-2.8.0/lib/slather/project.rb:588:infind_binary_files': No product binary found in derivedData/Build/ProfileData/11AFE767-46B6-4B1E-ABCA-2702D08F1E98. (StandardError) from /Users/user/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/slather-2.8.0/lib/slather/project.rb:462:in configure_binary_file' from /Users/user/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/slather-2.8.0/lib/slather/project.rb:342:inconfigure' from /Users/user/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/slather-2.8.0/lib/slather/command/coverage_command.rb:62:in execute' from /Users/user/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/clamp-1.3.2/lib/clamp/command.rb:66:inrun' from /Users/user/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/clamp-1.3.2/lib/clamp/subcommand/execution.rb:18:in execute' from /Users/user/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/clamp-1.3.2/lib/clamp/command.rb:66:inrun' from /Users/user/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/clamp-1.3.2/lib/clamp/command.rb:140:in run' from /Users/user/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/slather-2.8.0/bin/slather:17:in<top (required)>' from /Users/user/.rbenv/versions/3.3.0/bin/slather:25:in load' from /Users/user/.rbenv/versions/3.3.0/bin/slather:25:in

' No product binary found in derivedData/Build/ProfileData/11AFE767-46B6-4B1E-ABCA-2702D08F1E98.

Are you sure your project is generating coverage? Make sure you enable code coverage in the Test section of your Xcode scheme.
Did you specify your Xcode scheme? (--scheme or 'scheme' in .slather.yml)
If you're using a workspace, did you specify it? (--workspace or 'workspace' in .slather.yml)
If you use a different Xcode configuration, did you specify it? (--configuration or 'configuration' in .slather.yml)

from /Users/user/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/fastlane-2.219.0/fastlane_core/lib/fastlane_core/ui/ui.rb:17:in `method_missing'
from /Users/user/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/fastlane-2.219.0/fastlane/lib/fastlane/helper/sh_helper.rb:80:in `sh_control_output'
from /Users/user/.rbenv/versions/3.3.0/lib/ruby/3.3.0/forwardable.rb:240:in `sh'
from /Users/user/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/fastlane-2.219.0/fastlane/lib/fastlane/actions/slather.rb:52:in `run'
from /Users/user/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/fastlane-2.219.0/fastlane/lib/fastlane/runner.rb:263:in `block (2 levels) in execute_action'
from /Users/user/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/fastlane-2.219.0/fastlane/lib/fastlane/actions/actions_helper.rb:69:in `execute_action'
from /Users/user/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/fastlane-2.219.0/fastlane/lib/fastlane/runner.rb:255:in `block in execute_action'
from /Users/user/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/fastlane-2.219.0/fastlane/lib/fastlane/runner.rb:229:in `chdir'
from /Users/user/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/fastlane-2.219.0/fastlane/lib/fastlane/runner.rb:229:in `execute_action'
from /Users/user/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/fastlane-2.219.0/fastlane/lib/fastlane/runner.rb:157:in `trigger_action_by_name'
from /Users/user/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/fastlane-2.219.0/fastlane/lib/fastlane/fast_file.rb:159:in `method_missing'
from fastfiles/Fastfile_testing:79:in `block (2 levels) in parsing_binding'
from /Users/user/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/fastlane-2.219.0/fastlane/lib/fastlane/lane.rb:41:in `call'
from /Users/user/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/fastlane-2.219.0/fastlane/lib/fastlane/runner.rb:49:in `block in execute'
from /Users/user/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/fastlane-2.219.0/fastlane/lib/fastlane/runner.rb:45:in `chdir'
from /Users/user/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/fastlane-2.219.0/fastlane/lib/fastlane/runner.rb:45:in `execute'
from /Users/user/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/fastlane-2.219.0/fastlane/lib/fastlane/lane_manager.rb:46:in `cruise_lane'
from /Users/user/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/fastlane-2.219.0/fastlane/lib/fastlane/command_line_handler.rb:34:in `handle'
from /Users/user/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/fastlane-2.219.0/fastlane/lib/fastlane/commands_generator.rb:110:in `block (2 levels) in run'
from /Users/user/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/commander-4.6.0/lib/commander/command.rb:187:in `call'
from /Users/user/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/commander-4.6.0/lib/commander/command.rb:157:in `run'
from /Users/user/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/commander-4.6.0/lib/commander/runner.rb:444:in `run_active_command'
from /Users/user/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/fastlane-2.219.0/fastlane_core/lib/fastlane_core/ui/fastlane_runner.rb:124:in `run!'
from /Users/user/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/commander-4.6.0/lib/commander/delegates.rb:18:in `run!'
from /Users/user/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/fastlane-2.219.0/fastlane/lib/fastlane/commands_generator.rb:354:in `run'
from /Users/user/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/fastlane-2.219.0/fastlane/lib/fastlane/commands_generator.rb:43:in `start'
from /Users/user/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/fastlane-2.219.0/fastlane/lib/fastlane/cli_tools_distributor.rb:123:in `take_off'
from /Users/user/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/fastlane-2.219.0/bin/fastlane:23:in `<top (required)>'
from /Users/user/.rbenv/versions/3.3.0/bin/fastlane:25:in `load'
from /Users/user/.rbenv/versions/3.3.0/bin/fastlane:25:in `<top (required)>'
from /Users/user/.rbenv/versions/3.3.0/lib/ruby/site_ruby/3.3.0/bundler/cli/exec.rb:58:in `load'
from /Users/user/.rbenv/versions/3.3.0/lib/ruby/site_ruby/3.3.0/bundler/cli/exec.rb:58:in `kernel_load'
from /Users/user/.rbenv/versions/3.3.0/lib/ruby/site_ruby/3.3.0/bundler/cli/exec.rb:23:in `run'
from /Users/user/.rbenv/versions/3.3.0/lib/ruby/site_ruby/3.3.0/bundler/cli.rb:451:in `exec'
from /Users/user/.rbenv/versions/3.3.0/lib/ruby/site_ruby/3.3.0/bundler/vendor/thor/lib/thor/command.rb:28:in `run'
from /Users/user/.rbenv/versions/3.3.0/lib/ruby/site_ruby/3.3.0/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command'
from /Users/user/.rbenv/versions/3.3.0/lib/ruby/site_ruby/3.3.0/bundler/vendor/thor/lib/thor.rb:527:in `dispatch'
from /Users/user/.rbenv/versions/3.3.0/lib/ruby/site_ruby/3.3.0/bundler/cli.rb:34:in `dispatch'
from /Users/user/.rbenv/versions/3.3.0/lib/ruby/site_ruby/3.3.0/bundler/vendor/thor/lib/thor/base.rb:584:in `start'
from /Users/user/.rbenv/versions/3.3.0/lib/ruby/site_ruby/3.3.0/bundler/cli.rb:28:in `start'
from /Users/user/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/bundler-2.5.6/exe/bundle:28:in `block in <top (required)>'
from /Users/user/.rbenv/versions/3.3.0/lib/ruby/site_ruby/3.3.0/bundler/friendly_errors.rb:117:in `with_friendly_errors'
from /Users/user/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/bundler-2.5.6/exe/bundle:20:in `<top (required)>'
from /Users/user/.rbenv/versions/3.3.0/bin/bundle:25:in `load'
from /Users/user/.rbenv/versions/3.3.0/bin/bundle:25:in `<main>'


Please note that xcodeproj, workspace and scheme are passed correctly. I verified that it works fine when I don't use Xcode test plans.
jarrodlombardo-EventBase commented 2 months ago

I think this is a dup of https://github.com/SlatherOrg/slather/issues/541 Once I rebuilt all of my upstream dependencies with XCode 15, they all work in Slather, even with Test Plans.

adincebic commented 2 months ago

I already built everything using Xcode 15.3 and it still doesn't work.

jarrodlombardo-EventBase commented 2 months ago

Separately from Slather, can you see Code Coverage in XCode? The Code Coverage in XCode is based on the same profdata file Slather is looking for in DerivedData. Until you fix the issues in the XCode part of the build and test that generate that file, Slather can't do anything.

adincebic commented 2 months ago

Yes I can see it.

jarrodlombardo-EventBase commented 2 months ago
  1. When you build your build, where does the output .app, .xctest, .framework, or .dylib get placed?
  2. Can you please share the output of ls -la for the folder(s) in the answer to 1?
  3. Can you please share the output of ls -la for the derivedData/Build/ProfileData/11AFE767-46B6-4B1E-ABCA-2702D08F1E98 folder?

What I think is happening is A) Something is not getting built correctly and placed in the dirs Slather knows to search or B) Slather is not searching the correct dirs.