Coverage.profdata not found - Xcode 14.3 - Xcode 15.0

salvatoreboemia commented 1 year ago

[x] Updated fastlane/slather to the latest version
[x] I read the Contribution Guidelines
[x] I read docs.fastlane.tools
[x] I searched for existing GitHub issues

Issue Description

I upadated on GitHub my pipeline in order to use latest version of macOS and Xcode (13 and 14.3), but the slather scan failed (only with new macOS and Xcode version)

Command executed

scan(
      workspace: MyWorkspace,
      clean: true,
      device: "iPhone 8",
      # devices: ["iPhone 8", "iPad Air"],
      #output_files: "TEST-report.xml",
      scheme: MyTarget,
      code_coverage: true,
      skip_build: true,
      skip_testing: "ProjectNameTargetAppUITests",
      output_directory: "./DerivedData/test-output/scan",
      derived_data_path: "./DerivedData")

    slather(
      build_directory: "./DerivedData",
      scheme: MyTarget,
      workspace: ProjectWorkspace,
      proj: MyProject,
      output_directory: "./DerivedData/test-output/slatherOutput", 
      cobertura_xml: true,
      verbose: true,
      ignore: ["Pods/*", "fastlane/*", "build/*", "DerivedData/*"])      
  end

Complete output when running fastlane, including the stack trace and command used

Exit status of command 'slather coverage --cobertura-xml --build-directory ./DerivedData --output-directory ./DerivedData/test-output/slatherOutput --ignore Pods/\* --ignore fastlane/\* --ignore build/\* --ignore DerivedData/\* --verbose --scheme ProjectName --workspace ProjectName.xcworkspace ./ProjectName.xcodeproj' was 1 instead of 0.

Slathering...
/usr/local/lib/ruby/gems/3.0.0/gems/slather-2.7.4/lib/slather/project.rb:239:in `profdata_coverage_dir': No coverage directory found. (StandardError)
    from /usr/local/lib/ruby/gems/3.0.0/gems/slather-2.7.4/lib/slather/project.rb:517:in `find_binary_files'
    from /usr/local/lib/ruby/gems/3.0.0/gems/slather-2.7.4/lib/slather/project.rb:336:in `configure'
    from /usr/local/lib/ruby/gems/3.0.0/gems/slather-2.7.4/lib/slather/command/coverage_command.rb:59:in `execute'
    from /usr/local/lib/ruby/gems/3.0.0/gems/clamp-1.3.2/lib/clamp/command.rb:66:in `run'
    from /usr/local/lib/ruby/gems/3.0.0/gems/clamp-1.3.2/lib/clamp/subcommand/execution.rb:18:in `execute'
    from /usr/local/lib/ruby/gems/3.0.0/gems/clamp-1.3.2/lib/clamp/command.rb:66:in `run'
    from /usr/local/lib/ruby/gems/3.0.0/gems/clamp-1.3.2/lib/clamp/command.rb:140:in `run'
    from /usr/local/lib/ruby/gems/3.0.0/gems/slather-2.7.4/bin/slather:17:in `<top (required)>'
    from /usr/local/lib/ruby/gems/3.0.0/bin/slather:25:in `load'
    from /usr/local/lib/ruby/gems/3.0.0/bin/slather:25:in `<main>'
No coverage directory found.

    Are you sure your project is generating coverage? Make sure you enable code coverage in the Test section of your Xcode scheme.
    Did you specify your Xcode scheme? (--scheme or 'scheme' in .slather.yml)
    If you're using a workspace, did you specify it? (--workspace or 'workspace' in .slather.yml)
    If you use a different Xcode configuration, did you specify it? (--configuration or 'configuration' in .slather.yml)

Environment

GitHub Env

Do you have any suggestions in order to fix this issue on GitHub and fastlane?

Thanks a lot

salvatoreboemia commented 1 year ago

News on this issue on Xcode 14.3 and 14.3.1?

thanks

salvatoreboemia commented 8 months ago

News? I tried also on Xcode 15 and GitHub, but I got the same error

salvatoreboemia commented 8 months ago

With Xcode 14.2

Processing coverage file: /Users/runner/work/repo/repo/DerivedData/Build/ProfileData/
C9A-EA9C-4732-A6CD-ED50D/Coverage.profdata

With Xcode 14.3.x or Xcode 15

/usr/local/lib/ruby/gems/3.0.0/gems/slather-2.7.5/lib/slather/project.rb:239:in `
profdata_coverage_dir': No coverage directory found. (StandardError)

No one on this issue? @ksuther @marklarr

thanks

evelynloo commented 6 months ago

@salvatoreboemia I am having the same error, do you manage to solve it?

salvatoreboemia commented 6 months ago

@salvatoreboemia I am having the same error, do you manage to solve it?

Not able to manage... I removed slather step

If you will find a fix please update this post.

Thanks

vadimkrutovlv commented 6 months ago

Before you do anything with your CI configuration please make sure that Xcode generates coverage report locally, in our case we didn't have this file generated, so it's not a surprise that slather did fail.

Please try to check build logs or test report (depending on the Xcode version you're using) if you can find similar error to this one

Showing All Messages
Failed to merge raw profiles in directory /{my project dir}/DerivedData/{my project name}/Build/ProfileData/A98EC493-3AB4-4B1C-B7FC-BC5D77B23EE3 to destination /{my project dir}/DerivedData/{my project name}/Build/ProfileData/A98EC493-3AB4-4B1C-B7FC-BC5D77B23EE3/Coverage.profdata: Aggregation tool '/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/llvm-profdata' failed with exit code 1: warning: /{my project dir}/DerivedData/{my project name}/Build/ProfileData/A98EC493-3AB4-4B1C-B7FC-BC5D77B23EE3/997F01E3-3735-4897-AE00-31EBB0A9E8D3-57885.profraw: invalid instrumentation profile data (file header is corrupt)
error: no profile can be merged

If this is a case then solution described bellow might help you, I really hope it does.

We spent a lot of time debugging this problem, but none of the suggestions we found helped.

We tried to update our Podfile with these settings:

config.build_settings['CLANG_ENABLE_CODE_COVERAGE'] = 'NO'
config.build_settings['swiftc'] = '$(SRCROOT)/SWIFT_EXEC-no-coverage'

In our project Instrument Program Flow & Generate Legacy Test Coverage build settings were already set to NO

We also tried to change Mach-O type for our cocoa pods dependencies to static library and many other things.

it appears that one of our dependencies was a binary (xcframework), most probably distributed via older version of the XCode, after updating it coverage was back.

AlexCookeTrader commented 5 months ago

I'm also facing this issue with Xcode 15.0 and 15.1. Would love to hear of a solution to this issue..

jarrodlombardo-EventBase commented 5 months ago

The solution is to make sure ALL of your upstream dependencies have been built with XCode 14.3 or newer. If ANY of your deps is not built with XCode 14.3 or newer, the .profraw file will be corrupt, so lldb-profdata will be unable to create a .prodata file for slather to parse. https://developer.apple.com/forums/thread/734611?answerId=775545022#775545022

edward-sentongo commented 1 month ago

I am currently facing the same issue. Any update on when this could get resolved?

jarrodlombardo-EventBase commented 1 month ago

@edward-sentongo Are you 100% sure it's not https://github.com/SlatherOrg/slather/issues/541#issuecomment-1883864542 ?

Does XCode show coverage % for all of the files you care about?

If XCode is not showing the coverage, you have to fix that first and it's unrelated to Slather.
If XCode is showing the coverage, but slather can't find it the files, you might have #545 or #557

edward-sentongo commented 1 month ago

@jarrodlombardo-EventBase Xcode does show code coverage percentages for those files. I'll look into other solutions you suggested. Thanks

SlatherOrg / slather