[Question] Multiple projects in a workspace

[tabuchid]

We have a single repo that has multiple projects under a single workspace because they share dependancies. We handle testing with a shared scheme that builds the test targets for each project. Is there a way to setup slather to handle this scenario?

[tabuchid]

Any thoughts on this?

[marklarr]

You might have to play around with these two arguments

 --build-directory, -b BUILD_DIRECTORY The directory where gcno files will be written to. Defaults to derived data.
 --source-directory SOURCE_DIRECTORY The directory where your source files are located.

but it should be able to work. All slather really does is go look in derived data (build-directory) for gcno files that match those in your source-directory. Let me know if you can figure it out!

[larslockefeer]

I'm having a similar requirement: our project is set up as a Core framework target containing all model code and a UI target using that framework. Both targets have their own test bundle and both test bundles are executed when xcodebuild test is run.

Manually, I have verified that the coverage data generated by Xcode (I'm using the work-in-progress profdata support branch) contains all information required. However, in order to extract it, I need to run the xcrun llvm-cov command twice, once with the path to the app binary as a parameter and once with the path to the framework binary:

xcrun llvm-cov report -instr-profile Coverage.profdata MyApp.app/MyApp
xcrun llvm-cov report -instr-profile Coverage.profdata MyApp.app/Frameworks/MyAppCore.framework/MyAppCore

It would be great if there were a configuration option to supply these paths when running slather, rather than deriving them automatically, as is currently done in binary_file in project.rb. Of course, the current behaviour could be used as fallback.

Is this a use-case that you are interested in supporting? Or am I missing an already existing option?

[TeresaP]

Is there a good solution for this yet? Our team uses xcworkspace files with several projects underneath and we'd like all the code coverage rolled up.

Edit:

I finally figured it out by reading through pull requests. I wish someone would document this use case on the main wiki.

Here's the solution in case anyone else runs into the problem.

--binary-basename - maps to the name of your project (and Slather then finds the matching binary inside of your Intermediates///Products//AppName.app/Frameworks/Project.framework/ folder) --output-directory - change this to wherever you want your output to go --jenkins** - I don't know if this does anything, but we're running it on a Jenkins job

# Generate xml and html for each xcproj
for xcproj in $(find "${WORKSPACE}" -path "*.xcodeproj"); do
   # Get the name of the project
   base_file_name=$(basename "${xcproj}")
   base_file_name=${base_file_name%.*}

   # Generate reports
   slather coverage --jenkins --input-format profdata --scheme "${SCHEMA}" --binary-basename "${base_file_name}" --html --ignore "../**/*/Xcode*" --output-directory "${WORKSPACE}/CodeCoverage/html_results/${base_file_name}" "${xcproj}"
   slather coverage --jenkins --input-format profdata --scheme "${SCHEMA}" --binary-basename "${base_file_name}" --cobertura-xml --ignore "../**/*/Xcode*" --output-directory "${WORKSPACE}/CodeCoverage/cobertura_xml_results/${base_file_name}" "${xcproj}"
done

This creates several .xml/html files. To combine the XML files, you can use https://gist.github.com/TeresaP/a55d78b67636c5fd8464. I didn't have time to heavily modify the script, and it chokes on feeding it a lot of files, so I just had it merge each file in individually. YMMV.

Syntax:

cd "${WORKSPACE}/CodeCoverage/cobertura_xml_results/"

xml_files=$(find . -maxdepth 2 -name "*.xml")
xml_files_array=(${xml_files})
array_length=${#xml_files_array[@]}

for (( i=0; i<array_length; i++ ))
do
  if [ ${i} == 0 ]; then
  # do nothing
      :
  elif [ ${i} == 1 ]; then
    # Must be a relative path to the xml
    python "${WORKSPACE}/tools/jenkins/merge-xml-coverage.py" "${xml_files_array[0]}" "${xml_files_array[i]}"
  else
    python "${WORKSPACE}/tools/jenkins/merge-xml-coverage.py" "${xml_files_array[i]}" "coverage-merged.xml"
  fi
done

[bootstraponline]

Is there a reason why the slather gem doesn't support this out of the box? Multiple projects in a workspace is a common use case.

[ksuther]

193 and #188 should make this easier to do with a single invocation of slather.

[bootstraponline]

I figured out how to collect coverage numbers across projects without using xml. The API to do this is painful.

    # [project, scheme]
    targets = [
        %w[AAA BBB],
        %w[CCC],
        %w[DDD],
    ]

    total_coverage = []
    schemes = []
    project = nil

    targets.each do |xcode_project, scheme|
      scheme = xcode_project unless scheme # default scheme to project
      schemes << scheme

      xcodeproj_path_to_open = "../#{xcode_project}/#{xcode_project}.xcodeproj"
      project = Slather::Project.open(xcodeproj_path_to_open)
      project.source_files = ["#{xcode_project}/#{scheme}/**/*.swift"]
      project.scheme = scheme
      project.binary_basename = [scheme]
      project.workspace = 'ZZZ.xcworkspace'

      %i[cobertura_xml html].each do |format|
        out_dir = File.join(Dir.pwd, "coverage_#{format}", scheme)
        FileUtils.mkdir_p File.dirname out_dir
        project.output_directory = out_dir
        project.coverage_service = format
        project.configure
        project.post
      end

      total_coverage.concat(project.send(:coverage_files))
    end

    out_dir_name = 'coverage_total'
    out_dir = File.join(Dir.pwd, out_dir_name)
    FileUtils.mkdir_p File.dirname out_dir
    project.output_directory = out_dir

    # Override name so the total report is titled correctly.
    project.xcodeproj = "Total Coverage (#{schemes.join(', ')})"
    project.create_html_reports(total_coverage)
    project.generate_reports(project.instance_variable_get(:@docs))
    puts "open ./fastlane/#{out_dir_name}/index.html"
  end

[sebromero]

I've got a related problem: My pod project contains tests for iOS as well as OsX. Therefore the tests have to run in two separate passes. This means that I not only have to create and merge the two XML coverage results, but neither can I use slather's built-in Coveralls upload mechanism, since the merge has to be done before any coverage file is uploaded. Can anyone think of a more elegant solution to this? Does #188 fix this? I'm getting an error warning: 4 functions have mismatched data. because the .profdata is probably overwritten after each test run and the targets are built for different platforms.

[bootstraponline]

This means that I not only have to create and merge the two XML coverage results, but neither can I use slather's built-in Coveralls upload mechanism, since the merge has to be done before any coverage file is uploaded.

If you use the API to merge the results (like I did above) then you should be able to invoke the upload method directly. Unfortunately this workflow isn't supported via the CLI.

[TeresaP]

Is anyone considering adding this one-line functionality? I thought with the workspace flag, we'd be able to get the results in a single pass instead of iterating over each project. Even if we could list all of the projects at the end in a space delimited format, it would be better than the current method.

[antranapp]

Is there any official tutorials for this issue? I can not find any. I have a main app project and 2 frameworks project in the same worksapce. I want to merge all test of the main app and the tests of the 2 frameworks together to display in jenkins but don't know how to archive it.

[serges147]

That is easy. Just make additional "All_Unit_Tests" shared scheme in your workspace, append there as dependencies all your test targets, and then build & test this scheme. Exactly what we are doing in our workspace (with multiple projects and test targets there).

[NayaraAlves2912]

@serges147 Well, I also have two projects inside a workspace, but it did not work to create the test dependencies. It only takes cover from the test classes, not the project class, can anyone help me?