search

mathworks / matlab-circleci-orb

Continuous Integration with MATLAB on CircleCI
https://circleci.com/orbs/registry/orb/mathworks/matlab
Other
5 stars 5 forks source link

circle ci run-tests options 'use-parallel' and 'test-results-html' do not work #57

Open adastar opened 7 months ago

adastar commented 7 months ago

I have the circle ci yml file:

version: 2.1 orbs: matlab: mathworks/matlab@dev:v1-rc jobs: build:

environment:

#   MATHWORKS_RMC_BASE_URL: https://mw-ci-static-dev.s3.amazonaws.com/run-matlab-command/v2
#   MATHWORKS_LICENSING_ENDPOINT: stage

machine:
  image: ubuntu-2204:2023.10.1
steps:
  - checkout
  - matlab/install:
      release: R2023b
  - matlab/run-command:
      command: magic(3)
  - matlab/run-tests:
      source-folder: code
      use-parallel: Yes
      test-results-html: results.html
  - matlab/run-build:
      tasks: clean mtest

The build finished without error. But in the test log, I do not see it starts parallel pool, I also do not see the results.html is generated. There is no log shows the option does not work either.

davidbuzinski commented 7 months ago

Hi @adastar,

It looks like this issue was created in response to the following pipeline run: https://app.circleci.com/pipelines/github/adastar/ci-sample-repo-bash/6/workflows/a6e09310-fc1e-4db7-8351-44084bde627c/jobs/5

If Parallel_Computing_Toolbox is not installed, the tests will run serially. Here is the relevant warning in the log that mentions this:

Warning: Unable to find a runInParallel method on matlab.unittest.TestRunner. Running tests in serial instead.

It is near the top and might not be easily visible. We will consider ways to make this more apparent.

In addition, a test report was generated. It is on the filesystem of the runner. Here is the relevant output in the log that mentions this:

Generating test report. Please wait. Preparing content for the test report. Adding content to the test report. Writing test report to file. Test report has been saved to: /home/circleci/project/results.html/index.html

If you would like to view test results in the job logs of your browser, you have to save them as job artifact. We will consider ways to document this better in the future.

You can add this to your job list after matlab/run-tests in order to store test results that are output in a folder results/index.html

      - store_test_results: 
          path: results/test-results.xml

Thanks, David