Unexpected change of behavior for sfdx force:apex:test:run --synchronous

[froucher]

sfdx force:apex:test:run --synchronous behaviour

There is an unexpected change of the behaviour of sfdx force:apex:test:run --synchronous, because now returns a new message "Specify a test class or test methods when running tests synchronously".

Steps To Reproduce:

$ sfdx --version
sfdx-cli/7.96.0 linux-x64 node-v15.14.0 
$ sfdx force:apex:test:run -l RunLocalTests -y -c -r human
ERROR running force:apex:test:run: Specify a test class or test methods when running tests synchronously

Expected result

For us to run all tests synchronously is important for our continuous integration jobs.

Additional information

SFDX CLI Version(to find the version of the CLI engine run sfdx --version): sfdx-cli/7.96.0

SFDX plugin Version(to find the version of the CLI plugin run sfdx plugins --core)

@oclif/plugin-autocomplete 0.3.0 (core) @oclif/plugin-commands 1.3.0 (core) @oclif/plugin-help 3.2.2 (core) @oclif/plugin-not-found 1.2.4 (core) @oclif/plugin-plugins 1.10.0 (core) @oclif/plugin-update 1.4.0-3 (core) @oclif/plugin-warn-if-update-available 1.7.0 (core) @oclif/plugin-which 1.0.3 (core) @salesforce/sfdx-trust 3.6.0 (core) alias 1.1.9 (core) auth 1.5.1 (core) config 1.2.8 (core) generator 1.1.5 (core) salesforcedx 51.7.1 (core) ├─ data 0.4.3 (core) ├─ apex 0.1.21 (core) ├─ custom-metadata 1.0.11 (core) ├─ limits 1.0.5 (core) ├─ org 1.6.1 (core) ├─ schema 1.0.5 (core) ├─ templates 51.3.0 (core) ├─ user 1.2.8 (core) ├─ @salesforce/sfdx-plugin-lwc-test 0.1.7 (core) └─ salesforce-alm 51.6.10 (core) sfdx-cli 7.96.0 (core) telemetry 1.1.1 (core)

OS and version: docker alpine: Linux cc246b97cbac 4.19.121-linuxkit forcedotcom/cli#1 SMP Thu Jan 21 15:36:34 UTC 2021 x86_64 Linux

[github-actions[bot]]

Thank you for filing this issue. We appreciate your feedback and will review the issue as soon as possible. Remember, however, that GitHub isn't a mechanism for receiving support under any agreement or SLA. If you require immediate assistance, contact Salesforce Customer Support.

[CsfGitHub]

I confirm I'm experiencing the same issue

[AnanyaJha]

Hi @froucher thanks for reporting this! As part of version 51.7 of the salesforcedx plugin, we released new versions of the sfdx force:apex:test:run & sfdx force:apex:test:report commands. More info here https://developer.salesforce.com/blogs/2021/04/clearer-apex-commands.html. The new version fixed many long standing bugs, which changed the behavior in certain scenarios.

The synchronous flag only supports running tests from a single class - this can be specified using the --classnames parameter or by including test methods via the --tests parameter. In the past, running force:apex:test:run -y did not surface any errors, but behind the scenes, the tests were actually running asynchronously since synchronous test runs only support single test classes. With the new version, we’ve surfaced the error so you also know what’s going on. Hope that helps, let me know if you have any other questions!

[froucher]

Many thanks @AnanyaJha for the information, I have an additional question, about if we are going to have this new behaviour definitely, or will be a new version of the plugin that is going to maintain the synchronous behaviour with all tests.

In our side, that means if we make some workaround to make it work, or if we wait to update the cli in the ci environment.

Thanks for the support.

[smaddox-sf]

Hi @froucher - We will be restoring the prior behavior in the CLI for the near-term. As @AnanyaJha pointed out, the synchronous flag only supports running tests from a single class so the synchronous behavior you've been working with might not have been quite what you expected. We will put the CLI behavior back as it was for now and come back to fix this bug at a later date. (When we do that fix, we will add a warning on the command months in advance so you can adjust to the new behavior)

[froucher]

Thanks @smaddox-sf for the information and the support.

[gsbasso]

I'm going to question whether -y was always essentially ignored unless a class was specified. The reason I say this is because in the past some tests were randomly failing (when run asynchronously) and when we added -y these random test failures essentially disappeared.

[AnanyaJha]

Hi everyone! Thank you all for the patience as we worked on getting this fix out, it has now been released with the newest version of the Apex plugin v0.2.0. Run sfdx plugins:install apex to update to the new version - it should be bundled with the new version of the cli in 2 weeks. If you find any other issues, please don't hesitate to reach out or open a new issue!

[rajdanala]

@froucher Are you able to run synchronous tests. Did you find any workaround?