-l --testlevel not working for sfdx force:apex:test:run

[pogilvieCB]

Summary

this command -> sfdx force:apex:test:run -l RunLocalTests -w 20 -u installTesting gives this error ->ERROR running force:apex:test:run: Always provide a classes, suites, tests, or testLevel property.

scripts are breaking.

Steps To Reproduce:

sfdx --version
sfdx-cli/7.97.0 darwin-x64 node-v15.14.0

sfdx

Repository to reproduce: dreamhouse-lwc

NOTE: If your issue is not reproducable by dreamhouse-lwc, i.e. requires specific metadata or files, we require a link to a simple Salesforce project repository with a script to setup a scratch org that reproduces your problem.

1. This is step 1.
2. This is step 2. All steps should start with '1.'

Expected result

Should run localtests in specified org.

Actual result

Errror

Additional information

SFDX CLI Version(to find the version of the CLI engine run sfdx --version):

sfdx --version
sfdx-cli/7.97.0 darwin-x64 node-v15.14.0

SFDX plugin Version(to find the version of the CLI plugin run sfdx plugins --core)

peter: (pogilvie/SFDC1-7881-signature):~/projects/eng-salesforce/packages/SecureKey2->  sfdx plugins --core
@copado/copado-cli 1.1.0
@oclif/plugin-autocomplete 0.3.0 (core)
@oclif/plugin-commands 1.3.0 (core)
@oclif/plugin-help 3.2.2 (core)
@oclif/plugin-not-found 1.2.4 (core)
@oclif/plugin-plugins 1.10.0 (core)
@oclif/plugin-update 1.4.0-3 (core)
@oclif/plugin-warn-if-update-available 1.7.0 (core)
@oclif/plugin-which 1.0.3 (core)
@pogilvie/sfdx 0.0.6 (link) /Users/peter/Projects/personal/sfdx
@salesforce/sfdx-trust 3.6.0 (core)
alias 1.1.9 (core)
auth 1.5.1 (core)
case 0.0.0 (link) /Users/peter/Projects/coinbase/case
config 1.2.8 (core)
generator 1.1.5 (core)
salesforcedx 51.8.0 (core)
├─ data 0.4.3 (core)
├─ org 1.6.2 (core)
├─ apex 0.1.21 (core)
├─ custom-metadata 1.0.12 (core)
├─ schema 1.0.5 (core)
├─ limits 1.0.5 (core)
├─ templates 51.3.0 (core)
├─ salesforce-alm 51.6.11 (core)
├─ user 1.2.8 (core)
└─ @salesforce/sfdx-plugin-lwc-test 0.1.7 (core)
sfdx-cli 7.97.0 (core)
sfdx-flowdoc-plugin 0.5.1
sfdx-sms 0.0.1 (link) /Users/peter/Projects/sfdx-sms
telemetry 1.1.2 (core)
**OS and version**:

[github-actions[bot]]

Thank you for filing this issue. We appreciate your feedback and will review the issue as soon as possible. Remember, however, that GitHub isn't a mechanism for receiving support under any agreement or SLA. If you require immediate assistance, contact Salesforce Customer Support.

[pogilvieCB]

Well, the issue was at that I had only installed a managed package in a scratch org (there were not any local tests in the org). If I run

sfdx force:apex:test:run -l RunAllTestsInOrg -w 20 -u installTesting it works. The -w flag only works now if the -y flag is also set. This caught me by surprise.

[AnanyaJha]

Hi @pogilvieCB thanks for reporting this! Can you share the version of the plugins you're currently using? Try running the command sfdx plugins --core and share the output. Do you mind sharing a bit more about what you mean by the -w flag is no longer working for you, are you finding that the command is exiting early or not respecting the time limit that is set?

[pogilvieCB]

@AnanyaJha please see the output of sfdx plugins --core already at the top of this issue.

so I just re-ran this again on the same sandbox and found

-l RunLocalTests was working (no error from the cli this time) -w 20 blocked the run but ...

It didn't print the result w/o polling.

`peter: (master):~/projects/eng-salesforce-> sfdx force:apex:test:run -l RunLocalTests -w 20 -u peter

Run "sfdx force:apex:test:report -i 7078A00001oSefr -u peter.ogilvie@coinbase.com.peter" to retrieve test results peter: (master):~/projects/eng-salesforce-> sfdx force:apex:test:report -i 7078A00001oSefr -u peter.ogilvie@coinbase.com.peter `

[pogilvieCB]

@AnanyaJha if I use the -y (synchronous flag) I'll get the correct result. This is new semantics. My scripts expect that when the -w flag is specified the test blocks for the specified time or the test complete AND prints out the test results.

[AnanyaJha]

Thanks for the info @pogilvieCB, we'll work on reproducing this issue and will post back with an update.

[pogilvieCB]

Just one more clarification. -l seems to work working fine now

[AnanyaJha]

Hi @pogilvieCB to confirm, the issue you're seeing is that when the --synchronous flag is not specified, you're seeing the command wait for the results and then print Run "sfdx force:apex:test:report -i <testrunid> -u <yourdevhub>" to retrieve test results instead of the results directly? But when the --synchronous flag is used, you are seeing the results printed as expected?

[pogilvieCB]

Yes, the current issue is: unless I specify -y (--synchronous) the -w flag waits until the test is complete but does not display the test result. I'm prompted to run report to get the results..

Expected result: results are printed after the command completes when the -w flag is specified. (and it does not timeout)

[uip-robot-zz]

This issue has been linked to a new work item: W-9346875

[randi274]

@pogilvieCB can you also try adding the --resultformat you're looking for (human, tap, junit, json), or the -r? So: sfdx force:apex:test:run -l RunAllTestsInOrg -w 20 -u installTesting -r human

You can also add --json if you prefer that format. This came up in another issue recently. It seems like it might just be a gap in documentation, but seems to be working as designed otherwise.

[randi274]

@pogilvieCB we've done some work on our side, so this should be fixed now. Closing this, but let us know if you run into any additional issues.

[fransf-wtax]

Still getting this error, whether I specify -l RunLocalTests or not:

sf apex run test -l RunLocalTests
Error (1): Always provide a classes, suites, tests, or testLevel property.

Output of sf plugins --core:

@dxatscale/sfpowerscripts 25.5.3
├─ @oclif/plugin-help 5.2.17
└─ @oclif/plugin-commands 3.3.1
@oclif/plugin-autocomplete 3.0.16 (core)
@oclif/plugin-not-found 3.1.7 (core)
@oclif/plugin-plugins 5.0.18 (core)
@oclif/plugin-search 1.0.23 (core)
@oclif/plugin-update 4.2.9 (core)
@oclif/plugin-version 2.0.17 (core)
@oclif/plugin-warn-if-update-available 3.0.16 (core)
@oclif/plugin-which 3.1.8 (core)
@salesforce/cli 2.40.7 (core)
apex 3.1.11 (core)
auth 3.6.5 (core)
data 3.3.2 (core)
deploy-retrieve 3.6.8 (core)
info 3.2.4 (core)
limits 3.3.4 (core)
marketplace 1.2.4 (core)
org 4.1.5 (core)
packaging 1.27.0 (1.27.0)
schema 3.3.4 (core)
settings 2.2.3 (core)
signups 2.2.5 (2.2.5)
sobject 1.3.6 (core)
source 3.3.5 (core)
telemetry 3.3.4 (core)
templates 56.2.4 (core)
texei-sfdx-plugin 2.5.0
trust 3.6.7 (core)
user 3.5.4 (core)

Uninstalled JIT Plugins:
community 3.2.6
custom-metadata 3.3.4
dev 2.3.0
devops-center 1.2.9
env 3.0.29
functions 1.23.0
@salesforce/sfdx-plugin-lwc-test 1.2.0
@salesforce/sfdx-scanner 3.24.0

[fransf-wtax]

Well, the issue was at that I had only installed a managed package in a scratch org (there were not any local tests in the org).

OK this seems to have been the issue with me as well.

The error message needs improving. The problem is not that the user didn't "provide a classes, suites, tests, or testLevel property", but rather that there were no tests to run on the org. Not sure why the issue was closed when this doesn't seem to have happened?