pnp / cli-microsoft365

Manage Microsoft 365 and SharePoint Framework projects on any platform
https://aka.ms/cli-m365
MIT License
926 stars 325 forks source link

CI/CD --> 'm365' is not recognized as an internal or external command, operable program or batch file. #3612

Closed martinlingstuyl closed 1 year ago

martinlingstuyl commented 2 years ago

Hi @AkashSatoliya, I created a separate issue for this as the comment trails might overlap otherwise.


Hi @martinlingstuyl,

I am also getting the same error. I am trying to create CD pipeline in Azure DevOps to deploy sppkg to SharePoint. I was getting error "The system cannot find the file specified." when running task to login to Sharepoint.

When I tried to login using m365 cli on one of my VM I was getting the same error:

m365 login --authType password --userName user@contoso.com --password ***
The system cannot find the file specified.

When I ran "npm view @pnp/cli-microsoft365" to check details on m365 CLI, I got below output:

C:\Users\User>npm view @pnp/cli-microsoft365

@pnp/cli-microsoft365@5.6.0 | MIT | deps: 24 | versions: 488
Manage Microsoft 365 and SharePoint Framework projects on any platform
https://pnp.github.io/cli-microsoft365/

keywords: office 365, microsoft 365, sharepoint framework, o365, m365, spfx, sha
repoint online, sharepoint, microsoft teams, microsoft graph, microsoft flow, az
ure active directory, azure ad, azure, microsoft

bin: m365, m365_comp, microsoft365

But When I run "m365 version" I'm getting below error:

C:\Windows\system32>m365 version
'm365' is not recognized as an internal or external command,
operable program or batch file.

Nodejs path(C:\Program Files\nodejs) was already included in PATH variable.

nodejs version: V14.15.0 Operating system: Windows Shell: PowerShell

Originally posted by @AkashSatoliya in https://github.com/pnp/cli-microsoft365/issues/3565#issuecomment-1223752643

martinlingstuyl commented 2 years ago

Hi @akashsatoliya,

Could you show us the relevant parts of your yaml file?

Also: are you using the CLI for Microsoft 365 action?

AkashSatoliya commented 2 years ago

Hi @martinlingstuyl,

Thank you so much for looking into it.

I am using classic pipeline for CD release. I have added tasks mentioned in this document.

Tasks are as follows:

  1. Node.js tool installer, v12.22.1
  2. npm install Microsoft 365 CLI: install -g @pnp/cli-microsoft365
  3. Command line to connect to SPO: m365 login -t password -u $(username) -p $(password)
  4. Command line to Add the solution to SP App Catalog: m365 spo app add -p $(System.DefaultWorkingDirectory)/D:\a\r1\a\_Application_CI\drop\App_Folder1\App_Folder2\sharepoint\solution\Package_Name.sppkg
  5. Command line to Deploy SP App: m365 spo app deploy --name Package_Name.sppkg

Variables defined are :

  1. catalogsite: sites/appcatalog
  2. username
  3. password
  4. tenant

I have tried running CD pipeline using MS hosted Windows and Ubuntu agents. The deployment tasks runs successfully till step 2 i.e. npm install Microsoft 365 CLI. I am getting error for step 3, i.e. Command line to connect to SPO. The error is as below:

========================== Starting Command Output ===========================

[debug]Entering Invoke-VstsTool.

[debug] Arguments: '/D /E:ON /V:OFF /S /C "CALL "D:\a_temp\032fe971-b3c4-4662-b7dc-544dd7b568d1.cmd""'

[debug] FileName: 'C:\Windows\system32\cmd.exe'

[debug] WorkingDirectory: 'D:\a\r1\a'

[command]"C:\Windows\system32\cmd.exe" /D /E:ON /V:OFF /S /C "CALL "D:\a_temp\032fe971-b3c4-4662-b7dc-544dd7b568d1.cmd""

The system cannot find the file specified.

[debug]Exit code: 1

[debug]Leaving Invoke-VstsTool.

[error]Cmd.exe exited with code '1'.

I am not using CLI for Microsoft 365 action. I have configured only steps mentioned in the document.

I tried providing tenant details in login command, similar to o365 cli, like below but didn't work as well. m365 spo login $(tenant)/$(catalogsite) -t password -u $(username) -p $(password) or m365 login $(tenant)/$(catalogsite) -t password -u $(username) -p $(password)

waldekmastykarz commented 2 years ago

The line

install -g @pnp/cli-microsoft365

should be:

npm install -g @pnp/cli-microsoft365

Could you please try it and see if it makes any difference?

AkashSatoliya commented 2 years ago

Hi @waldekmastykarz,

Thank you so much for your suggestion. As suggested, I tried npm install -g @pnp/cli-microsoft365 but it failed while installing Microsoft 365 CLI in step 2 with below error:

[warning]Couldn't find a debug log in the cache or working directory

[error]Error: Npm failed with return code: 1

waldekmastykarz commented 2 years ago

Thank you for trying it and such a quick reply. Is there a way to get more detailed output log of this step that could help us understand what's wrong? Could you also share with us the configuration of this in your pipe? Thank you!

AkashSatoliya commented 2 years ago

Hi @waldekmastykarz,

As mentioned earlier here, the first error I got was The system cannot find the file specified. when running Azure DevOps task to login SharePoint. Then I tried installed M365 CLI on VM and tried to login manually to SharePoint and got same error The system cannot find the file specified..

The configuration steps for CD pipeline are as below:

  1. Node.js tool installer, v12.22.1
  2. npm task with Custom command to install Microsoft 365 CLI: npm install -g @pnp/cli-microsoft365
  3. Command line to connect to SPO: m365 login -t password -u $(username) -p $(password)
  4. Command line to Add the solution to SP App Catalog: m365 spo app add -p $(System.DefaultWorkingDirectory)/D:/a/r1/a/_Application_CI/drop/Folder1/Folder2/sharepoint/solution/Package_Name.sppkg
  5. Command line to Deploy SP App: m365 spo app deploy --name Package_Name.sppkg

Variables defined are :

  1. catalogsite: sites/appcatalog
  2. username
  3. password
  4. tenant

As per your suggestion, I updated the step 2 (npm task to install Microsoft 365 CLI) with command npm install -g @pnp/cli-microsoft365. Now pipeline fails on step 2 with error as mentioned here. Error Logs are as below:

 ##[section]Starting: Connect to SPO  ==============================================================================  Task         : Command line  Description  : Run a command line script using Bash on Linux and macOS and cmd.exe on Windows  Version      : 2.201.1  Author       : Microsoft Corporation  Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/command-line  ==============================================================================  Generating script.  Script contents: shell  m365 login -t password -u username -p ***  ========================== Starting Command Output ===========================  ##[command]"C:\Windows\system32\cmd.exe" /D /E:ON /V:OFF /S /C "CALL "D:\a_temp\068d3114-e622-4a2c-b5d3-fb2012e21a0f.cmd""  The system cannot find the file specified.  ##[error]Cmd.exe exited with code '1'.  ##[section]Finishing: Connect to SPO

Detailed debug logs for step 2 are too long(450+ lines). Is it ok to post entire logs here?

martinlingstuyl commented 2 years ago

Hi @AkashSatoliya,

Okay, rollback to the step where the install completed successfully,

Could you run the following commands in a cmd task on a windows pipeline and see what the output is?

Check if the CLI is in fact installed:

npm list -g --depth 0 

Check the path where global modules are installed and verify if that path is in the PATH variable

npm root -g

echo %PATH%
AkashSatoliya commented 2 years ago

Hi @martinlingstuyl,

Thanks for reply.

Below are the updates for shared command:

  1. Check if the CLI is in fact installed:

    npm list -g --depth 0
    @pnp/cli-microsoft365@5.6.0
  2. Path where global modules are installed

    npm root -g
    C:\npm\prefix\node_modules
  3. Verify if that path is in the PATH variable. Global module path C:\npm\prefix\node_modules is not available in PATH variable. Instead C:\npm\prefix is available in PATH variable.

    echo %PATH%
martinlingstuyl commented 2 years ago

Okay, That's the reason. It cannot find the m365 executable, so it cannot execute its commands.

I find that odd though. You are running on GitHub? Azure DevOps? Your own host? Hosted vm?

The workaround is to add the global npm folder to your path. You can do this using the following statement:

set PATH=%PATH%;C:\npm\prefix\node_modules\

This adds the directory to the path in the current session and should cause m365 to be reachable.

AkashSatoliya commented 2 years ago

Hi @martinlingstuyl

I am running it on Azure DevOps Microsoft hosted agent.

I tried above command but it adds npm path to current task only. When next task is run path is not available in path variable.

I tried adding npm path to system path variable using below PowerShell and setx command but they also saved it to current task only, not to the system. When next task is run value is not available.

-Setx command

setx /M path "%path%;C:\npm\prefix\node_modules
martinlingstuyl commented 2 years ago

Hi @AkashSatoliya,

What's the content of your $PATH? Maybe some node.js folder is in it. In that case you could change the place where the npm install globally installs the packages to something already in the PATH. You can do this using an environmentvariable npm_config_cache, or using the --prefix option

npm install --prefix <path/to/prefix_folder> -g

You could also install the cli package locally (npm install @pnp/cli-microsoft365) and run the executable something like this:

./node_modules/.bin/m365 login
AkashSatoliya commented 2 years ago

@martinlingstuyl,

Thank you for your suggestion.

I have tried running command with PowerShell instead of command tasks.

$PATH = [Environment]::GetEnvironmentVariable("PATH")
$npm_path = "C:\npm\prefix\node_modules"
[Environment]::SetEnvironmentVariable("PATH", $NewPath)
Write-Output $Env:PATH

m365 login -t password -u ${username} -p ${password}

With above script I'm not getting error for m365 command. It attempts to login but fails with below error

Error: network_error: Network request failed. Please check network trace to determine root cause. | Fetch client threw: Error: HTTP status code 400 | Attempted to reach: https://login.microsoftonline.com/organizations/oauth2/v2.0/token

As per this document it is known issue with m365 cli and it has been resolved in m365 cli version 5.5. However I'm using m365 cli v5.6.0 but still getting this error.

martinlingstuyl commented 2 years ago

Hi @AkashSatoliya,

Okay, great that the command is now being executed at least.

Did you try running the login command on your local pc? Do you have the same issues there?

AkashSatoliya commented 2 years ago

Hi @martinlingstuyl,

I tried to run login command on local but there I am still getting 'm365' is not recognized as an internal or external command, operable program or batch file. error. Even though npm module folder path C:\npm\prefix\node_modules is already there in system PATH variable.

AkashSatoliya commented 2 years ago

Hi @martinlingstuyl,

I tried installing version 5.5.0 and latest version using install -g @pnp/cli-microsoft365@latest and run the command but still it fails with same network_error 400

martinlingstuyl commented 2 years ago

I'm not sure, but is it possible that the syntax ${password} is incorrect when you want to reference a pipeline variable in a powershell task? I believe it should be $(password)

AkashSatoliya commented 2 years ago

Hi @martinlingstuyl,

When trying with $(password) it results in variable reference error. PowerShell works with ${password} and $password. When tried with ${password} and $password it tries to login and fails with network error.

martinlingstuyl commented 2 years ago

Ok, so just to be sure: did you try this on your local pc as well? Signin In using username and password? Same problem there?

AkashSatoliya commented 2 years ago

Hi @martinlingstuyl,

I tried to run login command on local but there I am still getting 'm365' is not recognized as an internal or external command, operable program or batch file. error. Even though npm module folder path C:\npm\prefix\node_modules is already there in system PATH variable.

martinlingstuyl commented 2 years ago

Hi @AkashSatoliya,

On your local machine, could you run the following script to check if CLI is installed, if so where, and if that directory is on the path?

Same as before... If the directory is not on the path, add it.

npm list -g --depth 0 

npm root -g

write-host $Env:PATH
martinlingstuyl commented 2 years ago

When trying with $(password) it results in variable reference error.

@AkashSatoliya, reading this again. I think this means that the variable $(password) might be correctly written, but is not available.

Did you read the documentation on setting variables in a pipeline?: https://docs.microsoft.com/en-us/azure/devops/pipelines/process/variables?view=azure-devops&tabs=classic%2Cbatch#set-variables-in-pipeline

I think you need to define your variables first, and then change you script to:

m365 login -t password -u "$(username)" -p "$(password)"
AkashSatoliya commented 2 years ago

Hi @martinlingstuyl ,

Thank you so much for reply.

I've already defined the password variable in variables section of the pipeline. When I tried to run the pipeline with m365 login -t password -u "$(username)" -p "$(password)", it throws below error:

Error: network_error: Network request failed. Please check network trace to determine root cause. | Fetch client threw: Error: HTTP status code 400 | Attempted to reach: https://login.microsoftonline.com/organizations/oauth2/v2.0/token

When tried to manually reach https://login.microsoftonline.com/organizations/oauth2/v2.0/token, I'm getting below error

AADSTS900561: The endpoint only accepts POST, OPTIONS requests. Received a GET request.

Is there any change required in the URL?

Also could you please suggest if we can use an account, using which we can manually deploy to SharePoint site, but it doesn't have the administrative permissions on the tenant.

martinlingstuyl commented 2 years ago

Hi @AkashSatoliya, You can only call login.microsoftonline.com using a POST request. But that is not relevant to the current point.

I just tried it out: created a build pipeline with username and password variables. The yaml is as follows. You can import it if you want. The following works for me as is:

# Variable 'password' was defined in the Variables tab
# Variable 'username' was defined in the Variables tab
jobs:
- job: Job_1
  displayName: Agent job 1
  pool:
    vmImage: windows-2019
  steps:
  - checkout: self
  - task: NodeTool@0
    displayName: Use Node 16.x
    inputs:
      versionSpec: 16.x
  - task: Npm@1
    displayName: npm custom
    inputs:
      command: custom
      verbose: false
      customCommand: install @pnp/cli-microsoft365 -g
  - task: PowerShell@2
    displayName: PowerShell Script
    inputs:
      targetType: inline
      script: >-
        # Write your PowerShell commands here.

        Write-Host "Hello World"

        $status = m365 status --output text

        Write-Host "Connected: $status"

        Write-Host "Signing in with $(username)"

        m365 login -t password -u "$(username)" -p "$(password)"

        $status = m365 status --output text

        Write-Host "Connected: $status"
...

If this doesn't work with you please try to run the m365 login command on your laptop in PowerShell and see if you can get it signed in. If it gives the same Network error, it may still be a bug with some special characters in your password. Although we fixed that in a previous release.

Please let me know how this works out for you.

martinlingstuyl commented 2 years ago

Also could you please suggest if we can use an account, using which we can manually deploy to SharePoint site, but it doesn't have the administrative permissions on the tenant.

I'm not sure, but I believe you can just give an account permissions on the App Catalog. That could be enough so that you would not need the SharePoint admin role.

AkashSatoliya commented 2 years ago

Hi @martinlingstuyl ,

I tried to run the m365 login command on my system in PowerShell and it failed with same Network error. It looks like it might be a bug with special character in password. A part of password is highlighted like a variable. When debugged the script, the part of password which is like a variable it empty. Hence the value of password variable goes like incorrect password.

AkashSatoliya commented 2 years ago

Hi @martinlingstuyl

I tried with new password which does not include special character used to define variable in PowerShell but still I'm getting same network error.

martinlingstuyl commented 2 years ago

Is it possible you have a $ in your password? You could try escaping that:

"mypass`$word"

If that solves it locally, it might not solve it in the pipeline. (Do try though) if it does not you could try updating your password.

martinlingstuyl commented 2 years ago

By the way: that's the backtick "`" that you need to use to escape dollar chars in posh. It's not a single quote.

AkashSatoliya commented 2 years ago

Hi @martinlingstuyl

My previous password included $ but new password doesn't have $ in it. However it includes & and anything after & gets excluded when running 'm365 login' command When I tried to run the login command in command prompt I am getting below error:

Error: network_error: Network request failed. Please check network trace to determine root cause. | Fetch client threw: Error: HTTP status code 400 | Attempted to reach: https://login.microsoftonline.com/organizations/oauth2/v2.0/token
'XXxxXX' is not recognized as an internal or external command, operable program or batch file.

Part "XXxxXX" is after & in password which gets excluded and treated as command.

martinlingstuyl commented 2 years ago

Ok, try escaping the & with the backtick, like I showed with the dollar sign. It should work locally.

martinlingstuyl commented 2 years ago

Ah wait, no, escaping the ampersand is not necessary in PowerShell.

I've just tried it myself and I can sign in successfully using a password with ampersand.

martinlingstuyl commented 2 years ago

I tried on PowerShell 5, 7 and windows cmd. They all work fine on my end.

What version of CLI have you got locally? I'm on 5.7, the latest.

AkashSatoliya commented 2 years ago

I tried on PowerShell 5, 7 and windows cmd 6.3.9600, running on windows server 2012. When I tried with escaping ampersand it results in same network error. When tried to provide password within double quotes it results in same network error. When tried to provide password without double quotes it results in 'XXxxXX' is not recognized as an internal or external command, operable program or batch file.

martinlingstuyl commented 2 years ago

When tried to provide password within double quotes it results in same network error.

Just to be clear: you should indeed use quotes around the password

So just to be sure: You are now running CLI 5.7 and have a password without any special chars?

By the way: The network error also occurs if you make a mistake in writing your password or upn. We do not have a nice 'credentials invalid' error message here, it appears.

AkashSatoliya commented 2 years ago

I am using PowerShell version 5 and 7(tried on both). Password includes special characters. UPN and Passwords are correct as I'm using the same credentials for login and deployment using browser.

martinlingstuyl commented 2 years ago

You are now running CLI 5.7 and have a password without any special chars?

What is the output when you run m365 version?

AkashSatoliya commented 2 years ago

it's 5.7.

martinlingstuyl commented 2 years ago

hmm, I'm at a loss.

I can successfully sign in with any special char:

m365 login -t password -u "someupn@contoso.com" -p "zxc098!@#$%^&*()"
m365 login -t password -u "someupn@contoso.com" -p "zxc{}[]~,./<>?:;\098"

Might there be some corporate proxy in the way possibly?

To validate this: does executing the following (device code login) work as expected?

m365 login

Maybe you can try running the command on a different PC outside of your corporate network. Or try another user account. Just to try to rule things out.

AkashSatoliya commented 2 years ago

Hi @martinlingstuyl

Thanks for reply.

I tried with another account using method like m365 login -t password -u "someupn@contoso.com" -p "zxc098!@#$%^&*()" and it resulted in same network error. When tried to login with m365 login, it went to next step and showed message to open the page https://microsoft.com/devicelogin and enter a code for device login. After entering code it needs access resources in organization that only an admin can grant. As I don't have admin I'll request admin to grant permission to the account. After granting device access, can we use the same account to login using pipeline?

martinlingstuyl commented 2 years ago

Ok, we've found the problem:

You need to consent to the PnP application before being able to use the m365 login -t password. I've just gone through our documentation, and I can find this exactly nowhere. So I'll create an issue for us to update the documentation about in what cases you can expect this error and that consent is necessary.

Thanks for getting to the bottom with me on this and sorry for all the trouble!

martinlingstuyl commented 2 years ago

As I don't have admin I'll request admin to grant permission to the account.

So if you take this step, you should be able to connect afterwards. Also: MFA should be disabled on your account, but I assume you guessed that.

AkashSatoliya commented 2 years ago

Hi @martinlingstuyl

Thank you so much for looking into it. I've requested for permissions. I'll update once permission is granted and I'm able to test it.

AkashSatoliya commented 2 years ago

Hi @martinlingstuyl

Finally able to move one step ahead but still getting same error.

After running 'm365 login' command and feeding the code to https://microsoft.com/devicelogin, I directly got below message instead of asking for permissions as shown in screenshot on https://pnp.github.io/cli-microsoft365/user-guide/connecting-office-365/#log-in-using-the-default-device-code-flow

PnP Management Shell

You have signed in to the PnP Management Shell application on your device. You may now close this window.

I'm not sure if consent happened appropriately as it didn't show any list of permissions and directly gave above message.

As mentioned in message, I closed the window and tried to login with method like m365 login -t password -u "someupn@contoso.com" -p "zxc098!@#$%^&*()[]{}" but I'm still getting same error

Error: network_error: Network request failed. Please check network trace to determine root cause. | Fetch client threw: Error: HTTP status code 400 | Attempted to reach: https://login.microsoftonline.com/organizations/oauth2/v2.0/token

I tried with a simple password without any special characters as well but still getting the same error.

martinlingstuyl commented 2 years ago

Hi @AkashSatoliya, just to be sure: MFA is disabled on that account?

AkashSatoliya commented 2 years ago

Hi @martinlingstuyl ,

Yes, MFA has already been disabled for this account.

AkashSatoliya commented 2 years ago

Hi @martinlingstuyl,

Any luck with this error? Also I wonder if consent is completed appropriately as it didn't show the list of permissions. Is it expected behavior?

Thanks.

martinlingstuyl commented 2 years ago

Hi @AkashSatoliya, if consent is completed you do not get another message about it, so yes, that's expected behavior.

Tell me: if you run m365 login (the regular interactive sign in) and use the account you were using on the password type signin, does that work? Can you sign in successfully and execute commands?

AkashSatoliya commented 2 years ago

Hi @martinlingstuyl

Thanks for reply. Yes, I'm able to run 'm365 login' command interactively using the account I was trying to login using username and password. After device login I'm able to run commands. For example, I was able to run below commands successfully:

m365 aad user get --id "@meId"
m365 spo site get --url https://contoso.sharepoint.com/sites/contoso

But it fails when I try to login using same account using m365 login -t password -u "someupn@contoso.com" -p "zxc098!@#$%^&*()[]{}" method.

AkashSatoliya commented 2 years ago

Hi @martinlingstuyl

Is there any specific requirement for running login command with username and password?

I tried to run same command in Azure CD pipeline but it results in same error.

martinlingstuyl commented 2 years ago

Hi @AkashSatoliya, OK, good to hear it is in fact running using the interactive way.

My recent research has showed that the username/password login flow can return this error in the following scenario's:

I also got this error just now because I had accidentally written -userName instead of --userName. (missing one dash) The validation did not throw for a missing userName, which it does when just omitting the userName option entirely.

There might be more scenario's though...