search

halliba / az-pipelines-2-sharepoint

Azure DevOps extension - upload build files to SharePoint Online
GNU General Public License v3.0
9 stars 4 forks source link

#[error]Could not validate drive id. #8

Open shiva-gattu opened 1 year ago

shiva-gattu commented 1 year ago

I'm facing and error when i try to move files from Azure repos to sharepoint using this extension. I get an error as could not validate drive Id. not sure where I'm missing

halliba commented 1 year ago

Hi,

are you still having this issue? If so: I just release an update and added some debug messages. Can you please set the variable system.debug to true and re-run the task?

Thank you!

MHebes commented 1 year ago

I am getting a similar issue. I assume I'm doing the authentication setup in Azure incorrectly or something?

My app has these permissions:

image

Here's my task:

  steps:
  - powershell: echo test > '$(Build.BinariesDirectory)/test.txt'
  - task: az-pipelines-2-sharepoint@0
    condition: and(succeeded(), eq(variables.isTag, 'true'))
    inputs:
      tenantId: '$(SigningTenantID)'
      clientId: '$(SharePointUploadClientID)'
      clientSecret: '$(SigningClientSecret)'
      driveId: '$(SharePointUrl)'
      targetFolder: '$(SharePointFolder)'
      contents: '$(Build.BinariesDirectory)/test.txt'
      conflictBehaviour: 'rename'
      cleanTargetFolder: false
      flattenFolders: false
      failOnEmptySource: false

Here's the full expanded error message:

{
  "statusCode": 401,
  "code": "InvalidAuthenticationToken",
  "requestId": "531c2b02-e2f5-49bc-85bd-336d958c8a05",
  "date": "2023-06-15T19:58:35.000Z",
  "body": {
    "code": "InvalidAuthenticationToken",
    "message": "CompactToken parsing failed with error code: 80049217",
    "innerError": {
      "date": "2023-06-15T15:58:35",
      "request-id": "531c2b02-e2f5-49bc-85bd-336d958c8a05",
      "client-request-id": "c211014b-5ed8-2804-cfe2-77494f7070bf"
    }
  }
}

And here's the full logs:

##[section]Starting: azpipelines2sharepoint
==============================================================================
Task         : Upload files to SharePoint Online
Description  : Upload files to SharePoint Online
Version      : 0.3.4
Author       : halliba (github.com/halliba)
Help         : [github.com/halliba/az-pipelines-2-sharepoint](https://github.com/halliba/az-pipelines-2-sharepoint)
==============================================================================
##[debug]Using node path: C:\agent\externals\node10\bin\node.exe
##[debug]agent.TempDirectory=C:\agent\_work\_temp
##[debug]loading inputs and endpoints
##[debug]loading ENDPOINT_AUTH_PARAMETER_SYSTEMVSSCONNECTION_ACCESSTOKEN
##[debug]loading ENDPOINT_AUTH_SCHEME_SYSTEMVSSCONNECTION
##[debug]loading ENDPOINT_AUTH_SYSTEMVSSCONNECTION
##[debug]loading INPUT_CLEANTARGETFOLDER
##[debug]loading INPUT_CLIENTID
##[debug]loading INPUT_CLIENTSECRET
##[debug]loading INPUT_CONFLICTBEHAVIOUR
##[debug]loading INPUT_CONTENTS
##[debug]loading INPUT_DRIVEID
##[debug]loading INPUT_FAILONEMPTYSOURCE
##[debug]loading INPUT_FLATTENFOLDERS
##[debug]loading INPUT_SOURCEFOLDER
##[debug]loading INPUT_TARGETFOLDER
##[debug]loading INPUT_TENANTID
##[debug]loading SECRET_SHAREPOINTPASSWORD
##[debug]loading SECRET_SHAREPOINTUPLOADCLIENTSECRET
##[debug]loading SECRET_SIGNINGCLIENTSECRET
##[debug]loading SECRET_SYSTEM_ACCESSTOKEN
##[debug]loaded 18
##[debug]Agent.ProxyUrl=undefined
##[debug]Agent.CAInfo=undefined
##[debug]Agent.ClientCert=undefined
##[debug]Agent.SkipCertValidation=undefined
##[debug]tenantId=11111111-1111-1111-1111-111111111111
##[debug]clientId=22222222-2222-2222-2222-222222222222
##[debug]clientSecret=***
##[debug]driveId=https://company.sharepoint.com/sites/SiteName/Shared%20Documents
##[debug]targetFolder=/Sunfish/Releases/Test
##[debug]sourceFolder=C:\agent\_work\2\s
##[debug]contents=C:\agent\_work\2\b/test.txt
##[debug]conflictBehaviour=rename
##[debug]cleanTargetFolder=false
##[debug]flattenFolders=false
##[debug]failOnEmptySource=false
Found 0 files in 'C:\agent\_work\2\s'.
##[debug]Files: []
Validating drive id / url: 'https://company.sharepoint.com/sites/SiteName/Shared%20Documents'.
##[error]{"statusCode":401,"code":"InvalidAuthenticationToken","requestId":"531c2b02-e2f5-49bc-85bd-336d958c8a05","date":"2023-06-15T19:58:35.000Z","body":"{\"code\":\"InvalidAuthenticationToken\",\"message\":\"CompactToken parsing failed with error code: 80049217\",\"innerError\":{\"date\":\"2023-06-15T15:58:35\",\"request-id\":\"531c2b02-e2f5-49bc-85bd-336d958c8a05\",\"client-request-id\":\"c211014b-5ed8-2804-cfe2-77494f7070bf\"}}"}
##[debug]Processed: ##vso[task.issue type=error;]{"statusCode":401,"code":"InvalidAuthenticationToken","requestId":"531c2b02-e2f5-49bc-85bd-336d958c8a05","date":"2023-06-15T19:58:35.000Z","body":"{\"code\":\"InvalidAuthenticationToken\",\"message\":\"CompactToken parsing failed with error code: 80049217\",\"innerError\":{\"date\":\"2023-06-15T15:58:35\",\"request-id\":\"531c2b02-e2f5-49bc-85bd-336d958c8a05\",\"client-request-id\":\"c211014b-5ed8-2804-cfe2-77494f7070bf\"}}"}
##[debug]task result: Failed
##[error]Could not validate drive id.
##[debug]Processed: ##vso[task.issue type=error;]Could not validate drive id.
##[debug]Processed: ##vso[task.complete result=Failed;]Could not validate drive id.
##[section]Finishing: azpipelines2sharepoint

Again, I'm not sure this is an issue with the extension. I'm just not certain what kind of authentication I'm supposed to be using.

MHebes commented 1 year ago

I combined https://learn.microsoft.com/en-us/azure/active-directory/develop/tutorial-v2-nodejs-console with your code (primarily the drive-utils.ts), and it seems to work great.

Specifically, I used the @azure/msal-node package to get a token instead of the raw fetch(`https://login.microsoftonline.com/${tenantId}/oauth2/v2.0/token`, ...) that this library currently uses.

This comment on an issue for onedrive's api suggests that you may be missing the resource query param in getAccessTokenAsync, but I'm not sure because your grant_type is different from that person's.

MHebes commented 1 year ago

Nope, that wasn't it. In my test script I switched back from msal to your getAccessTokenAsync without modifications and it works great.

MHebes commented 1 year ago

I was using the wrong pipeline variable for my client secret lol ignore all previous posts. Learned a lot about graph APIs.

VassilisM commented 7 months ago

Hi. Having the same issue. Granted, not sure if the clientID and secret are correct, there are so many in there. But regarding this "token", no idea where to get it from or where to use it. UPDATE I found out that the client must be the service principal connected with the project in the pipeline. And that the secret is actually hidden, it should have been saved upon the creation of this entity and not use the guid of it. It was not saved... Little to do without permissions. Waiting on new one to test again. UPDATE 2 Nope. Same.

[error]{"statusCode":401,"code":"InvalidAuthenticationToken","requestId":... {\"code\":\"InvalidAuthenticationToken\",\"message\":\"IDX14100: JWT is not well formed, there are no dots (.). ...

[error]Could not validate drive id.

halliba commented 7 months ago

Did you create an App Registration in Entra ID / Azure AD according to the readme? The token mentioned in the error message is nothing you can configure. It is acquired dynamically form Entra ID with the Client ID and Client Secret that you've configured. It looks like the extension receives some invalid response from Entra ID and passes it on to Microsoft Graph, so the issue is probably in the App Registration or Permissions. I will try to replicate the problem....

VassilisM commented 7 months ago

I did not. I took the clientID and secret (well, IT generated a new one) from the registered service principal of the project. At least that is what another team is using, but not with this task/extension... If I were to create (I do not know if I have the rights), why would I do that for? As a sharepoint "proxy"? But you know what? I will try doing that as well. I am at test run #28 right now using a powershell script to do the upload manually. Whatever works first... UPDATE I do not have access to create a new one. UPDATE 2 Interesting... I have established that the values saved at the pipeline library weren't used, so the parameters passed to the task were, well, inexistent. Hence the error. As soon as I replaced them with the fixed values, I got a 403, access denied, so it must be some other settings in the apps, and probably nothing to do with this extension task! :-)

halliba commented 7 months ago

Hi, yes it should be possible to reuse any other service principal, but I have not tested it. Also I'm not 100% sure, what you mean by "service principal of the project". In any case, you have to make sure, that the app registration of the service principal has the Files.ReadWrite.All permission set, or the Sites.Selected with appropriate permissions on SharePoint. For the latter, I've heard from multiple users, that they had problems setting it up, so I have to verfiy that.. You might want to hint your IT to this repo's readme, they should be able to configure the app / service principal accordingly

rdinizz commented 6 months ago

I was having a similar problem and asked to IT department to create a new app registration as described in docs and the token problem was solved, but now i'm facing access denied:

`Found 1 files in '/Users/runner/work/1/a/output/iphoneos/Release'. Validating drive id / url: 'https://tenant.sharepoint.com/sites/project/Digital%20Guest%20and%20Brand/'.

[error]{"statusCode":403,"code":"accessDenied","requestId":"x","date":"2024-02-20T19:56:55.000Z","body":"{\"code\":\"accessDenied\",\"message\":\"Access denied\",\"innerError\":{\"date\":\"2024-02-20T19:56:55\",\"request-id\":\"x\",\"client-request-id\":\"x\"}}"}

[error]Could not validate drive id.

Finishing: azpipelines2sharepoint`

Screenshot 2024-02-21 at 11 00 02

The permissions were added as you can see above, i sent the IT team the blog links as well. Is there any config needed on sharepoint side? I need to ask somebody else as i don't have permissions there.

@halliba any ideas?

UPDATE: i asked IT department to do this process https://blog.dan-toft.dk/2022/12/sites-selected-permissions/#so-as-a-developer-what-do-i-do to grant our app access to the sites we want it to use but when they try to list the sites (GET | https://graph.microsoft.com/v1.0/sites?select=webUrl,Title,Id&$search="*") the site in question does not appear there, so i can't get the site id required to give the permissions

halliba commented 6 months ago

Hi, to find the site-id required for the grant command, you can use a GET request with this format /sites/{hostname}:/{server-relative-path} in Graph Explorer. hostname would be contoso.sharepoint.com and server-relative-path would be sites/my-site. I know, that there are problems with nested sites and the discovery of those via Graph Explorer - so that might be the problem. In General Microsoft Graph works best, if you can address the resource directly, not via the search capabilities.

See also these links: https://stackoverflow.com/questions/45406451/how-can-i-get-the-siteid-of-the-current-site-with-microsoft-graph-api https://learn.microsoft.com/en-us/graph/api/site-get?view=graph-rest-1.0&tabs=http#access-a-site-by-server-relative-url

rdinizz commented 6 months ago

Thank you @halliba i forgot to give an update but basically i found 2 ways of giving the permissions to an ad app:

It can be done either via Graph API


POST <-  
URL: https://graph.microsoft.com/v1.0/sites/<site-id>/permissions
site-id format example: "tenant.sharepoint.com,a222a734-4f89-4a91-86c1-6d17a45701c1,e2c8a4bf-e015-474b-a2f6-c9c615c55e67"
BODY:
{
    "roles": [
        "write"
    ],
    "grantedToIdentities": [
        {
            "application": {
                "id": "clientId",
                "displayName": "App Name" 
            }
        }
    ]
}

The site-id needed is in this format: {tenant.sharepoint.com},{siteId},{webId} I was not able to get the siteId via graph queries, but i managed to get it by adding _api/site/id and _api/web/id at the end of the site url, example https://contoso.sharepoint.com/sites/some-project/_api/site/id to get siteId, and https://contoso.sharepoint.com/sites/some-project/_api/web/id to get webId

Or running this in PnP Powershell

Grant-PnPAzureADAppSitePermission -AppId 'clientID' -DisplayName 'App Name' -Site 'https://tenant.sharepoint.com/sites/site' -Permissions Write

The problem was that nobody in the team had permissions to run it, so i escalated it.

Palanikr commented 5 months ago

@rdinizz Do we need to add azure app registration details(clientid, tenant id) in sharepoint as well. ? to communicate with azure devops pipeline @halliba

rdinizz commented 5 months ago

@rdinizz Do we need to add azure app registration details(clientid, tenant id) in sharepoint as well. ? to communicate with azure devops pipeline @halliba

You need to grant permissions to the app that you created via graph api or pnp powershell. Then it will be able to authenticate and see the site folders from the pipelines.

Palanikr commented 5 months ago

@rdinizz @halliba @MHebes I have grant permissions in the azure portal by giving access to the following api's

Microsoft GRAPH API Directory.ReadWrite.All Directory.AccessAsUser.All User.ReadWrite.All Group.ReadWrite.All GroupMember.ReadWrite.All Sites.FullControl.All sites.read.all(application) user.read.all SharePoint Online API AllSites.FullControl AllSites.Manage Sites.FullControl.All Sites.Search.All User.ReadWrite.All User.ReadWrite.All user.read.all

Still I'm receiving ""Either scp or roles claim need to be present in the token" error when i run the pipeline