Provisioning Engine: Property bag entries not being persisted when applying provisioning template

[nprieto95]

Scenario and context

We are developing a managed (at least in concept) application that relies on SharePoint. We create new SharePoint site collections based on the users' demand, but for technical restrictions this creation happens asynchronously and needs to be done with the OAuth Client Credentials flow (in other words, it is the app to create the site collections, not the user). Then this same app needs to activate the Document ID feature and change its settings (which are property bag entries).

Restrictions:

1. This needs to be automated, no supervision from an admin or any kind of user
2. It is a C# application and we can't use PowerShell

The app will be pre-registered in the SharePoint tenant with these permissions consented:

<AppPermissionRequests AllowAppOnlyPolicy="true">
    <AppPermissionRequest Scope="http://sharepoint/content/tenant" Right="FullControl"/>
</AppPermissionRequests>

What is the "PnP way" of doing this? I classified this as a bug because I thought that this would be abstracted away by the provisioning engine, but feel free to correct this thinking if I didn't get it right.

Category

[x] Bug [ ] Enhancement

Environment

[x] Office 365 / SharePoint Online [ ] SharePoint 2016 [ ] SharePoint 2013

Expected or Desired Behavior

The property bag entries specified in the template should be persisted in SharePoint.

Observed Behavior

The property bag entries specified in the template are not being persisted in SharePoint.

Steps to Reproduce

Using this template:

<?xml version="1.0"?>
<pnp:Provisioning xmlns:pnp="http://schemas.dev.office.com/PnP/2018/07/ProvisioningSchema">
  <pnp:Preferences Generator="OfficeDevPnP.Core, Version=3.2.1810.0, Culture=neutral, PublicKeyToken=5e633289e95c321a" />
  <pnp:Templates ID="CONTAINER-opportunitySite">
    <pnp:ProvisioningTemplate ID="opportunitySite" Version="0" Scope="Undefined">
      <pnp:Features>
        <pnp:SiteFeatures>
          <pnp:Feature ID="b50e3104-6812-424f-a011-cc90e6327318" />
        </pnp:SiteFeatures>
      </pnp:Features>
      <pnp:PropertyBagEntries>
        <pnp:PropertyBagEntry Key="docid_msft_hier_siteprefix" Value="PROPMGT" Overwrite="true" />
        <pnp:PropertyBagEntry Key="docid_enabled" Value="1" Overwrite="true" />
      </pnp:PropertyBagEntries>
    </pnp:ProvisioningTemplate>
  </pnp:Templates>
</pnp:Provisioning>

Run this code:

using Microsoft.SharePoint.Client;
using OfficeDevPnP.Core;
using OfficeDevPnP.Core.Framework.Provisioning.Providers;
using OfficeDevPnP.Core.Framework.Provisioning.Providers.Xml;
using System;
using static System.Configuration.ConfigurationManager;

namespace OpportunitySiteProvisioner
{
    class Program
    {
        static void Main(string[] args)
        {
            // Grab site from arguments
            var siteUrl = args[0];
            Console.WriteLine($"Site url is \"{siteUrl}\"");
            // Grab client credentials from configuration
            var appId = AppSettings["AppId"];
            var appSecret = AppSettings["AppSecret"];
            // Load the provisioning template
            Console.WriteLine("Loading template...");
            TemplateProviderBase templateProvider = new XMLFileSystemTemplateProvider(".\\", string.Empty);
            var template = templateProvider.GetTemplate("OpportunitySite.xml");
            Console.WriteLine("Template successfully parsed.");
            // Authenticate using the client credentials flow
            Console.WriteLine("Attempting OAuth authentication...");
            var authenticationManager = new AuthenticationManager();
            using (var context = authenticationManager.GetAppOnlyAuthenticatedContext(siteUrl, appId, appSecret))
            {
                Console.WriteLine("Succesfully authenticated. Provisioning...");
                // Apply the provisioning template
                context.Web.ApplyProvisioningTemplate(template);
                Console.WriteLine("Provisioning was successful.");
            }
        }
    }
}

[nprieto95]

By the way, the feature is being activated, so, as the title says, the problem with the Property Bag entries only.

[Ravikadri-zz]

Assuming this is on modern sites, I think noscript (DenyAddAndCustomizePages) should be disabled. Else it wont add new property values. Try disabling the noscript and try to apply the template with the property bag values.

PS1 command: Set-SPOsite -DenyAddAndCustomizePages 0

Refer the below article https://docs.microsoft.com/en-us/sharepoint/dev/solution-guidance/modern-experience-customizations-customize-sites

Hope this helps. -Ravi

[nprieto95]

Ravi

It helps. However, I'm not really sure how to do this from C#. I tried doing this:

// Grab site from arguments
var siteUrl = args[0];
var adminSiteUrl = new Uri(siteUrl.Replace(".sharepoint.com", "-admin.sharepoint.com")).GetLeftPart(UriPartial.Authority);
Console.WriteLine($"Site url is \"{siteUrl}\"");
// Grab client credentials from configuration
var appId = AppSettings["AppId"];
var appSecret = AppSettings["AppSecret"];
// Load the provisioning template
Console.WriteLine("Loading template...");
TemplateProviderBase templateProvider = new XMLFileSystemTemplateProvider(".\\", string.Empty);
var template = templateProvider.GetTemplate("OpportunitySite.xml");
Console.WriteLine("Template successfully parsed.");
// Authenticate using the client credentials flow
Console.WriteLine("Attempting OAuth authentication...");
var authenticationManager = new AuthenticationManager();
using (var adminContext = authenticationManager.GetAppOnlyAuthenticatedContext(adminSiteUrl, appId, appSecret))
{
    // Prepare site
    var tenant = new Tenant(adminContext);
    var siteProperties = tenant.GetSitePropertiesByUrl(siteUrl, true);
    adminContext.Load(siteProperties);
    adminContext.ExecuteQuery();
    var priorValue = siteProperties.DenyAddAndCustomizePages;
    siteProperties.DenyAddAndCustomizePages = DenyAddAndCustomizePagesStatus.Disabled;
    siteProperties.Update();
    adminContext.ExecuteQuery();
    using (var context = authenticationManager.GetAppOnlyAuthenticatedContext(siteUrl, appId, appSecret))
    {
        Console.WriteLine("Succesfully authenticated. Provisioning...");
        // Apply the provisioning template
        context.Web.ApplyProvisioningTemplate(template);  // <- 401 HERE
        Console.WriteLine("Provisioning was successful.");
    }
    siteProperties.DenyAddAndCustomizePages = priorValue;
    siteProperties.Update();
    adminContext.ExecuteQuery();
}

But now when I run this I get a 401 when I ApplyProvisioningTemplate. I checked going back to the code I posted at the beginning of the thread and it goes back to "normal" (only activates the feature but not the property bag entries).

The app does have permissions; its permission request XML is as follows:

<AppPermissionRequests AllowAppOnlyPolicy="true">
    <AppPermissionRequest Scope="http://sharepoint/content/tenant" Right="FullControl"/>
</AppPermissionRequests>

And it was consented from the /_layouts/AppInv.aspx form.

What is the correct way to do this? I will expand on my scenario in the original post so that the problem becomes more clear.

[Ravikadri-zz]

Check this out. It has sample has the CSOM example

https://asishpadhy.com/2018/04/23/how-to-set-property-bag-values-in-sharepoint-modern-sites-using-sharepoint-online-net-csom/

[nprieto95]

Thank you!

This is the final code that worked for me:

// Grab site from arguments
var siteUrl = args[0];
var adminSiteUrl = new Uri(siteUrl.Replace(".sharepoint.com", "-admin.sharepoint.com")).GetLeftPart(UriPartial.Authority);
Console.WriteLine($"Site url is \"{siteUrl}\"");
// Grab client credentials from configuration
var appId = AppSettings["AppId"];
var appSecret = AppSettings["AppSecret"];
// Load the provisioning template
Console.WriteLine("Loading template...");
TemplateProviderBase templateProvider = new XMLFileSystemTemplateProvider(".\\", string.Empty);
var template = templateProvider.GetTemplate("OpportunitySite.xml");
Console.WriteLine("Template successfully parsed.");
// Authenticate using the client credentials flow
Console.WriteLine("Attempting OAuth authentication...");
// Prepare site
var adminAuthenticationManager = new AuthenticationManager();
using (var adminContext = adminAuthenticationManager.GetAppOnlyAuthenticatedContext(adminSiteUrl, appId, appSecret))
{
    var tenant = new Tenant(adminContext);
    tenant.SetSiteProperties(siteUrl, noScriptSite: false);
    adminContext.ExecuteQuery();
    var authenticationManager = new AuthenticationManager();
    using (var context = authenticationManager.GetAppOnlyAuthenticatedContext(siteUrl, appId, appSecret))
    {
        Console.WriteLine("Succesfully authenticated. Provisioning...");
        // Apply the provisioning template
        context.Web.ApplyProvisioningTemplate(template);
        Console.WriteLine("Provisioning was successful.");
    }
}

I had to use two separate authentication managers, that solved the 401.

Do you think this is a bad practice? Is there any other way to achieve this? I understand the technical reason that makes this necessary, but from a developer perspective I think that the code needed to do this disrupts the main idea, which is simply provisioning the site template. Besides, if I was able to do that with the same context I had before, could this not be abstracted away in the ApplyProvisioningTemplate method? Or is there a conceptual reason we are not doing that today?

I ask these questions to better understand why this is not a bug in case it isn't.

[nprieto95]

One more thought: also from a developer perspective, if the code is asking the provisioning engine to set some property bag entries and the provisioning engine is not able to do that, shouldn't it throw an exception instead of silently failing?

I'm not necessarily saying that this is a problem in the provisioning engine... maybe it goes deeper into CSOM itself (I honestly don't know), but don't you agree it would be better to get an exception thrown when it is not possible to fulfill the request instead of returning back to the caller with no hint of a failure?

[Ravikadri-zz]

agreed. May be you can raise it as an enhancement feature. Instead of throwing error, just show warning stating it cant add property bag values because of the settings, I think that would be ideal.

-Ravi