search

microsoft / Microsoft365DSC

Manages, configures, extracts and monitors Microsoft 365 tenant configurations
https://aka.ms/M365DSC
MIT License
1.53k stars 470 forks source link

Azure Automation Account #472

Closed HaloX69 closed 3 years ago

HaloX69 commented 4 years ago

I am attempting to configure a state management leveraging this module in an Azure Automation Account. I have manually installed all of the dependent modules with the needed versions, and yet when I attempt to install Microsoft365DSC, Azure indicates that the needed resources are not found. When I manually upload the module it fails without a reason provided. I do hope this module is supported in Automation Accounts where we can run DSC. If it doesn't is there plans to make the needed changes to support in Azure Automation Accounts? Some organizations don't have the resources for on-prem, but would still want to apply DSC to their cloud resources.

ThorstenLoeschmann commented 4 years ago

Hi, Microsoft365DSC is supported in Azure Automation. I set it up the last time last month and it was working. I agree the handling of the modules is somewhat cumbersome since you have to upload the required modules manually. Have you tried to save the modules to your local machine either from the PowerShell Gallery or the GitHub Repo and zip them up and then upload them to Azure Automation? Thats how I did it the last time, if I recall correctly. Please also verify that after uploading the required modules manually all modules are in the "Available" state before uploading the Microsoft365DSC module.

HaloX69 commented 4 years ago

If this module won't run in an Azure Automation Account have you tested this module in a Hybrid Runbook Worker configuration? Obviously ideally I would like to be notified when a O365 service configuration is out of compliance and is auto-corrected.

ThorstenLoeschmann commented 4 years ago

Could you please try to save the modules in the required version, locally to your machine (either from the PSGallery or the GitHub Repo), zip every single one in its own archive and manually upload them one by one into your Automation Accounts Modules instead of trying to import it directly from the PS Gallery?

NikCharlebois commented 4 years ago

Just to add to what Thorsten says. The module is fully supported in Azure Automation. However, the sub-dependency engine of Azure Automation being what it is, you will need to manually import each dependencies first, and then import the Microsoft365DSC project.

HaloX69 commented 4 years ago

Thorsten, as I indicated I did individually import the dependent resources in advance. I was not able to import the Microsoft365DSC module at all. It took several attempts to manually import the module and it worked finally. I created a configuration, SPOSite, with compiled fine, but how is it applied since Azure Automation SC looks to leverage a node VM/Server on-prem/cloud? Have you tried to apply the compiled configuration?

HaloX69 commented 4 years ago

Will I need to have a host VM in Azure to execute the configuration, and then assign that node to the compiled configuration?

HaloX69 commented 4 years ago

Built a Windows 2016 server in Azure and associated with a configuration. I keep getting the exception, Installation of module Microsoft365DSC failed since the module directory already exists at C:\Program Files\WindowsPowerShell\Modules\Microsoft365DSC\1.0.4.39. It stops and doesn't proceed. The configuration is a simple SPOSite configuration.

What is your testing environment? Is there system requirements that aren't posted?

NikCharlebois commented 4 years ago

Can you post your config in here? I want to check if it is referencing a specific version of the module. To answer you previous question, when using Azure Automation DSC for M365DSC, you need to deploy the compiled configuration to a VM. This VM acts as a middle-man (proxy) to then communicate back to Microsoft 365. If you wish to use an agentless process I recommend you look at using Azure DevOPS pipelines instead.

ThorstenLoeschmann commented 4 years ago

Hi, could you please connect to the machine and check if all the required modules are installed correctly in the modules directory? Thank you.

HaloX69 commented 4 years ago

I would have figured the pull server would have pushed the resources. I manually installed the modules. The configuration is a simple SPOSite which fails with the below message. The credntial is stored in the Automation Credential resources and I extract to create a PSCredential object.

I suspect that it is trying to prompt for authentication. Below this message I have the configuration. I think the problem is the account I am using is two-factor enabled. Logically all of our service admin accounts are two-factor. Is there anyway we can leverage service principles? I know the SharePoint PNP PowerShell modules support this.

I have also included my configuration.

{ "Exception": { "Message": "PowerShell DSC resource MSFT_SPOSite failed to execute Test-TargetResource functionality with error message: One or more errors occurred.: Showing a modal dialog box or form when the application is not running in UserInteractive mode is not a valid operation. Specify the ServiceNotification or DefaultDesktopOnly style to display a notification from a service application. ", "Data": {

                           },
                  "InnerException":  {
                                         "ErrorRecord":  "One or more errors occurred.: Showing a modal dialog box or form when the application is not running in UserInteractive mode is not a valid operation. Specify the ServiceNotification or DefaultDesktopOnly style to display a notification from a service application.",
                                         "WasThrownFromThrowStatement":  true,
                                         "Message":  "One or more errors occurred.: Showing a modal dialog box or form when the application is not running in UserInteractive mode is not a valid operation. Specify the ServiceNotification or DefaultDesktopOnly style to display a notification from a service application.",
                                         "Data":  "System.Collections.ListDictionaryInternal",
                                         "InnerException":  "Microsoft.Open.Azure.AD.CommonLibrary.AadAuthenticationFailedException: One or more errors occurred.: Showing a modal dialog box or form when the application is not running in UserInteractive mode is not a valid operation. Specify the ServiceNotification or DefaultDesktopOnly style to display a notification from a service application. ---\u003e System.AggregateException: One or more errors occurred. ---\u003e System.InvalidOperationException: Showing a modal dialog box or form when the application is not running in UserInteractive mode is not a valid operation. Specify the ServiceNotification or DefaultDesktopOnly style to display a notification from a service application.\r\n   at System.Windows.Forms.Form.ShowDialog(IWin32Window owner)\r\n   at Microsoft.IdentityModel.Clients.ActiveDirectory.Internal.WindowsFormsWebAuthenticationDialog.ShowBrowser()\r\n   at Microsoft.IdentityModel.Clients.ActiveDirectory.Internal.WindowsFormsWebAuthenticationDialog.OnAuthenticate()\r\n   at Microsoft.IdentityModel.Clients.ActiveDirectory.Internal.WindowsFormsWebAuthenticationDialogBase.AuthenticateAAD(Uri requestUri, Uri callbackUri)\r\n   at Microsoft.IdentityModel.Clients.ActiveDirectory.Internal.Platform.InteractiveWebUI.OnAuthenticate()\r\n   at Microsoft.IdentityModel.Clients.ActiveDirectory.Internal.Platform.WebUI.\u003c\u003ec__DisplayClass12_0.\u003cAcquireAuthorizationAsync\u003eb__0()\r\n   at Microsoft.IdentityModel.Clients.ActiveDirectory.Internal.Platform.WebUI.\u003cAcquireAuthorizationAsync\u003ed__12.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n   at Microsoft.IdentityModel.Clients.ActiveDirectory.Internal.Flows.AcquireTokenInteractiveHandler.\u003cAcquireAuthorizationAsync\u003ed__11.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n   at Microsoft.IdentityModel.Clients.ActiveDirectory.Internal.Flows.AcquireTokenInteractiveHandler.\u003cPreTokenRequestAsync\u003ed__10.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n   at Microsoft.IdentityModel.Clients.ActiveDirectory.Internal.Flows.AcquireTokenHandlerBase.\u003cRunAsync\u003ed__57.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n   at Microsoft.IdentityModel.Clients.ActiveDirectory.AuthenticationContext.\u003cAcquireTokenCommonAsync\u003ed__39.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n   at Microsoft.IdentityModel.Clients.ActiveDirectory.AuthenticationContext.\u003cAcquireTokenAsync\u003ed__30.MoveNext()\r\n   --- End of inner exception stack trace ---\r\n   at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)\r\n   at Microsoft.Open.Azure.AD.CommonLibrary.UserTokenProvider.DoAcquireToken(AdalConfiguration config, PromptBehavior promptBehavior, String userId, SecureString password) in X:\\bt\\1087869\\repo\\src\\dev\\PowerShell.V2\\CommonLibrary\\TokenProvider\\UserTokenProvider.cs:line 213\r\n   at Microsoft.Open.Azure.AD.CommonLibrary.UserTokenProvider.SafeAquireToken(AdalConfiguration config, ShowDialog showDialog, String userId, SecureString password, Exception\u0026 ex) in X:\\bt\\1087869\\repo\\src\\dev\\PowerShell.V2\\CommonLibrary\\TokenProvider\\UserTokenProvider.cs:line 152\r\n   --- End of inner exception stack trace ---\r\n   at Microsoft.Open.Azure.AD.CommonLibrary.UserTokenProvider.AcquireToken(AdalConfiguration config, ShowDialog promptBehavior, String userId, SecureString password) in X:\\bt\\1087869\\repo\\src\\dev\\PowerShell.V2\\CommonLibrary\\TokenProvider\\UserTokenProvider.cs:line 134\r\n   at Microsoft.Open.Azure.AD.CommonLibrary.UserTokenProvider.GetAccessToken(AdalConfiguration config, ShowDialog promptBehavior, String userId, SecureString password, AccountType credentialType) in X:\\bt\\1087869\\repo\\src\\dev\\PowerShell.V2\\CommonLibrary\\TokenProvider\\UserTokenProvider.cs:line 57\r\n   at Microsoft.Open.Azure.AD.CommonLibrary.AuthenticationFactory.Authenticate(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior, TokenCache tokenCache, Endpoint resourceId) in X:\\bt\\1087869\\repo\\src\\dev\\PowerShell.V2\\CommonLibrary\\AuthenticationFactory.cs:line 59\r\n   at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.AcquireAccessToken(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password, ShowDialog promptBehavior) in X:\\bt\\1087869\\repo\\src\\dev\\PowerShell.V2\\CommonLibrary\\RMProfileClient.cs:line 127\r\n   at Microsoft.Open.Azure.AD.CommonLibrary.RMProfileClient.Login(AzureAccount account, AzureEnvironment environment, String tenantId, SecureString password) in X:\\bt\\1087869\\repo\\src\\dev\\PowerShell.V2\\CommonLibrary\\RMProfileClient.cs:line 55\r\n   at Microsoft.Open.Azure.AD.CommonLibrary.ConnectAzureAD.ProcessRecord() in X:\\bt\\1087869\\repo\\src\\dev\\PowerShell.V2\\CommonLibrary\\ConnectAzureAD.cs:line 163",
                                         "TargetSite":  "System.Collections.ObjectModel.Collection`1[System.Management.Automation.PSObject] Invoke(System.Collections.IEnumerable)",
                                         "StackTrace":  "   at System.Management.Automation.Runspaces.PipelineBase.Invoke(IEnumerable input)\r\n   at System.Management.Automation.PowerShell.Worker.ConstructPipelineAndDoWork(Runspace rs, Boolean performSyncInvoke)\r\n   at System.Management.Automation.PowerShell.Worker.CreateRunspaceIfNeededAndDoWork(Runspace rsToUse, Boolean isSync)\r\n   at System.Management.Automation.PowerShell.CoreInvokeHelper[TInput,TOutput](PSDataCollection`1 input, PSDataCollection`1 output, PSInvocationSettings settings)\r\n   at System.Management.Automation.PowerShell.CoreInvoke[TInput,TOutput](PSDataCollection`1 input, PSDataCollection`1 output, PSInvocationSettings settings)\r\n   at System.Management.Automation.PowerShell.Invoke(IEnumerable input, PSInvocationSettings settings)\r\n   at Microsoft.PowerShell.DesiredStateConfiguration.Internal.ResourceProviderAdapter.ExecuteCommand(PowerShell powerShell, ResourceModuleInfo resInfo, String operationCmd, List`1 acceptedProperties, CimInstance nonResourcePropeties, CimInstance resourceConfiguration, LCMDebugMode debugMode, PSInvocationSettings pSInvocationSettings, UInt32\u0026 resultStatusHandle, Collection`1\u0026 result, ErrorRecord\u0026 errorRecord, PSModuleInfo localRunSpaceModuleInfo)",
                                         "HelpLink":  null,
                                         "Source":  "System.Management.Automation",
                                         "HResult":  -2146233087
                                     },
                  "TargetSite":  null,
                  "StackTrace":  null,
                  "HelpLink":  null,
                  "Source":  null,
                  "HResult":  -2146233079
              },
"TargetObject":  null,
"CategoryInfo":  {
                     "Category":  7,
                     "Activity":  "",
                     "Reason":  "InvalidOperationException",
                     "TargetName":  "",
                     "TargetType":  ""
                 },
"FullyQualifiedErrorId":  "ProviderOperationExecutionFailure",
"ErrorDetails":  null,
"InvocationInfo":  null,
"ScriptStackTrace":  null,
"PipelineIterationInfo":  [

                          ]

}

node localhost { SPOSite DemoSite { Url = "https://XXXXXXX.sharepoint.com/sites/testsiteDSC" StorageMaximumLevel = 26214400 LocaleId = 1033 Template = "STS#3" GlobalAdminAccount = $credsGlobalAdmin Owner = "XXXX.XXX@XXXXXXX.onmicrosoft.com" Title = "TestSite DSC created" TimeZoneId = 13 Ensure = "Present" StorageWarningLevel = 25574400 SharingCapability = "Disabled" CommentsOnSitePagesDisabled = $false DisableAppViews = "NotDisabled" DisableCompanyWideSharingLinks = "NotDisabled" DisableFlows = $false DefaultSharingLinkType = "None" DefaultLinkPermission = "None" } } }

ThorstenLoeschmann commented 4 years ago

You are right, normally the pull server should lay down all the required modules on the machine, somehow it doesn't seem to understand / respect the dependencies in the module. Multi-factor is one of the big challenges we are facing, since not all M365 PowerShell related modules support a service principal it has not been implemented (yet).

HaloX69 commented 4 years ago

I am using the example SPOSITE configuration provided in the module just changing the url to reflect my tenant domain. I’ve not changed anything just trying to get a baseline config to work.

Get Outlook for iOShttps://aka.ms/o0ukef

From: Nik Charlebois notifications@github.com Sent: Monday, May 4, 2020 6:19:43 PM To: microsoft/Microsoft365DSC Microsoft365DSC@noreply.github.com Cc: HaloX69 davejoye@hotmail.com; Author author@noreply.github.com Subject: Re: [microsoft/Microsoft365DSC] Azure Automation Account (#472)

Can you post your config in here? I want to check if it is referencing a specific version of the module. To answer you previous question, when using Azure Automation DSC for M365DSC, you need to deploy the compiled configuration to a VM. This VM acts as a middle-man (proxy) to then communicate back to Microsoft 365. If you wish to use an agentless process I recommend you look at using Azure DevOPS pipelines instead.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/microsoft/Microsoft365DSC/issues/472#issuecomment-623738178, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ACASBEFX3WILEHG3SCDWEWTRP45P7ANCNFSM4MXEHJQQ.

desmay commented 4 years ago

@HaloX69 We did release support for service principals for SharePoint. You can use 2 different methods. You can pass in TenantID,ClientId,CertificateThumprint or TenantID, Client ID, CertificatePath and CertificatePassword.

We are working on documentation for using service principals and hope to have it done soon.

fs366e2spm commented 4 years ago

Hi, could you please provide more (detailed) information or description on how to get Microsoft365DSC to run in a Runbook?

Would be great - thanks

Robert1976 commented 3 years ago

I have been trying to run a DSC config in an Azure Automation runbook. I installed the module using the "Deploy to Azure Automation" option in the PowerShell Gallery. I created an example config like this:

Configuration NameOfTheConfiguration
{
    Import-DSCResource -ModuleName Microsoft365DSC
    $GlobalAdminAccount = Get-AutomationPSCredential account
    Node Localhost
    {
        SPOSite MyHRSite
        {
            Title              = "SP Site title"
            Url                = "https://****.sharepoint.com/sites/ExampleSite"
            Owner              = "***@***.nl"
            TimeZoneId         = 4
            GlobalAdminAccount = $GlobalAdminAccount
            Ensure             = "Present"
            SharingCapability  = "Disabled"
        }
    }
}

NameOfTheConfiguration -Output .\Output\ 

Start-DSCConfiguration -Path .\output\ -Wait -Verbose -Force

Then I get the following error:

Perform operation 'Invoke CimMethod' with following parameters, ''methodName' = SendConfigurationApply,'className' = MSFT_DSCLocalConfigurationManager,'namespaceName' = root/Microsoft/Windows/DesiredStateConfiguration'. Access is denied.

Can somebody explain what I am doing wrong?

NikCharlebois commented 3 years ago

Adding @hmank as he might have an idea here.

Robert1976 commented 3 years ago

I have got it working now using a hybrid runbook worker. The steps I have taken is:

(1) Create an Azure VM and add this VM to the automation account as a hybrid runbook worker. (2) Add the same VM as a DSC node to the automation account. (3) Install the Microsoft365DSC module to the automation account using the "Deploy to Azure Automation" option in the PowerShell Gallery. (4) Run your Microsoft365DSC runbook on the hybrid worker.

HaloX69 commented 3 years ago

I did get it to work both as a DSC node and hybrid. Our InfoSec group is still greatly displeased the number of login accounts needed instead of service principles to support EXO, Teams (really the Skype cmdlets), SAC resources.

I am working on a way to only leverage svc prin from configuration and then the svc prin gets the cloud account, create PSCred (GlobalAdminAccount) this way the password isn’t know or accessible from Azure Automation or the operators and the password can be rotated daily

Get Outlook for iOShttps://aka.ms/o0ukef

From: ThorstenLoeschmann notifications@github.com Sent: Monday, May 4, 2020 1:55:12 AM To: microsoft/Microsoft365DSC Microsoft365DSC@noreply.github.com Cc: HaloX69 davejoye@hotmail.com; Author author@noreply.github.com Subject: Re: [microsoft/Microsoft365DSC] Azure Automation Account (#472)

Hi, Microsoft365DSC is supported in Azure Automation. I set it up the last time last month and it was working. I agree the handling of the modules is somewhat cumbersome since you have to upload the required modules manually. Have you tried to save the modules to your local machine either from the PowerShell Gallery or the GitHub Repo and zip them up and then upload them to Azure Automation? Thats how I did it the last time, if I recall correctly. Please also verify that after uploading the required modules manually all modules are in the "Available" state before uploading the Microsoft365DSC module.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/microsoft/Microsoft365DSC/issues/472#issuecomment-623271520, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ACASBEBEY6N2CIJUNSLDERLRPZKEBANCNFSM4MXEHJQQ.

NikCharlebois commented 3 years ago

Regarding Azure Automation, the folks at O365Eh.com have a great blog/webcast on how to use M365DSC within Azure Automaton: https://o365eh.com/2020/10/27/episode-74-using-microsoft-dsc-as-a-runbook-in-azure-automation/