Azure / missionlz

Azure landing zone for SCCA-compliant organizations.
MIT License
220 stars 122 forks source link

Spike: Deploy Zero Trust Workbook to Sentinel #258

Closed brooke-hamilton closed 2 years ago

brooke-hamilton commented 3 years ago

Benefit/Result/Outcome
So that I can be sure that MLZ implements the security guidelines of Zero Trust, and to have an MLZ example that includes additional deployment technologies like shell scripts and manual Azure portal configuration.

Description Azure Sentinel has the Zero Trust (TIC 3.0) workbook. (https://devblogs.microsoft.com/azuregov/zero-trust-in-azure-for-government/). This is a spike to determine if we can deploy the ZT workbook using shell scripts and Bicep or both. Manual steps in the Azure portal could also be included if the Sentinel APIs and resource providers do not support the work. If successful, the result will go into the src/bicep/examples folder as an optional deployment.

One of the key outcomes for this work could be having a meaningful example that combines script, Bicep/ARM, and manual Azure portal configuration steps.

Acceptance Criteria

glennmusa commented 3 years ago

FYSA, as of this writing, the AppInsights SDK for Go that backs the azurerm provider does not support Workbook resources:

https://github.com/terraform-providers/terraform-provider-azurerm/issues/5956#issuecomment-757799517 https://github.com/Azure/azure-sdk-for-go/issues/14106

Doesn't mean we cannot create it by other means, but it will end up more source to maintain.

shawngib commented 3 years ago

Happy to lean in here if needed but essentially @glennmusa is correct. Workbooks are hard, even in ARM template since they have the ability to contain strings and characters that are difficult to encode for the proper client being used. In addition, the post linked above by @brooke-hamilton is both a blog and example/preview of doing it not technically a fully supported functionality. TJ even points out in the blog it is limited in its control set coverage.

brooke-hamilton commented 3 years ago

@glennmusa and @shawngib thanks for this info. Based on what you said I removed this issue from the current release backlog and I have reached out to TJ for a discussion on automation mechanisms for the TIC 3.0 workbook. Let's leave this issue in a triage state until we get a firm automation solution.

lisamurphy-msft commented 2 years ago

Can I get some clarification here; from the last correspondence posted Jun 30. What specifically has changed that indicates this needs to be in our prioritized backlog at this time? I will work on researching this issue further.

lisamurphy-msft commented 2 years ago

Determine if Terraform does not support this (TBD) Determine a bicep based approach for implementation

Decompilation failed with fatal error "[169:74]: Unable to pick unique name for variable _ZeroTrustTIC3.0_workbook"

lisamurphy-msft commented 2 years ago

I would be remiss if I didn't include this feature enhancement request: https://github.com/Azure/missionlz/issues/573 presently there is no bicep implementation instructions for Sentinel. Given that this effort is only able to be deployed via bicep, we very likely will need to have implementation instructions for Sentinel included in the examples folder.

brooke-hamilton commented 2 years ago

I would be remiss if I didn't include this feature enhancement request: #573 presently there is no bicep implementation instructions for Sentinel. Given that this effort is only able to be deployed via bicep, we very likely will need to have implementation instructions for Sentinel included in the examples folder.

Sentinel is deployed by the main Bicep deployment when the parameter deploySentinel=true is set. I will update the scope of #573 request clarification of the description in the README for the Terraform deployment of Sentinel. The purpose of that example is to show how to do a Terraform deployment after MLZ has been deployed using Bicep.

lisamurphy-msft commented 2 years ago

Pulling this back into the in-progress workflow since the Sentinel instructions for bicep have been updated as per #613

brooke-hamilton commented 2 years ago

Pulling this back into the in-progress workflow since the Sentinel instructions for bicep have been updated as per #613

@lisamurphy-msft there may be some confusion between PR #613 (and its related backlog item #573). #613 was a documentation issue to clarify the purpose of the Sentinel example that was written in Terraform. This backlog item, #258, is to deploy the Zero Trust workbook into sentinel. #258 and #613/#573 are not related.

thbanasi commented 2 years ago

Recommend adding the following workbooks/analytics/playbooks to the Sentinel instance: Zero Trust (TIC 3.0), Maturity Model for Event Log Management (M-21-31), Insider Risk Management, Threat Intelligence, Threat Analysis & Response (MITRE ATT&CK), CMMC2.0, Azure Security Benchmarkv3, IT/OT Threat Monitoring, NIST SP 800-53

lisamurphy-msft commented 2 years ago

@brooke-hamilton my mistake! It does indeed appear that although clarification in the TF implementation documentation has been updated; we still do not have clear instructions for bicep implementation of Sentinel through MLZ. Although you provided this instruction fairly succinctly in comments; we might need to address this.

lisamurphy-msft commented 2 years ago

@thbanasi Thank you for providing the additional information and insight into this; that is sincerely appreciated. This might be somewhat out of scope for this particular issue, however. Will address with the team on if we need to expand the scope of this issue or if we need another issue as follow-on to address adding additional sentinel workbook instructions.

lisamurphy-msft commented 2 years ago

Able to deploy via the "easy-button" located in the Sentinel Github repository Unable to deploy using the cli and referencing the ARM template source.

Have the following error:

{'code': 'InvalidTemplate', 'message': "Deployment template validation failed: 'The template resource '/Microsoft.SecurityInsights/223f9a0d-5415-4526-a3ad-6cff24c44dbe' for type 'Microsoft.OperationalInsights/workspaces/providers/alertRules' at line '224' and column '77' has incorrect segment lengths. A nested resource type must have identical number of segments as its resource name. A root resource type must have segment length one greater than its resource name. Please see https://aka.ms/arm-template/#resources for usage details.'.", 'additionalInfo': [{'type': 'TemplateViolation', 'info': {'lineNumber': 224, 'linePosition': 77, 'path': 'resources[1].type'}}]}

while completing the equivalent of:

az deployment group create \
--name $deploymentName
--resource-group $resourceGroupName
 --template-uri https://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Solutions/ZeroTrust\(TIC3.0\)/Package/main
Template.json

Investigating

thbanasi commented 2 years ago

This is the template-uri you need: https://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Solutions/ZeroTrust(TIC3.0)/Package/mainTemplate.json

brooke-hamilton commented 2 years ago

I tried this too. When I click the Deploy to Azure button in the Sentinel GitHub repo I get this error in the Azure portal when I click Review + create. I get the same error when trying to deploy using the Azure CLI--the same as @lisamurphy-msft reported.

{"code":"InvalidTemplate","message":"Deployment template validation failed: 'The template resource '/Microsoft.SecurityInsights/0b315195-6b74-4380-b954-dcf2cfe765bd' for type 'Microsoft.OperationalInsights/workspaces/providers/alertRules' at line '224' and column '77' has incorrect segment lengths. A nested resource type must have identical number of segments as its resource name. A root resource type must have segment length one greater than its resource name. Please see https://aka.ms/arm-template/#resources for usage details.'."}

I suggest opening an issue or bug in the Sentinel repo.

lisamurphy-msft commented 2 years ago

Since this is confirmed as not working, I have created the following issue: https://github.com/Azure/Azure-Sentinel/issues/4146

thbanasi commented 2 years ago

I've tested/replicated the error. Make sure you're adding the workspace name + resource group in deployment options image

eschroedMSFT commented 2 years ago

@lisamurphy-msft checking in for latest status.

lisamurphy-msft commented 2 years ago

Pending PR push, appears that what looks like a json validation error was in fact a missing parameter as per the assertion by @thbanasi

thbanasi commented 2 years ago

Team, I submitted the PR for the next version of the Microsoft Sentinel: Zero Trust (TIC 3.0) Solution over the weekend. The new version will be available in GH and Sentinel Content Hub within 2 weeks. I wanted to make sure you guys knew about the updates because there are substantial improvements in content based in customer feedback/learnings. https://github.com/Azure/Azure-Sentinel/pull/4275

lisamurphy-msft commented 2 years ago

@thbanasi thank you for the update!