Expand field() expressions for Azure Policy

ArmaanMcleod commented 2 years ago

In Azure Policy, we can have field expressions like field('type'). We should be able to expand these expressions when emitting JSON rules.

Related to #181

BernieWhite commented 1 year ago

Repoduction:

Export-AzPolicyAssignmentRuleData -AssignmentFile .\policy\nnn.assignment.json -OutputPath .\policy-rule\

Some additional information about these errors when running Export-AzPolicyAssignmentRuleData includes:

Export-AzPolicyAssignmentRuleData: An error occurred evaluating expression '[field('type')]' line 63. The function "field" was not found.
Export-AzPolicyAssignmentRuleData: An error occurred evaluating expression '[first(split(field('fullName'), '/'))]' line 1066. The function "field" was not found.

Additional error stack:

xception             :
    Type           : PSRule.Rules.Azure.Data.Template.ExpressionEvaluationException
    Expression     : [field('name')]
    TargetSite     :
        Name          : EvaluateExpression
        DeclaringType : PSRule.Rules.Azure.Data.Template.TemplateVisitor, Microsoft.PSRule.Rules.Azure.Core, Version=0.0.1.0, Culture=neutral, PublicKeyToken=null
        MemberType    : Method
        Module        : Microsoft.PSRule.Rules.Azure.Core.dll
    Message        : An error occurred evaluating expression '[field('name')]' line 34403. The function "field" was not found.
    InnerException :
        Type       : System.NotImplementedException
        TargetSite :
            Name          : Element
            DeclaringType : PSRule.Rules.Azure.Data.Template.ExpressionBuilder, Microsoft.PSRule.Rules.Azure.Core, Version=0.0.1.0, Culture=neutral, PublicKeyToken=null
            MemberType    : Method
            Module        : Microsoft.PSRule.Rules.Azure.Core.dll
        Message    : The function "field" was not found.
        Source     : Microsoft.PSRule.Rules.Azure.Core
        HResult    : -2147467263
        StackTrace :
   at PSRule.Rules.Azure.Data.Template.ExpressionBuilder.Element(TokenStream stream, ExpressionToken element) in
C:\Dev\Workspace\PSRule.Rules.Azure\src\PSRule.Rules.Azure\Data\Template\ExpressionBuilder.cs:line 61
   at PSRule.Rules.Azure.Data.Template.ExpressionBuilder.Lexer(TokenStream stream) in C:\Dev\Workspace\PSRule.Rules.Azure\src\PSRule.Rules.Azure\Data\Template\ExpressionBuilder.cs:line 42
   at PSRule.Rules.Azure.Data.Template.ExpressionBuilder.Build(String s) in C:\Dev\Workspace\PSRule.Rules.Azure\src\PSRule.Rules.Azure\Data\Template\ExpressionBuilder.cs:line 31
   at PSRule.Rules.Azure.Data.Policy.PolicyAssignmentVisitor.PolicyAssignmentContext.BuildExpression(String expression) in
C:\Dev\Workspace\PSRule.Rules.Azure\src\PSRule.Rules.Azure\Data\Policy\PolicyAssignmentVisitor.cs:line 114
   at PSRule.Rules.Azure.Data.Template.TemplateVisitor.<>c__DisplayClass80_0`1.<Expression>b__0() in
C:\Dev\Workspace\PSRule.Rules.Azure\src\PSRule.Rules.Azure\Data\Template\TemplateVisitor.cs:line 1345
   at PSRule.Rules.Azure.Data.Template.TemplateVisitor.EvaluateExpression[T](ITemplateContext context, String value, IJsonLineInfo lineInfo) in
C:\Dev\Workspace\PSRule.Rules.Azure\src\PSRule.Rules.Azure\Data\Template\TemplateVisitor.cs:line 1671
    Source         : Microsoft.PSRule.Rules.Azure.Core
    HResult        : -2146233088
    StackTrace     :
   at PSRule.Rules.Azure.Data.Template.TemplateVisitor.EvaluateExpression[T](ITemplateContext context, String value, IJsonLineInfo lineInfo) in
C:\Dev\Workspace\PSRule.Rules.Azure\src\PSRule.Rules.Azure\Data\Template\TemplateVisitor.cs:line 1675
   at PSRule.Rules.Azure.Data.Template.TemplateVisitor.EvaluateExpression[T](ITemplateContext context, JToken value) in
C:\Dev\Workspace\PSRule.Rules.Azure\src\PSRule.Rules.Azure\Data\Template\TemplateVisitor.cs:line 1660
   at PSRule.Rules.Azure.Data.Template.TemplateVisitor.ExpandPropertyToken(ITemplateContext context, JToken value) in
C:\Dev\Workspace\PSRule.Rules.Azure\src\PSRule.Rules.Azure\Data\Template\TemplateVisitor.cs:line 1377
   at PSRule.Rules.Azure.Data.Policy.PolicyAssignmentVisitor.PolicyAssignmentContext.ExpandPolicyRule(JToken policyRule) in
C:\Dev\Workspace\PSRule.Rules.Azure\src\PSRule.Rules.Azure\Data\Policy\PolicyAssignmentVisitor.cs:line 536
   at PSRule.Rules.Azure.Data.Policy.PolicyAssignmentVisitor.PolicyAssignmentContext.ExpandPolicyRule(JToken policyRule) in
C:\Dev\Workspace\PSRule.Rules.Azure\src\PSRule.Rules.Azure\Data\Policy\PolicyAssignmentVisitor.cs:line 542
   at PSRule.Rules.Azure.Data.Policy.PolicyAssignmentVisitor.PolicyAssignmentContext.ExpandPolicyRule(JToken policyRule) in
C:\Dev\Workspace\PSRule.Rules.Azure\src\PSRule.Rules.Azure\Data\Policy\PolicyAssignmentVisitor.cs:line 542
   at PSRule.Rules.Azure.Data.Policy.PolicyAssignmentVisitor.PolicyAssignmentContext.AddDefinition(JObject definition, String definitionId) in
C:\Dev\Workspace\PSRule.Rules.Azure\src\PSRule.Rules.Azure\Data\Policy\PolicyAssignmentVisitor.cs:line 156
   at PSRule.Rules.Azure.Data.Policy.PolicyAssignmentVisitor.VisitDefinitions(PolicyAssignmentContext context, IEnumerable`1 definitions) in
C:\Dev\Workspace\PSRule.Rules.Azure\src\PSRule.Rules.Azure\Data\Policy\PolicyAssignmentVisitor.cs:line 687
   at PSRule.Rules.Azure.Data.Policy.PolicyAssignmentVisitor.Assignment(PolicyAssignmentContext context, JObject assignment) in
C:\Dev\Workspace\PSRule.Rules.Azure\src\PSRule.Rules.Azure\Data\Policy\PolicyAssignmentVisitor.cs:line 706
   at PSRule.Rules.Azure.Data.Policy.PolicyAssignmentVisitor.Visit(PolicyAssignmentContext context, JObject assignment) in
C:\Dev\Workspace\PSRule.Rules.Azure\src\PSRule.Rules.Azure\Data\Policy\PolicyAssignmentVisitor.cs:line 665
   at PSRule.Rules.Azure.Data.Policy.PolicyAssignmentHelper.ProcessAssignment(String assignmentFile, PolicyAssignmentContext& assignmentContext) in
C:\Dev\Workspace\PSRule.Rules.Azure\src\PSRule.Rules.Azure\Data\Policy\PolicyAssignmentHelper.cs:line 50
CategoryInfo          : NotSpecified: ({
                        "identity": {…7a7"
                        }
                        ]
                        }:JObject) [Export-AzPolicyAssignmentRuleData], ExpressionEvaluationException
FullyQualifiedErrorId : System.NotImplementedException,Export-AzPolicyAssignmentRuleData
InvocationInfo        :
    MyCommand        : Export-AzPolicyAssignmentRuleData
    ScriptLineNumber : 1
    OffsetInLine     : 1
    HistoryId        : 5
    Line             : Export-AzPolicyAssignmentRuleData -AssignmentFile .\policy\d6d94deb-09b7-40cb-b64f-e63d1beda4f2.assignment.json -OutputPath .\policy-rule\
    PositionMessage  : At line:1 char:1
                       + Export-AzPolicyAssignmentRuleData -AssignmentFile .\policy\d6d94deb-0 …
                       + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    InvocationName   : Export-AzPolicyAssignmentRuleData
    CommandOrigin    : Internal
ScriptStackTrace      : at Export-AzPolicyAssignmentRuleData<Process>, C:\Dev\Workspace\PSRule.Rules.Azure\out\modules\PSRule.Rules.Azure\PSRule.Rules.Azure.psm1: line 421
                        at <ScriptBlock>, <No file>: line 1
PipelineIterationInfo :

BernieWhite commented 1 year ago

Simple cases such as [field('type')] can be converted to a PSRule object path.

Complex cases will need additional complex expression support upstream in PSRule. For example:

[first(split(field('fullName'), '/'))]
[if(equals(field('Microsoft.Compute/virtualMachines/storageProfile.osDisk.osType'), 'Windows'), field('Microsoft.Compute/virtualMachines/osProfile.windowsConfiguration.patchSettings.assessmentMode'), field('Microsoft.Compute/virtualMachines/osProfile.linuxConfiguration.patchSettings.assessmentMode'))]
[empty(field('Microsoft.Network/virtualNetworks/subnets[*].ipConfigurations[*].id'))]
[concat('/subscriptions/', subscription().subscriptionId, '/resourceGroups/*/providers/Microsoft.Network/virtualNetworks/', first(split(field('fullName'), '/')), '/subnets/AzureFirewallSubnet')]

Marc013 commented 6 months ago

@BernieWhite, Can you please indicate whether there is any attention on this issue? Currently, this issue prevents 100% coverage (currently reaching ~75%) for creating rules based on assigned Azure policy initiatives.

It would be great when issue The function "field" was not found. would be solved as soon as possible.

BernieWhite commented 5 months ago

Hi @Marc013, yes we're making progress on field cases. But if you have specific cases that are failing we'd like to know about them as separate issues so we can directly track and close them.

Specifically if you have failing built-in policies, please post their resource IDs, so we can add them to tests. i.e. /providers/Microsoft.Authorization/policyDefinitions/bef3f64c-5290-43b7-85b0-9b254eef4c47

If you have custom policies that are failing, we would appreciate original policy JSON with placeholders for anything sensitive and error message.

Marc013 commented 5 months ago

@BernieWhite I've created the bugs as requested and referenced this issue. Please reach out when any more information is required.

BernieWhite commented 5 months ago

Thanks @Marc013

Azure / PSRule.Rules.Azure

Expand field() expressions for Azure Policy #1323