Closed slavizh closed 8 months ago
Unfortunately it looks like we don't log enough information to identify which module the problem is occurring in. What I can see from the message is that it's occurring in a grandchild module - e.g. with a structure like:
bicepparam
-> module
-> module
-> module <- the problem is in this file, between characters 468 and 520
This may not help narrow it down too much, depending on how complex your deployment is. The easiest option I've found to figure out the character position (which doesn't include line numbers) is to select from the start of the file, and look at this in VSCode:
If that doesn't help figure out where the problem is, I can add some logging and give you a nightly build to get more detail.
@anthony-c-martin I am not seeing any errors. As in order to reproduce this outside of our environment (as I have already reverted to 0.25) I am using WSL with Ubuntu. I have tried running VSC in WSL mode but I am not getting any errors there. Due to that may be nightly bicep file would be better to try on so I can execute the command instead of opening vsc.
@anthony-c-martin may be I have forgot to mention that the issue does not happen only on one template. It happens on multiple ones. I think all that have user defined schema. I have not tested every one of them but I got reported for at least 3 of them. The one mentioned is just one example. Let me know where I can download this debug bicep cli when ready and I will test to see what it shows.
I've submitted a PR which will trigger a nightly build: https://github.com/Azure/bicep/pull/13655. This will add more identifying information to the error to allow us to locate the problematic file + location.
If you want to try it in a build pipeline, you should be able to use the nightly install script. Here's a powershell example:
# <path_to_binary> here should be replaced with the path to where you want bicep.exe to be installed (e.g. C:\foo\bar would result in C:\foo\bar\bicep.exe)
iex "& { $(irm https://aka.ms/bicep/nightly-cli.ps1) } -RunId 8329672150 -BinaryPath <path_to_binary>"
@anthony-c-martin I am having trouble doing this. First I cannot do it on Windows as the problem is visible only on Ubuntu. I have tried:
bash <(curl -Ls https://aka.ms/bicep/nightly-cli.sh) --run-id 8329672150 --binary-path /home/stan
mv: cannot stat '/tmp/tmp.nkOoEz9Xym/bicep': No such file or directory
but got error as you can see.
Strange... does the following work?
gh run download -R Azure/bicep 8329672150 -n "bicep-release-linux-x64" --dir /home/stan
chmod +x /home/stan/bicep
/home/stan/bicep --version
@anthony-c-martin yes. thank you! I think we have the line number. Below is the error generated by the template:
Unhandled exception. System.ArgumentException: Unable to determine parent of specified node of type 'FunctionCallSyntax' at span '[483:535]' because it has not been indexed. Source file: file:///home/stan/modules/key-vault.bicep, line: 15, char: 23. Node contents: union(defaultVirtualNetworkRule, keyVault.network.virtualNetworkRules[i]).
at Bicep.Core.Syntax.SyntaxHierarchy.GetParent(SyntaxBase node)
at Bicep.Core.Semantics.Binder.GetParent(SyntaxBase syntax)
at Bicep.Core.Syntax.ISyntaxHierarchyExtensions.EnumerateAncestorsUpwards(ISyntaxHierarchy hierarchy, SyntaxBase syntax)+MoveNext()
at System.Linq.Enumerable.OfTypeIterator[TResult](IEnumerable)+MoveNext()
at System.Linq.Enumerable.TryGetFirst[TSource](IEnumerable`1, Boolean& )
at System.Linq.Enumerable.FirstOrDefault[TSource](IEnumerable`1 )
at Bicep.Core.Syntax.ISyntaxHierarchyExtensions.GetNearestAncestor[TSyntax](ISyntaxHierarchy hierarchy, SyntaxBase syntax)
at Bicep.Core.TypeSystem.DeclaredTypeManager.IsWithinTypeSyntax(SyntaxBase syntax)
at Bicep.Core.TypeSystem.DeclaredTypeManager.GetTypeAssignment(SyntaxBase syntax)
at Bicep.Core.TypeSystem.DeclaredTypeManager.<GetDeclaredTypeAssignment>b__8_0(SyntaxBase key)
at System.Collections.Concurrent.ConcurrentDictionary`2.GetOrAdd(TKey, Func`2)
at Bicep.Core.TypeSystem.DeclaredTypeManager.GetDeclaredTypeAssignment(SyntaxBase syntax)
at Bicep.Core.TypeSystem.DeclaredTypeManager.GetDeclaredType(SyntaxBase syntax)
at Bicep.Core.TypeSystem.TypeManager.GetDeclaredType(SyntaxBase syntax)
at Bicep.Core.Semantics.SymbolHelper.TryGetSymbolInfo(IBinder binder, Func`2 getDeclaredTypeFunc, SyntaxBase syntax)
at Bicep.Core.Semantics.SemanticModel.GetSymbolInfo(SyntaxBase syntax)
at Bicep.Core.Intermediate.ExpressionBuilder.ConvertPropertyAccess(PropertyAccessSyntax propertyAccess)
at Bicep.Core.Intermediate.ExpressionBuilder.ConvertWithoutLowering(SyntaxBase syntax)
at Bicep.Core.Intermediate.ExpressionBuilder.Convert(SyntaxBase syntax)
at Bicep.Core.Emit.ExpressionConverter.ConvertToIntermediateExpression(SyntaxBase syntax)
at Bicep.Core.Emit.ExpressionConverter.ConvertExpression(SyntaxBase syntax)
at Bicep.Core.Emit.ScopeHelper.FormatFullyQualifiedResourceId(EmitterContext context, ExpressionConverter converter, ScopeData scopeData, String fullyQualifiedType, IEnumerable`1 nameSegments)
at Bicep.Core.Emit.ExpressionConverter.GetFullyQualifiedResourceId(ResourceMetadata resource)
at Bicep.Core.Emit.ExpressionConverter.ConvertResourcePropertyAccess(ResourceReferenceExpression reference, PropertyAccessExpression expression)
at Bicep.Core.Emit.ExpressionConverter.ConvertBaseExpression(AccessExpression expression)
at Bicep.Core.Emit.ExpressionConverter.ProcessAccessExpression(AccessExpression expression)
at Bicep.Core.Emit.ExpressionConverter.ConvertAccessExpression(AccessExpression expression, IEnumerable`1 additionalProperties)
at Bicep.Core.Emit.ExpressionConverter.ConvertExpression(Expression expression)
at Bicep.Core.Emit.ExpressionEmitter.EmitLanguageExpression(Expression expression)
at Bicep.Core.Emit.ExpressionEmitter.EmitExpression(Expression expression)
at Bicep.Core.Emit.ExpressionEmitter.<>c__DisplayClass22_0.<EmitProperty>b__0()
at Bicep.Core.Emit.PositionTrackingJsonTextWriter.WritePropertyWithPosition(IPositionable keyPosition, String name, Action valueFunc)
at Bicep.Core.Emit.ExpressionEmitter.EmitPropertyInternal(LanguageExpression expressionKey, Action valueFunc, IPositionable location, Boolean skipCopyCheck)
at Bicep.Core.Emit.ExpressionEmitter.EmitProperty(Expression name, Expression expression)
at Bicep.Core.Emit.ExpressionEmitter.EmitProperty(String name, Expression expression)
at Bicep.Core.Emit.ExpressionEmitter.EmitObjectProperties(ObjectExpression object)
at Bicep.Core.Emit.ExpressionEmitter.EmitExpression(Expression expression)
at Bicep.Core.Emit.ExpressionEmitter.<>c__DisplayClass22_0.<EmitProperty>b__0()
at Bicep.Core.Emit.PositionTrackingJsonTextWriter.WritePropertyWithPosition(IPositionable keyPosition, String name, Action valueFunc)
at Bicep.Core.Emit.ExpressionEmitter.EmitPropertyInternal(LanguageExpression expressionKey, Action valueFunc, IPositionable location, Boolean skipCopyCheck)
at Bicep.Core.Emit.ExpressionEmitter.EmitProperty(Expression name, Expression expression)
at Bicep.Core.Emit.ExpressionEmitter.EmitProperty(String name, Expression expression)
at Bicep.Core.Emit.ExpressionEmitter.<>c__DisplayClass17_0.<EmitCopyObject>b__1()
at Bicep.Core.Emit.PositionTrackingJsonTextWriter.WriteObjectWithPosition(IPositionable sourcePosition, Action propertiesFunc)
at Bicep.Core.Emit.ExpressionEmitter.EmitCopyObject(String name, Expression forExpression, Expression input, String copyIndexOverride, Nullable`1 batchSize)
at Bicep.Core.Emit.ExpressionEmitter.<>c__DisplayClass18_0.<EmitObjectProperties>b__1()
at Bicep.Core.Emit.PositionTrackingJsonTextWriter.WritePropertyWithPosition(IPositionable keyPosition, String name, Action valueFunc)
at Bicep.Core.Emit.ExpressionEmitter.EmitPropertyInternal(LanguageExpression expressionKey, Action valueFunc, IPositionable location, Boolean skipCopyCheck)
at Bicep.Core.Emit.ExpressionEmitter.EmitCopyProperty(Action valueFunc)
at Bicep.Core.Emit.ExpressionEmitter.EmitObjectProperties(ObjectExpression object)
at Bicep.Core.Emit.ExpressionEmitter.EmitExpression(Expression expression)
at Bicep.Core.Emit.ExpressionEmitter.<>c__DisplayClass22_0.<EmitProperty>b__0()
at Bicep.Core.Emit.PositionTrackingJsonTextWriter.WritePropertyWithPosition(IPositionable keyPosition, String name, Action valueFunc)
at Bicep.Core.Emit.ExpressionEmitter.EmitPropertyInternal(LanguageExpression expressionKey, Action valueFunc, IPositionable location, Boolean skipCopyCheck)
at Bicep.Core.Emit.ExpressionEmitter.EmitProperty(Expression name, Expression expression)
at Bicep.Core.Emit.ExpressionEmitter.EmitProperty(String name, Expression expression)
at Bicep.Core.Emit.ExpressionEmitter.EmitObjectProperties(ObjectExpression object)
at Bicep.Core.Emit.ExpressionEmitter.EmitExpression(Expression expression)
at Bicep.Core.Emit.ExpressionEmitter.<>c__DisplayClass22_0.<EmitProperty>b__0()
at Bicep.Core.Emit.PositionTrackingJsonTextWriter.WritePropertyWithPosition(IPositionable keyPosition, String name, Action valueFunc)
at Bicep.Core.Emit.ExpressionEmitter.EmitPropertyInternal(LanguageExpression expressionKey, Action valueFunc, IPositionable location, Boolean skipCopyCheck)
at Bicep.Core.Emit.ExpressionEmitter.EmitProperty(Expression name, Expression expression)
at Bicep.Core.Emit.ExpressionEmitter.EmitProperty(String name, Expression expression)
at Bicep.Core.Emit.ExpressionEmitter.EmitObjectProperties(ObjectExpression object)
at Bicep.Core.Emit.TemplateWriter.<>c__DisplayClass62_0.<EmitResource>b__0()
at Bicep.Core.Emit.PositionTrackingJsonTextWriter.WriteObjectWithPosition(IPositionable sourcePosition, Action propertiesFunc)
at Bicep.Core.Emit.ExpressionEmitter.EmitObject(Action writePropertiesFunc, IPositionable position)
at Bicep.Core.Emit.TemplateWriter.EmitResource(ExpressionEmitter emitter, DeclaredResourceExpression resource)
at Bicep.Core.Emit.TemplateWriter.<>c__DisplayClass61_0.<EmitResources>b__0()
at Bicep.Core.Emit.PositionTrackingJsonTextWriter.WriteArrayWithPosition(IPositionable sourcePosition, Action itemsFunc)
at Bicep.Core.Emit.ExpressionEmitter.EmitArray(Action writeItemsFunc, IPositionable position)
at Bicep.Core.Emit.ExpressionEmitter.<>c__DisplayClass33_0.<EmitArrayProperty>b__0()
at Bicep.Core.Emit.PositionTrackingJsonTextWriter.WritePropertyWithPosition(IPositionable keyPosition, String name, Action valueFunc)
at Bicep.Core.Emit.ExpressionEmitter.EmitProperty(String propertyName, Action writeValueFunc, IPositionable position)
at Bicep.Core.Emit.ExpressionEmitter.EmitArrayProperty(String propertyName, Action writeItemsFunc, IPositionable position)
at Bicep.Core.Emit.TemplateWriter.EmitResources(PositionTrackingJsonTextWriter jsonWriter, ExpressionEmitter emitter, ImmutableArray`1 resources, ImmutableArray`1 modules)
at Bicep.Core.Emit.TemplateWriter.GenerateTemplateWithoutHash(PositionTrackingJsonTextWriter jsonWriter)
at Bicep.Core.Emit.TemplateWriter.Write(SourceAwareJsonTextWriter writer)
at Bicep.Core.Emit.TemplateWriter.<>c__DisplayClass65_0.<EmitModule>b__2()
at Bicep.Core.Emit.PositionTrackingJsonTextWriter.WriteObjectWithPosition(IPositionable sourcePosition, Action propertiesFunc)
at Bicep.Core.Emit.ExpressionEmitter.EmitObject(Action writePropertiesFunc, IPositionable position)
at Bicep.Core.Emit.ExpressionEmitter.<>c__DisplayClass32_0.<EmitObjectProperty>b__0()
at Bicep.Core.Emit.PositionTrackingJsonTextWriter.WritePropertyWithPosition(IPositionable keyPosition, String name, Action valueFunc)
at Bicep.Core.Emit.ExpressionEmitter.EmitProperty(String propertyName, Action writeValueFunc, IPositionable position)
at Bicep.Core.Emit.ExpressionEmitter.EmitObjectProperty(String propertyName, Action writePropertiesFunc, IPositionable position)
at Bicep.Core.Emit.TemplateWriter.<>c__DisplayClass65_0.<EmitModule>b__0()
at Bicep.Core.Emit.PositionTrackingJsonTextWriter.WriteObjectWithPosition(IPositionable sourcePosition, Action propertiesFunc)
at Bicep.Core.Emit.ExpressionEmitter.EmitObject(Action writePropertiesFunc, IPositionable position)
at Bicep.Core.Emit.TemplateWriter.EmitModule(PositionTrackingJsonTextWriter jsonWriter, DeclaredModuleExpression module, ExpressionEmitter emitter)
at Bicep.Core.Emit.TemplateWriter.<>c__DisplayClass61_0.<EmitResources>b__0()
at Bicep.Core.Emit.PositionTrackingJsonTextWriter.WriteArrayWithPosition(IPositionable sourcePosition, Action itemsFunc)
at Bicep.Core.Emit.ExpressionEmitter.EmitArray(Action writeItemsFunc, IPositionable position)
at Bicep.Core.Emit.ExpressionEmitter.<>c__DisplayClass33_0.<EmitArrayProperty>b__0()
at Bicep.Core.Emit.PositionTrackingJsonTextWriter.WritePropertyWithPosition(IPositionable keyPosition, String name, Action valueFunc)
at Bicep.Core.Emit.ExpressionEmitter.EmitProperty(String propertyName, Action writeValueFunc, IPositionable position)
at Bicep.Core.Emit.ExpressionEmitter.EmitArrayProperty(String propertyName, Action writeItemsFunc, IPositionable position)
at Bicep.Core.Emit.TemplateWriter.EmitResources(PositionTrackingJsonTextWriter jsonWriter, ExpressionEmitter emitter, ImmutableArray`1 resources, ImmutableArray`1 modules)
at Bicep.Core.Emit.TemplateWriter.GenerateTemplateWithoutHash(PositionTrackingJsonTextWriter jsonWriter)
at Bicep.Core.Emit.TemplateWriter.Write(SourceAwareJsonTextWriter writer)
at Bicep.Core.Emit.TemplateWriter.<>c__DisplayClass65_0.<EmitModule>b__2()
at Bicep.Core.Emit.PositionTrackingJsonTextWriter.WriteObjectWithPosition(IPositionable sourcePosition, Action propertiesFunc)
at Bicep.Core.Emit.ExpressionEmitter.EmitObject(Action writePropertiesFunc, IPositionable position)
at Bicep.Core.Emit.ExpressionEmitter.<>c__DisplayClass32_0.<EmitObjectProperty>b__0()
at Bicep.Core.Emit.PositionTrackingJsonTextWriter.WritePropertyWithPosition(IPositionable keyPosition, String name, Action valueFunc)
at Bicep.Core.Emit.ExpressionEmitter.EmitProperty(String propertyName, Action writeValueFunc, IPositionable position)
at Bicep.Core.Emit.ExpressionEmitter.EmitObjectProperty(String propertyName, Action writePropertiesFunc, IPositionable position)
at Bicep.Core.Emit.TemplateWriter.<>c__DisplayClass65_0.<EmitModule>b__0()
at Bicep.Core.Emit.PositionTrackingJsonTextWriter.WriteObjectWithPosition(IPositionable sourcePosition, Action propertiesFunc)
at Bicep.Core.Emit.ExpressionEmitter.EmitObject(Action writePropertiesFunc, IPositionable position)
at Bicep.Core.Emit.TemplateWriter.EmitModule(PositionTrackingJsonTextWriter jsonWriter, DeclaredModuleExpression module, ExpressionEmitter emitter)
at Bicep.Core.Emit.TemplateWriter.<>c__DisplayClass61_2.<EmitResources>b__3()
at Bicep.Core.Emit.PositionTrackingJsonTextWriter.WritePropertyWithPosition(IPositionable keyPosition, String name, Action valueFunc)
at Bicep.Core.Emit.ExpressionEmitter.EmitProperty(String propertyName, Action writeValueFunc, IPositionable position)
at Bicep.Core.Emit.TemplateWriter.<>c__DisplayClass61_0.<EmitResources>b__1()
at Bicep.Core.Emit.PositionTrackingJsonTextWriter.WriteObjectWithPosition(IPositionable sourcePosition, Action propertiesFunc)
at Bicep.Core.Emit.ExpressionEmitter.EmitObject(Action writePropertiesFunc, IPositionable position)
at Bicep.Core.Emit.ExpressionEmitter.<>c__DisplayClass32_0.<EmitObjectProperty>b__0()
at Bicep.Core.Emit.PositionTrackingJsonTextWriter.WritePropertyWithPosition(IPositionable keyPosition, String name, Action valueFunc)
at Bicep.Core.Emit.ExpressionEmitter.EmitProperty(String propertyName, Action writeValueFunc, IPositionable position)
at Bicep.Core.Emit.ExpressionEmitter.EmitObjectProperty(String propertyName, Action writePropertiesFunc, IPositionable position)
at Bicep.Core.Emit.TemplateWriter.EmitResources(PositionTrackingJsonTextWriter jsonWriter, ExpressionEmitter emitter, ImmutableArray`1 resources, ImmutableArray`1 modules)
at Bicep.Core.Emit.TemplateWriter.GenerateTemplateWithoutHash(PositionTrackingJsonTextWriter jsonWriter)
at Bicep.Core.Emit.TemplateWriter.Write(SourceAwareJsonTextWriter writer)
at Bicep.Core.Emit.TemplateEmitter.<>c__DisplayClass8_0.<Emit>b__0()
at Bicep.Core.Emit.TemplateEmitter.EmitOrFail(Func`1 write)
at Bicep.Core.Emit.TemplateEmitter.Emit(TextWriter textWriter)
at Bicep.Core.Emit.CompilationEmitter.Template(SemanticModel model)
at Bicep.Core.Emit.CompilationEmitter.Parameters()
at Bicep.Cli.Services.OutputWriter.ParametersToStdout(Compilation compilation)
at Bicep.Cli.Commands.BuildParamsCommand.RunAsync(BuildParamsArguments args)
at Bicep.Cli.Program.RunAsync(String[] args, CancellationToken cancellationToken)
at Bicep.Cli.Program.<>c__DisplayClass3_0.<<Main>b__0>d.MoveNext()
--- End of stack trace from previous location ---
at Bicep.Cli.Program.RunWithCancellationAsync(Func`2 runFunc)
at Bicep.Cli.Program.Main(String[] args)
at Bicep.Cli.Program.<Main>(String[] args)
Aborted
line 14-22. Perfectly normal syntax that we use on many templates.
resource virtualNetworks 'Microsoft.Network/virtualNetworks@2023-09-01' existing = [for virtualNetworkRule in keyVault.network.virtualNetworkRules: {
name: virtualNetworkRule.virtualNetwork.name
scope: resourceGroup(union(defaultVirtualNetworkRule, virtualNetworkRule).virtualNetwork.subscriptionId, virtualNetworkRule.virtualNetwork.resourceGroup)
}]
resource subnets 'Microsoft.Network/virtualNetworks/subnets@2023-09-01' existing = [for (virtualNetworkRule, i) in keyVault.network.virtualNetworkRules: {
name: virtualNetworkRule.virtualNetwork.subnetName
parent: virtualNetworks[i]
}]
I believe this is a regression in #13546 with the introduction of this check. @jeskew would you mind taking a look?
It would also be good to understand where the nondeterminism (different results on different runs) is coming from - elsewhere we've tried to keep the compilation logic deterministic.
@anthony-c-martin lol. If that is the culprit it is something that I have requested :D. Yeah, it is also strange that it does not errors out on Windows but only on Linux.
I am also facing the same issue in Azur pipelines with Ubuntu agent. It was working till morning and started failing now.
I can see only build agent version image got changed as below
Succeeded agent Current image version: '20240310.1.0' Failed agent Current image version: '20240317.1.0'
Please let us know if anybody have any idea on this.
For a temporary workaround, I would recommend manually reverting back to an older version of Bicep. If you're using Azure CLI, you'll need to run the following before running your AzCLI commands to submit the deployment:
az bicep install --version v0.25.53
Bicep version Bicep CLI version 0.26.54 (5e20b29b58)
Describe the bug This issue started to appear when we upgraded to 0.26.54 yesterday. We run self-hosted agents on Ubuntu 20.04. Strangely I cannot reproduce this issue on my Windows with same Bicep version but I can reproduce it on WSL Ubuntu 20.04 with bicep 0.26.54 installed. I think issue is also only related to using user defined types.
To Reproduce
main.bicep
config.bicepparam
Error that is produced when we try to do bicep build-params and New-AzSubscriptionDeployment
Additionally I get another PowerShell error when I run with new-azsubscriptiondeployment:
As you can see besides the bicep error I also get: Cannot retrieve the dynamic parameters for the cmdlet.
I do not know if the issues are two or one.
The above reproduction is partial template code without user defined types. If needed I can provide the full template with user defined types in private.
Additional context Add any other context about the problem here.