Azure / bicep

Bicep is a declarative language for describing and deploying Azure resources
MIT License
3.25k stars 754 forks source link

Increase MaxLiteralCharacterLimit for loadTextContent (specifically for non-JSON content) #4293

Open JayDoubleu opened 3 years ago

JayDoubleu commented 3 years ago

Increase MaxLiteralCharacterLimit for loadTextContent https://github.com/Azure/bicep/blob/main/src/Bicep.Core/LanguageConstants.cs#L32

Some of logic app JSON code as well as Azure Workbooks cannot be used with this function due to them being larger than 131072 characters. A lot of these resources would be created via azure portal and then used within arm templates/bicep for deployment. It would be great to have the ability to load it's content from within a separate file.

Kaloszer commented 7 months ago

This might also occur on workbooks in sentinel as they can easily exceed that 131k character quota. Just now I'm exporting some and they're like 145k characters :(. @edit: yup:

Unhandled exception. Azure.Deployments.Core.Exceptions.ExpressionException: The language expression literal limit exceeded. Limit: '131072' and actual: '168361'.
jteves commented 7 months ago

We are using this to load ADX scripts and hitting the max character limit using loadTextContent. string(loadJsonContent()) is not a workable solution for our usecase.

+1 for being blocked loading ADX scripts, too.

woeterman94 commented 2 months ago

Any update on this?

woutervanranst commented 2 months ago

For those hitting this limit when defining APIM API definitions - minifying the openapi YAML and using loadTextContent or loadYamlContent should give you some temporary breathing space

Meertman commented 2 months ago

Our solution ended up being:

  1. Since we split up our Open API specification, because it was becoming troublesome to manage, we use the Redocly CLI (https://redocly.com/docs/cli) for bundling multiple files into one specification file. (Additionally we also perform a linting step to verify the validity of the specification)
  2. We then use Bicep for deploying the API on APIM, but without an API specification and policies (API policy and Operation policies)
  3. Lastly we use the REST API (https://learn.microsoft.com/en-us/rest/api/apimanagement/apis/create-or-update?view=rest-apimanagement-2022-08-01&tabs=HTTP) for importing the Open API specification and PowerShell module (https://learn.microsoft.com/en-us/powershell/module/az.apimanagement/set-azapimanagementpolicy?view=azps-12.3.0) for setting the policies, as this does not seem to have the same length constraints. (Depending on the path of the Open API specification, we update the existing version or import as a new version)

The reason we use the REST API is that this also allows us to provide the additional option to specifying that query parameters should not be translated into a template one. (https://learn.microsoft.com/en-us/rest/api/apimanagement/apis/create-or-update?view=rest-apimanagement-2022-08-01&tabs=HTTP#translaterequiredqueryparametersconduct) As it is not (yet?) supported by the PowerShell module or the Azure CLI. (Note that Bicep does already support this)

But this is a lot of additional complexity that we would have loved to avoid, so if there would be a solution for this, it would be great!

dewolfs commented 1 month ago

I encountered also this problem when importing an openAPI spec into Azure API Management using Bicep using loadTextContent.

miqm commented 1 month ago

I encountered also this problem when importing an openAPI spec into Azure API Management using Bicep using loadTextContent.

You can avoid it by loading openapi spec to object (load json/yaml content) and then serialise object into string-encoded-json using string function.

csharper4life commented 1 week ago

I encountered also this problem when importing an openAPI spec into Azure API Management using Bicep using loadTextContent.

You can avoid it by loading openapi spec to object (load json/yaml content) and then serialise object into string-encoded-json using string function.

Many thanks, this saved the day!

string(loadJsonContent('very-large-open-api-spec.json'))
JayDoubleu commented 1 week ago

Is there an ETA for this? It seems that the change is trivial and affects multiple users, yet this doesn't seem to be on the radar. Am I correct in assuming that this isn't being actioned because the ARM deployment size limit needs to be increased first?

woutervanranst commented 4 days ago

string(loadJsonContent('very-large-open-api-spec.json'))

weirdly enough this works, also for loadYamlContent, thanks