NPM package azure-streamanalytics-cicd nested ARM templates

[assaf-grth]

We are using the npm package and receive this error during Azure Devops CICD build: [error]Deployment template validation failed: 'The number of template input parameters limit exceeded. Limit: '256' and actual: '264'. Please see https://aka.ms/arm-template/#parameters for usage details.'.

The stream job that the process creates contains a lot of outputs which causes a very large amount of template parameters, breaking the build

Is there a way of mitigating such issue using this tool? Maybe some additional configuration that can be provided for its execution?

[pengyuli-ms]

Hi @assaf-grth, thanks a lot for your feedback!

We've investigated this issue, and going to fix it by using the nested parameter as described here to avoid the 256 upper limit.

And such fix required some refactoring due to our old design, we need some time and plan to include this fix in our Dec. Release.

As for now, I thought maybe it is possible to use linked template or split this job into two to reduce the number of outputs as a workaround?

[assaf-grth]

Thanks for the update @pengyuli-ms, We might be able to wait, Can you share a more exact date for the release handling this issue?

[pengyuli-ms]

Sure, we're planning to release it by the end of Dec. (we scheduled at mid but might delay a week or so.)

[assaf-grth]

Hi @pengyuli-ms Any updates regarding the fix? Is there a more accurate due date?

[pengyuli-ms]

Hi @assaf-grth, we're sorry to say that this feature might be delayed to next month, due to recent severe COVID situation in our region.

[assaf-grth]

Hope everyone are feeling well.. So the next due date is around middle of next month? Maybe there is a preview version we could start use?

[pengyuli-ms]

Yes, next month release will include this feature for sure, and sorry that we didn't have a preview version available for now, thanks for your understanding!

[assaf-grth]

Hi @pengyuli-ms Any updates regarding the fix?

[pengyuli-ms]

Hi @assaf-grth , we've completed the code development phase for this fix (actually we did a deep refactor to fix this bug and some other issues). Currently we're under test phase, and will release it once done, before Feb. 10th.

[pengyuli-ms]

Hi @assaf-grth , sorry for your waiting, we have released the new version of CI/CD npm package just now, and you could install and take a try. To fix this issue, we added --v2 option for build command to generate new-format ARM template, you could find more detailed help message by using --help. Thanks!

[assaf-grth]

Hi @pengyuli-ms thanks for the update and sorry for the late response

I checked the documentation and didn't see any mention of templates nesting or recommendation for case of more than 256 parameters

Should I use the command line with the addition of --v2 and the nesting support will kick in automatically? Or should I prepare differently to using the new package version?

[pengyuli-ms]

That's correct @assaf-grth , just need to add the --v2 option.

[ruiminwang]

Hi @assaf-grth, does the new npm package with --v2 parameter resolve your problem? We'll close this issue if your problem has been resolved.

[assaf-grth]

Hi @ruiminwang, we are still looking into this issue, in case of further questions I will reopen the ticket as needed Thank you for your assistance so far!

[assaf-grth]

Hi @pengyuli-ms @ruiminwang We had a problem with using the new version of the package We use a project based on VS (with asaproj extension)

When executing the package with this project type, the execution fails with this error

[Error] Failed to generate ARM template for project 'ASA_Job/ASA_Job.asaproj'. Exception details: System.Exception: Failed to load project '..some path...\ASA_Job.asaproj': Unexpected character encountered while parsing value: <. Path '', line 0, position 0.. 

It looks like the new version expects to receive only json files (the VS code project type) and we use the VS project version

Can you advice?

Thanks, Assaf

[pengyuli-ms]

Actually, the VS extension has been deprecated, and only the VS Code extension is currently supported for new features and bug fixes.

Therefore, we recommend using our VS Code extension instead. Migrating to the VS Code extension is quite simple. You just need to install it and export your ASA job from the Portal.

Here is a step-by-step guidance: https://learn.microsoft.com/en-us/azure/stream-analytics/migrate-to-vscode#option-1--export-from-the-azure-portal

[assaf-grth]

Thanks @pengyuli-ms for the update, Is there an official statement about this deprecation? I only found this message here which doesn't mention deprecation directly

[pengyuli-ms]

Sorry for any confusion caused. We will be making an official announcement about the deprecation soon. Thank you for pointing this out to us!