Validator is unable to find templates

[jetzerb]

Describe the bug I have a pipeline that references a template YAML file in a subdirectory of my repo. This works fine within Azure DevOps, but when I run the pipeline validator, I get an error

Invalid YAML Pipeline - Could not find /Pipelines/azure-pipelnies/step-get-version-info-git.yml in repository self hosted on https://github.com/ using commit... GitHub reported the error, "Not Found"

To Reproduce Create two YAML files:

• a template with some number of steps
• a build pipeline that references the template

Then run the validator on the pipeline YAML.

Expected behavior If a template is referenced, the validator should look to the local filesystem to find the file, include it, and validate the resulting YAML.

Screenshots

Additional context I see that this extension doesn't run locally, but rather sends the contents of the code editor window to your web service at https://yamlpipelinesvalidator.dev. The documentation around your custom yamlpipelinesvalidator.* settings isn't comprehensive. Perhaps you could update the README with more complete instructions? Seems like I probably need to

• upload my templates to ADO
• fill in the yamlpipelinesvalidator.* settings in VS Code
• open my overall pipeline YAML file in VS Code
• run the validator, which will use the provided URL, PAT, and pipeline id to validate the file

If so, that's unexpected given the current instructions in the README:

[tomaustin700]

Hi @jetzerb. For 99% of use cases users should not need to set PAT, URL and pipeline Id however for more complex pipelines this is required. You're right and validation isn't done locally and is done by a web service which works fine for single yaml file pipelines however templates are unsupported and you would need to provide the settings and commit the template files in order to validate against your pipeline to get that to work. What additions do you think are needed to the documentation?

Templates isn't something I've personally used and I see it getting more popular so you're right and a documentation/readme update should probably be done. Open to suggestions 👍

[jetzerb]

I've just recently started working with ADO build pipelines, but am using templates on my first real attempt. The pipeline that runs when a PR is created is a subset of the pipeline that runs when the PR is completed, so I made templates to hold the re-usable components.

At a minimum, I would suggest that the documentation be updated

• to note that it works only for single-file/self-contained YAML files (to avoid the issue that I ran into)
• to state that the YAML is sent to a 3rd party for validation (which will make a lot of employers nervous)
• with specific use cases / instructions for using the URL/Pat/BuildDef settings (sending a PAT that allows a 3rd party to read source code will definitely be a show-stopper for a lot of employers)

[tomaustin700]

@jetzerb Okay I will update some documentation. Just a few things of note: all of the validation logic is open source and can be seen here, the service is really an intermediary between the extension/site and the Azure DevOps API which does all the validation. Also the PAT only needs permission to Read & Execute build permissions so no need to be concerned about source code or anything like that.

[nspyke]

I've also run into this issue. I create ADO pipelines for a number of customers, and have a standard set of templates which I duplicate into each of my customers pipelines.

[joebiker]

I also use templates. For a number of pipelines: Azure Function, App Services, etc. I was hoping this plugin would save me time checking everything into Azure Dev Ops then executing, aaagghhh.