Open premun opened 1 year ago
I made some progress, it's in the branch but it will require quite a lot of work still unfortunately
@radical I returned to this after some time and made good progress to the point where this is something useful maybe even. You can try it out. It's a lot of hacky code assembled together very fast but it does give nice results.
PowerShell on Windows, but will work on other platforms too:
git clone https://github.com/premun/sharpliner
cd sharpliner
git checkout template-tools
cd sharpliner\src\tools\Sharpliner.Tools
dotnet run -- template-api -o RuntimeTemplates.cs `
D:\runtime\eng\pipelines\common\build-coreclr-and-libraries-job.yml `
D:\runtime\eng\pipelines\common\download-artifact-step.yml `
D:\runtime\eng\pipelines\common\download-specific-artifact-step.yml `
D:\runtime\eng\pipelines\common\evaluate-changed-darc-deps.yml `
D:\runtime\eng\pipelines\common\evaluate-changed-paths.yml `
D:\runtime\eng\pipelines\common\evaluate-default-paths.yml `
D:\runtime\eng\pipelines\common\evaluate-paths-job.yml `
D:\runtime\eng\pipelines\common\global-build-job.yml `
D:\runtime\eng\pipelines\common\internal-variables.yml `
D:\runtime\eng\pipelines\common\macos-sign-with-entitlements.yml `
D:\runtime\eng\pipelines\common\perf-variables.yml `
D:\runtime\eng\pipelines\common\platform-matrix-multijob.yml `
D:\runtime\eng\pipelines\common\platform-matrix.yml `
D:\runtime\eng\pipelines\common\restore-internal-tools.yml `
D:\runtime\eng\pipelines\common\upload-artifact-step.yml `
D:\runtime\eng\pipelines\common\upload-intermediate-artifacts-step.yml `
D:\runtime\eng\pipelines\common\variables.yml `
D:\runtime\eng\pipelines\common\xplat-setup.yml
This will produce a file RuntimeTemplates
with the C# APIs that you can include in your pipelines and they will include the right - template: ...
reference.
using System.Collections.Generic;
using Sharpliner.AzureDevOps;
using Sharpliner.AzureDevOps.ConditionedExpressions;
using Sharpliner.AzureDevOps.Tasks;
namespace Pipelines;
public static class RuntimeTemplates
{
// /eng/pipelines/common/download-artifact-step.yml
public static Template<Step> DownloadArtifactStep(
string unpackFolder = "",
bool cleanUnpackFolder = true,
string artifactFileName = "",
string artifactName = "",
string displayName = "") => new("/eng/pipelines/common/download-artifact-step.yml", new()
{
{ "unpackFolder", unpackFolder },
{ "cleanUnpackFolder", cleanUnpackFolder },
{ "artifactFileName", artifactFileName },
{ "artifactName", artifactName },
{ "displayName", displayName },
});
}
You can then reference this somewhere in your C# pipeline:
There are some problems with how the runtime YAML templates are written that make this non-100%. I will explain briefly.
In YAML, there are two ways to define what parameters the template expects.
parameters:
unpackFolder: ''
cleanUnpackFolder: true
artifactFileName: ''
artifactName: ''
displayName: ''
Fully specified parameters with types and other
parameters:
- name: myString
type: string
default: a string
# no default makes it required which is checked by AzDO before the pipeline runs
- name: myBoolean
type: boolean
Unfortunately, Arcade and Runtime both use the first and sometimes they specify in comments what is required. This way, the C# API cannot tell what needs a default value and what doesn't which makes it imperfect. For instance above you can see that all string parameters have default values ""
which is probably not true in how the template behaves.
This also means that we cannot always infer the type, e.g.:
parameters:
jobs: []
As a partial replacement for #150, we could also generate the template APIs from YAML files.
Goal
User will give us YAML directory, we will produce APIs for each template that could be called from C#.
Example
User gives us
install-dotnet.yml
template:We generate something like this:
which would return a YAML like referencing the template (we can get the path and everything), something like: