DSC Registry Proposal

[michaeltlombardi]

Summary of the new feature / enhancement

Working through the documentation and on the schemas, and considering the end-to-end workflow for developers, users, and integrating tools, I've been thinking about how we can make a DSC Resource manifest a useful source of truth/canonical contract.

One problem we have with the introduction of command-based DSC Resources is for how an entity can discover DSC Resources and pull them for use. We can't rely on a single generic mechanism, because the idiomatic publishing for command-based DSC Resources varies widely depending on the implementation language and supported operating systems.

I think the discover and pull operations can be bifurcated and that the discovery operation can be solved generically. In this proposed model, DSC Resource manifests are published to a registry, while DSC Resources (and/or their modules) are published to a repository, possibly multiple repositories.

Entities can discover available DSC Resources from a particular registry and retrieve their manifest independently of pulling the actual DSC Resource from its defined repository. In this model, the manifest is published to the registry independently. This proposal doesn't extend to enumerating the implementation of repository handling.

With a registry, developers and users could also publish DSC Configuration Documents as reusable artifacts, similar to how PSResourceGet users can publish and fetch scripts and modules independently.

This would make for a standardized way for entities to advertise and discover their reusable DSC Resources and Configuration Documents. Having a standardized model will also make it easier for organizations to manage their own mirrors and private registries.

Proposed technical implementation details (optional)

A DSC Resource registry could be implemented with ORAS or any other standardized package format. For this proposal, I'll use ORAS.

We would define a DSC Resource manifest as a unique artifact type and publish DSC Resources to an OCI Registry as that artifact, ideally with a dsc subcommand, like:

dsc registry publish resource ./foo.resource.json

Which would inspect the manifest and compose an ORAS push (with the Rust client, not the CLI) like:

oras push <RegistryURI>/<Type from manifest>:<Version from manifest> `
  --artifact-type application/vnd.microsoft.dsc.resource `
  foo.resource.json:text/json

DSC could handle adding any necessary annotations to the ORAS manifest, or dynamically adding the DSC Resource's instance schema as a separately-retrievable artifact to the manifest, etc.

In the same way, we would define a DSC Configuration Document as a unique artifact type to enable pushing and pulling those artifacts. While we haven't fully defined a model for DSC Resource modules that contain several resources with one source command (or in the case of DSC Resources implemented in an interpreted language, like Ruby, Python, and PowerShell, a gem or module), we could define a unique artifact type for DSC Resource Modules and publish them with their DSC Resources.

ORAS functionality is available in ACR in all public regions, though it's currently in preview.

Out of scope for this proposal, but technically possible, is actually using ORAS to simultaneously publish a DSC Resource manifest with the command it requires. For command-based DSC Resources, this would obviate the requirement for publishing to separate package repositories, like apt and winget. For PowerShell DSC Resources, which are always implemented in a PowerShell module, the module could be included as a layer artifact.

[michaeltlombardi]

Worth noting as the discussion around whether/how to sign and verify published DSC Resources and configuration documents, tools like cosign can sign ORAS artifacts, SBOMs, and more. Even if we don't use cosign, we should have a think about this earlier in the development cycle for DSC registries.

[Gijsreyn]

@michaeltlombardi Hi Mikey, I was looking through the history and liked what you've written down in this issue.

I really liked the part on potentially publishing configuration documents, essentially giving me a brain dump to take it one notch up. Having the ability to publish configuration documents, in turns, requires some version to be specified. What might bring in other capabilities, is the fact to use the ORAS artifacts, is leveraging the ARM syntax the way they do with Azure Verified Modules (AVM). To illustrate it in some code, take a look at the below example:

# include.oras.config.yaml

$schema: https://raw.githubusercontent.com/PowerShell/DSC/main/schemas/2024/04/config/document.json
resources:
- name: get os info
  type: Microsoft.DSC/Include
  artifact: dscr/public:dsc/security/windows/azure-baseline:<version>
  # Or something in the properties to pass parameter file
  parameters: 
    mode: Development

This brings in easy re-usability and standardization across the DSC community and potentially helps resource authors, to easily publish additional configuration documents to get started.

It's just my first initial brain dump idea and thought it was relevant to this issue to add. If that's not the case, just let me know.