Update on new Module Compliance Tool

[gavindidrichsen]

Module "Compliance" Tool:

What's causing the most pain?

• Puppet "experts"--if they exist in a company--are usually stretched to their limit and "running to stand still". For example, big customers typically have diverse application teams (e.g., database, app, etc) with regular need for updates and all of this must be coordinated somehow by the puppet management team.
• Measuring the performance of our puppet tools is difficult at best and impossible in some situations. The primary reason for this is that in general our tools have not been designed and implemented for observability/telemetry. See Google's OpenTelemetry for reference Google - Site Reliability Engineering .
• Maintenance of puppet code and infrastructure is a "nightmare", especially at scale. Because of this customer's tend to leave it as-is which ultimately makes code changes, refactoring, and upgrades even more difficult.

We need a dependency management tool like other package managers...

How do we start?

If the outer circle below is the fully fledged puppet module dependency management tool (or suite) then the small red circle in the top corner is an MVP that brings value straight away. The beginning here will be some sort of puppet Module Compliance Tool.

Proposed flow for this "Compliance" tool?

Is it syntactically ok?

• pdk validate does all this mostly.

Is its "contract" of dependencies valid?

• .fixtures.yml and unit/acceptance tests will fail if missing dependencies
• metadata.json valid?
• ruby version
• puppet version
• Puppetfile correct?
• bolt-project.yml correct?

Does it actually work with its dependencies?

• Any way to write tests in a module that could be run to verify old vs new resources, deprecated resources, old function names, new function names?  To me knowing what is a puppetlabs/stlib compliant "user" is something that puppetlabs/stlib should know about stlib.am_i_compliant(my_module_that_uses_stdlib)

The flow proposal

[gavindidrichsen]

See this repository for a collection of related repos that may influence or inspire the new tool: https://github.com/gavindidrichsen-puppetlabs/devx_epic_reducing_toil

[davidsandilands]

Ionnais and Paula will be discussing this on next PAB I suggest we might want to take some time on the EAB too

[gavindidrichsen]

@davidsandilands: Moving this into "No Status" until we have more clarity from product.