RFC: Consider defining a "maturity path" and stability metadata field

[clintoncwolfe]

🎛 Description

🙋 feature request

We'd like to be able to express the stability of individual resources (and ideally, individual features) in a robust way. This was proposed before in the earlier days of InSpec. With cloud platform support, we see a tension between wanting to offer stable, secure functionality; and using API introspection to rapidly gain broad coverage of cloud platforms. This has come up a few times; most recently on the inspec-gcp project issue 25.

One middle ground might be to allow resources to declare their level of stability.

• experimental Not to be relied on. Missing docs and or tests. May use runtime introspection of remote APIs to increase feature coverage, which makes them inherently unstable. May be listed on the website but otherwise undocumented. Must read source code to understand.
• beta Not to be relied on but improving. May use build-time API introspection. Incomplete docs but getting there. Some tests.
• stable Reliable. Fully documented and tested. Future work focuses on maintaining compatibility while expanding functionality.
• locked Reliable but closed to new features. Accepting bugfixes.
• deprecated Will be removed in next major release. Do not use.

These are just example names. Other names or criteria are welcome to be defined.

Additional behavior:

• warn if experimental resources used. Add CLI option to suppress warning.
• warn if deprecated resources used. Add CLI option to suppress warning.
• Make inspec check aware of stability.
• docs / examples / toolkit for making experimental-level resources for cloud platforms

🌍 InSpec and Platform Version

2.2.10

Attn: @arlimus @jquick @chris-rock @trickyearlobe

Aha! Link: https://chef.aha.io/features/SH-2321

[chris-rock]

I really like the idea of adding the stability index to resources. I think we also need to define a clear path for resource to get into inspec core. e.g. only stable and locked resources should be in core, all other resources should be in resource packs. That allows us to balance velocity/features with stability.

[clintoncwolfe]

I think there is a place in inspec core for experimental resources / features, especially if they are disabled by default or trigger warnings when used. I see several motivations for this:

• Publicity. If people want to know if InSpec supports GCP (for example), they look at inspec, and inspec.io . Having experimental resources listed on the main website and github repo (and clearly marked as such) allows us clearly communicate the breadth of coverage and roadmap.
• Individual Resource Cadence. Things tend to become more stable on a per-resource level, not resource pack. With AWS, for example, aws_iam_user was stable for months before the core merge; and today while in core, there are things that are underdeveloped.
• Migration is painful. Doing a cross-repo merge was not an experience I'd like to make routine :-) .

These aren't so much about "allowing experimental resources in core" as they are about "does the resource-pack development model work smoothly".

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. We value your input and contribution. Please leave a comment if this issue still affects you.

[clintoncwolfe]

I think this important to the ability of user to make decisions about whether to use a resource or not.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. We value your input and contribution. Please leave a comment if this issue still affects you.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. We value your input and contribution. Please leave a comment if this issue still affects you.