Proposal: manage unstable/experimental/preview features with central "feature flags"

[huonw]

Is your feature request related to a problem? Please describe.

Currently features that aren't ready for prime time have names that indicate this, e.g. experimental_test_shell_command target. environments-preview subsystem, pants.backend.experimental.tools.semgrep backend.

This has good attributes:

• lightweight: adding an experimental feature just requires finding the right way to name it.
• we've indicated what features are less stable/sturdy/tested/complete compared to other features, so users are (implicitly) opting in to the risk this implies

But, it has a few downsides:

• long names (e.g. experimental_test_shell_command vs. test_shell_command, environments-preview vs. environments)
• stabilising the feature requires users update their configuration. This is usually easy enough, and we can and do support the new and old name for a while, but it's just churn...
• it's a bit harder for contributors to get an overview of what's experimental

Describe the solution you'd like

Something like cargo-feature flags in Rust's Cargo: a configuration variable that indicates which unstable features the user has opted in to. (This is also similar to the `#![feature(...)] attribute for unstable feature flags in Rust code itself: 1, 2). In particular, one adds a single element to the list to acknowledge "I understand this is unstable" and then can forge ahead using that feature in Cargo as normal. When the feature is stabilised, the only update required is to remove the element from the list, no matter how many times the code uses the feature in question.

A similar thing could work in Pants:

• a new preview_features_opt_in = [...] GLOBAL option that accepts a list of feature names
• anything experimental requires that its feature is in that list

This leads to a few behaviours:

• If someone uses an experimental feature without including its reference in the list, they get an error telling them to add it.
• Once a feature is stabilised, we can indicate the feature is not required any more, and then users just need to delete the one string from that central list and they're good to go

Example

Suppose we've implemented this feature, and renamed the experimental features above. Then we might have a repo with configuration like:

# pants.toml
[GLOBAL]
pants_version = "2.34.0"

backend_packages = [
  "pants.backend.shell", # provides `test_shell_command` etc.
  "pants.backend.tools.semgrep", # preview! 
]

[environments.names] # preview!
a = "b"

# BUILD
shell_command(...) # from pants.backend.shell, NOT preview
test_shell_command(...) # from pants.backend.shell, preview

Running pants against that configuration might emit errors like:

The pants.backend.experimental.tools.semgrep backend is in preview, and may change without notice in future. To opt-in to this, please include "semgrep" in the [GLOBAL].preview_features_opt_in list. See \ for more details.

The environments feature is in preview, and may change without notice in future. To opt-in to this, please include "environments" in the [GLOBAL].preview_features_opt_in list. See \ for more details.

The test_shell_command target is in preview, and may change without notice in future. To opt-in to this, please include "test_shell_command" in the [GLOBAL].preview_features_opt_in list. See \ for more details.

(The (optional?) link could be to a https://github.com/pantsbuild/pants/labels/stablization tracking issue.)

Then, as prompted, the user could adjust their pants.toml to:

# pants.toml
[GLOBAL]
pants_version = "2.34.0"

preview_features_opt_in = ["semgrep", "environments", "test_shell_command"]

# ... no other changes ...

This is of course normal TOML, so users can add comments with more discussion about each feature as required.

Describe alternatives you've considered

• A different name for the feature, e.g. experimental_features or unstable_features.
• Rust and Cargo only allow using experimental features in nightly builds: we could do something similar, only allow using unstable features in dev releases. I do NOT think we should.
  • This is for extra resistance to de-facto stabilisation (e.g. if someone makes breaking changes to an unstable feature that has been around for years, there may be understandable pushback along the lines of "this has worked fine for ages!! why is it being broken!!": by requiring nightly, the contract with users is clearer: you're on your own).
  • I don't think this is appropriate for Pants: the community is smaller, and it seems better to make it easier for people to experiment with upcoming features.
• Removing the concept of "experiemental"/"preview" entirely. I think this is a bad idea, if we don't have (at least) two levels, I think everything is "stable" by default. This means breaking changes, half-complete features, and major known-issues aren't nearly as tolerable, impacting our ability to iterate/experiment.
• Leaving the distinction as text in docs: I think this is a bad idea too. We should be communicating/reminding users of the status of unstable features when they use the feature, not just in documentation. Docs are too easy to miss, and thus makes de-facto stabilisation more likely.

Additional context

N/A

[sureshjoshi]

I particularly like this idea as it would mean non-breaking changes when features become stable - which is one gripe I have with the current system. As in, a CLI command you used (or a target) might no longer be that thing (automatic migrations aside).

Rust and Cargo only allow using experimental features in nightly builds Yep, HARD pass here - for a language, I can see it. For a library/tool - it would be rough.

What happens when the a feature goes stable, and the "opt_in" flag is still present? I assume it will run, but just mention that flag is redundant?

[benjyw]

I like it a lot! Upvote from me .

[cburroughs]

This isn't directly related, something I've wanted a few times is "experimental flags". I don't think there is anything stopping this today, but it would be nice to have a pattern that was --experimental-super-long-string