Develop consistent language for describing module stability

[ben-albrecht]

Many modules include notes about the work-in-progress nature of the module with varying degrees of detail about planned features, changes, and improvements.

These messages are useful in making users aware of current limitations and future changes, but the inconsistency of their usage and level of detail makes it difficult to know how stable (or unstable) modules actually are.

Examples

Here are a few excerpts from 1.17 docs:

List:

This module is expected to change in the future.

Reflection:

There are several ways in which this module could be improved upon:

• the methods here might be better as type methods, so you could use R.numFields() instead of numFields(R).
• getField does not yet return a mutable value.

Barriers:

In the future, we expect to add more language-level support for a "task-team" concept. A task-team will more directly support collective operations such as barriers between the tasks within a team.

The current implementation is designed for correctness, but is not expected to perform well at scale. We expect performance at scale to improve as this barrier implementation is optimized and as the task-team concept is implemented and optimized.

Spawn:

New in version 1.12: Spawn module added.

DistributedBag:

This package module is new in 1.16 and may contain bugs. The interface may change. The documentation is being incrementally revised and improved.

MPI:

This is a prototype of the MPI interface for Chapel. The feature set, configuration support, and documentation are expected to improve over time.

ZMQ:

Chapel's support for ZeroMQ is a work in progress and may not (yet) support the full functionality of the ZeroMQ C API.

Exploring Solutions

It would be more informative to have a consistent system of specifying a module's stability. This could be implemented in a variety of ways.

To propose one solution:

Module stability could be denoted by a numbering or keyword system, with various categories, similar to the idea behind NASA's technology readiness levels. As a starting point for discussion, maybe we would have 3 categories:

1. development - work-in-progress, not well-tested, very little guaranteed
2. unstable - well-tested, but interface may significantly change
3. stable - well-tested and established. Ready for production code.

With a system like this, we could implement Sphinx directives, such as :unstable:, which would insert some consistent boiler-plate note about the stability of the module in the documentation.

[ben-albrecht]

A stability index may also be useful at the granularity of functions/methods and data structures.

[ben-albrecht]

It might make sense to extend this to compiler and language features as well.

[mppf]

We could make --warn-unstable warn if such an "unstable" module is used.

[gbtitus]

It might also be worthwhile to differentiate between interface stability and implementation stability, because the former will impact users more than the latter. In the referenced NASA spec it looks like somewhere above the range 5-7 the software interface rate of change drops off, while change in the implementation doesn't do so until above level 7. I'd say right now the runtime interface is pretty stable. It changes but not very fast, especially in areas that a lot of stuff depends on. But the runtime implementation is changing all the time.

[mppf]

I was expecting that we only cared about interface stability (for these purposes)

[gbtitus]

The Barriers module comments are an example of one that talks more about implementation changes than interface ones.

[mppf]

I wonder if the desired feature there is to specifically communicate about performance problems.

[ben-albrecht]

A boilerplate performance warning for cases where we consider the existing performance to be unacceptable and have a clear path to improving it in the future seems useful. I would consider performance warnings orthogonal to interface stability, however.