search

fairlearn / fairlearn-proposals

Proposal Documents for Fairlearn
MIT License
9 stars 6 forks source link

Proposal for Backwards Compatibility & Releases #11

Open riedgar-ms opened 4 years ago

riedgar-ms commented 4 years ago

We have been very cavalier with backwards compatibility to date. This is a proposal to address that, and provide some more stability to our users, while recognising that we're still early-stage.

kevinrobinson commented 4 years ago

These all seem like good ideas to me! My two cents from the outside would be to focus more on communicating about the project's status, scope and intentions. That might be as minimal as writing a paragraph or three bullets at the top of the README about what the team considers stable, and what it intends to focus on doing in the next 3-6 months. That might be more immediately valuable.

Similarly, I'd vote for removing code, guides and examples that are known to be incomplete, not representative of values in https://github.com/fairlearn/fairlearn/pull/436, etc. That may help elevate the role of improving the documentation, user guides and examples which may be a part of goals that the team has for project growth and adoption. Helping redpeople update version numbers seems good, but adding a changelog can point out the big things. A more immediate problem might be improving first-time encounters with the project so as to avoid things like https://github.com/fairlearn/fairlearn/issues/448.

riedgar-ms commented 4 years ago

@kevinrobinson , can you explain what you mean by "adding a changelog can point out the big things?" We already have a Changes.md file; do you mean something else?

Thank you for hopping in on https://github.com/fairlearn/fairlearn/issues/448 (memo to self: look at the issue list more frequently). We have a build as part of the PR gate which highlights those breaks, since we've been caught by that before. I'm not sure if it was green for a full week after I put out v0.4.6, though :-(

kevinrobinson commented 4 years ago

@riedgar-ms sure, no problem!

re: changelog yep, having a CHANGES.md seems great! In the scenario where a user is thinking "I am using this library, and want to understand what will break, and how I need to migrate when I upgrade," that may involve flipping the perspective a bit. In other words, if you're trying to support users in understanding the stability and changes in the library, it may help to revise the changelog away from a developer-centric list of work that has been done, and more towards a user-centric communication about what has changed, why it has changed, and what they can do about it.

riedgar-ms commented 4 years ago

Understood. We do have careful instructions for migrating from v0.2 (which had an even smaller userbase), but we haven't done that since. Your point that we should be doing so is a good one!

riedgar-ms commented 4 years ago

@kevinrobinson , have updated with a note about making sure Changes.md is useful for users and not just developers. Specifically by including migration information for breaking changes.

romanlutz commented 4 years ago

Understood. We do have careful instructions for migrating from v0.2 (which had an even smaller userbase), but we haven't done that since. Your point that we should be doing so is a good one!

Taking a step back, I think CHANGES.md has its place that makes sense. I don't mind additionally adding context on how to upgrade as @kevinrobinson describes. I'm not sure whether this should be mixed because it may make the document less useful for both developers and users.

riedgar-ms commented 4 years ago

Taking a step back, I think CHANGES.md has its place that makes sense. I don't mind additionally adding context on how to upgrade as @kevinrobinson describes. I'm not sure whether this should be mixed because it may make the document less useful for both developers and users.

Do you mean a Migration.md (say) which would be pointed to from the Changes.md @romanlutz ? I agree that the migration instructions could get long for the Changes.md - although that would be an incentive to avoid breaking changes :-)

romanlutz commented 4 years ago

Taking a step back, I think CHANGES.md has its place that makes sense. I don't mind additionally adding context on how to upgrade as @kevinrobinson describes. I'm not sure whether this should be mixed because it may make the document less useful for both developers and users.

Do you mean a Migration.md (say) which would be pointed to from the Changes.md @romanlutz ? I agree that the migration instructions could get long for the Changes.md - although that would be an incentive to avoid breaking changes :-)

Rather a ReST file that’s part of the webpage. It can have upgrade instructions for every version. We can link to it from anywhere we want.

adrinjalali commented 4 years ago

having a detailed changelog and a highlights post which could also include how to migrate separately is usually appreciated by users.