Concerns with the Project Maturity Model

[RLittlesII]

1. Communication of Project Maturity Model

   Myself, and other members of the Dotnet Foundation (DNF) were genuinely surprised by this development. I found out on twitter. I notified other members, who are OSS maintainers as well, and they were shocked. This speaks to a larger issue that the project leaders are doing a big amount of work, with minimal assistance from the foundation (@devlead). It seems the foundation is making assertions in the best interest of those maintainers without asking for input. This is a communication break down.

2. Proposal of SLA's?

   The verbiage reads like the next logical step is SLA's. For consumers, SLA's are great. For maintainers they are not. What I forsee happening is SLA's will drive maintainers away, and without a healthy community of contribution around a given project, that project will die and the consumers won't have anyone to provide the SLA's they've come to expect. The community (which includes maintainers much to some peoples dismay) would benefit greatly from a better understanding of how open source software works. Nothing here talks about consumer commitment to open source, which to me is the biggest piece excluded from the .NET OSS narrative.

   To be clear: I think that the DNF proposing SLA's for any of it's projects is a bad idea.

3. Issues currently faced by the .NET OSS community

   We (ReactiveUI) struggled with getting support in the past for our projects from the DNF, so it feels like they aren’t holding themselves accountable to their purpose. ReactiveUI was one of the early adopted projects in the DNF. We've struggled with lack of communication from board members, no major promotion past spots in the newsletter. DNF project leads have been forth coming with our criticism and requests for what we feel would be helpful.

   Our consumers suffer when we haven't had time to look at a bug report. Our consumers suffer when our builds are on shared resources. Our consumers suffer when the documentation is lacking. Our consumers suffer because we don't have the resources to help them resolve issues in a timely enough manner. Consumer suffering is really a result of Maintainer suffering. It isn't that our desire isn't there. It's that our resources, community contributions and time are fixed.

4. DNF Mission

   The DNF seems to be getting away from maintainer needs of funding, internal product managers, grooming of contributors. I was under the impression the whole point of the DNF was to help maintainers, which helps consumers. This begs the question, how does this project maturity model benefit the project? It's appears to be written with a consumer focus. I'm of the thought that helping project maintainers is the most direct way to help the project consumers. So where is the sell for a project to buy into this model? It's clear the consumer wins, do the projects have to lose for consumers to win?

5. Ecosystem

   I see a lot of use of the term Ecosystem. Which ecosystem are we talking about? The Microsoft Ecosystem? The .NET OSS Ecosystem? Or the Ecosystem that will be managed and maintained by the DNF based on this model being proposed? These are all different worlds.

   • Ecosystem tiers have different concerns
     • Corporate Backed Open Source
     • Organization Backed Open Source
     • Individual Project Open Source

6. Proposal: Maturity Ladder

   • Level 1 - Any dual licensed software automatically is not eligble. How a product is licensed shouldn't dictate maturity. (@daveaglick)
   • Level 3 - We required you to join the DNF for us to promote your hard work, or consider you mature feels immediately exclusive and against the mission
   • This model level seems to be introducing a project tier without the appropriate foundation tiers to match. (@devlead)
   • If the maturity ladder will keep projects from the promise of DNF help, the consumers of those projects won't benefit from the efforts of the DNF.
   • Very little of the verbiage in this "Project Maturity Model" coincides with the DNF mission.

7. Proposal: Project Forge

   • This seems like a proposal to kill "free enterprise" in the OSS space.
   • I feel this would be better served as an advisory board rather than the proposed bureaucratic oversight committee.
   • I think it's hard to state that a project directly out the gate is worthy of inclusion in the "core framework". Which it sounds like the Project Forge is just a staging ground for things that will eventually become Microsoft operated.
   • The forge should be a place that helps any project that adds value to the community get off the ground.

8. Proposal: Maintainer "Bench" Program

   • I think we should look at it as a "Contribution Encouragement" program.
   • People who have the passion will move into maintaining (speaking from personal experience).
   • We should focus on creating a good pool of contributors, so that we have people on the "bench" as it were.
   • OSS continuity and project succession is the focus here, how do we incentivize users to become contributors, and contributors to become maintainers?

9. Benefits

   • For Maintainers
     • Adoption is generally about visibility. Contribution has little to do with maturity or quality of the project, and much to do with the willingness of the individual developer to roll up their sleeves.
   • For contributors
     • Hand hold them through the process, make them feel comfortable. Remove barriers to contribution.
   • For users
     • The only thing that makes sense here is the security. People select libraries because their application has a need that is outside of it's core purpose. They don't just go to nuget and see trends and add libraries because they are popular.

10. OSS Contract

    "I built it. You consume it. We maintain it."

I believe we ultimately want the same thing. I think the approach is a good first step that could use a bit of help from maintainers that are currently producing projects of these levels. We have an internal core of projects across the maturity model, we should use their experiences as a basis for how best to help other projects. This will help drive a better consumer experience.

About the DotNet Foundation

The .NET Foundation supports .NET open source in a number of ways.

- Promote the broad spectrum of software available to .NET developers through NuGet.org, GitHub, and other venues.
- Advocate for the needs of .NET open source developers in the community.
- Evangelize the benefits of the .NET platform to a wider community of developers.
- Promote the benefits of the open source model to developers already using .NET.
- Offer administrative support for a number of .NET open source projects.

[haacked]

Thanks for the thoughtful feedback! The following are some of my thoughts on the matter. While a member of the board, I don't think I speak for the board in these matters as I haven't had time to really understand where everyone sits on these issues. But I hope this is helpful to hear my perspective. :smile:

It seems the foundation is making assertions in the best interest of those maintainers without asking for input. This is a communication break down.

To be fair, this repository is that request for input. With any major proposal, we tend to gradually expand the input. It starts with a proposal to the board, then we brought in a few maintainers to bounce the idea off of for obvious issues before posting it more broadly.

I tend to think proposing a new thing never lands well with everybody, but there's also always room for improvement. I am open to ideas on how better to handle such a proposal next time. We could define a more structured RFC process for example. It's food for thought.

Proposal of SLA's?

As far as I know, there is no SLA proposal in the ladder nor any plans to suggest it. I might be wrong, but until it is, we don't need to argue it. But it is helpful to know your position on it.

Issues currently faced by the .NET OSS community

Keep in mind there are multiple advisory groups. I believe this ladder is part of the Technical Review group. Each action group is working through their charters at different paces. Effort in one group doesn't come at the expense of the efforts of the other groups. I expect you'll hear more from other groups throughout the year.

I hear your frustrations though and they are valid. It might help to jump into the appropriate action group (Probably Project Support) and post in their discussions to give them a kick in the pants. Everyone on the board is holding down a full-time job (just as you are while contributing to OSS) and some of the efforts are progressing slowly and dependent on volunteers to really help push it along.

Any dual licensed software automatically is not eligible. How a product is licensed shouldn't dictate maturity.

That's not true. If one of the licenses fits the ladder, then the project is eligible. When you look at the ladder through the lens of judging the risk of depending on a project, if one of the licenses is permissive, then the risk is low. This should probably be clarified /cc @dotnet-foundation/technical-review @richlander

This seems like a proposal to kill "free enterprise" in the OSS space.

I read it very differently. It looks like the primary purpose is to identify gaps and encourage projects to fill that along with support. Perhaps that can be cleared up a bit to alleviate this concern.

We have an internal core of projects across the maturity model, we should use their experiences as a basis for how best to help other projects.

Totally agree. That's why this is still in the proposal state and soliciting feedback at the moment.

[glennawatson]

To be fair, this repository is that request for input. With any major proposal, we tend to gradually expand the input. It starts with a proposal to the board, then we brought in a few maintainers to bounce the idea off of for obvious issues before posting it more broadly.

I think what caught some of us by surprise was no notice sent to the leadership Slack or email list asking for feedback. I think this is where we felt the communication breakdown happened.

[mitchelsellers]

I would second @glennawatson on this one.

For us, there are items in the maturity model that will be almost impossible for the DNN Platform to achieve, which I'm more than willing to discuss.

[haacked]

I think what caught some of us by surprise was no notice sent to the leadership Slack or email list asking for feedback. I think this is where we felt the communication breakdown happened.

Very fair. I'll bring that up with the others. It wouldn't hurt us to have a set of guidelines for unrolling new proposals.

[haacked]

For us, there are items in the maturity model that will be almost impossible for the DNN Platform to achieve, which I'm more than willing to discuss.

Definitely interested in hearing those things. Might be useful to open an issue for each one if you haven't already.

[RLittlesII]

Thanks for the feedback @haacked some of this information is new to me. I am not really sure how things operate. There hasn't been a "Welcome to the DNF, this is how we do business." that I've seen.

Communication is the key to any relationship. In most cases we are at the front lines of what the DNF exists for, the promotion of .NET Open Source Software. So when we find out when the general public does, it seems odd.

To be fair, this repository is that request for input.

I wasn't aware this was the request. I saw this and thought it was the directive.

I tend to think proposing a new thing never lands well with everybody, but there's also always room for improvement.

This is why talking to project leads about things like these before it hits the public is a good thing. We can help groom it as we interact with the consumers of our given projects and hopefully have a pulse of what they want/need from us. To be fair, you can never please everyone. I don't expect to be agree with everything the DNF proposes.

Everyone on the board is holding down a full-time job (just as you are while contributing to OSS) and some of the efforts are progressing slowly and dependent on volunteers to really help push it along.

I get this. But this isn't something that was placed on us. We asked for this burden. We should be trying to find ways to make the blockers keeping us from doing work towards the goal non existent. We being all members of the DNF.

That's not true. If one of the licenses fits the ladder, then the project is eligible.

This is good information and good news. We should update the model to make this more clear, as I know I am not the only one who read it that way.

I read it very differently. It looks like the primary purpose is to identify gaps and encourage projects to fill that along with support.

Then it is ambiguous. I think some of us are still learning to work with the new Microsoft and trying not to let our previous encounters affect our current interactions. When I read it, it sounded more like a system of control. It's easy enough to modify the tone so its clear that it's advisory and not a ruling body. More so, that the help is available whether it came from the Forge or popped up organically.

[pascalberger]

To be fair, this repository is that request for input. With any major proposal, we tend to gradually expand the input. It starts with a proposal to the board, then we brought in a few maintainers to bounce the idea off of for obvious issues before posting it more broadly.

I was actually also a little bit taken by surprise by this, especially as it was directly announced publicly at .NET Conf and mentions :

This proposal was developed by a .NET Foundation working group, based on observed challenges and missed opportunities in the ecosystem.

Are .NET Foundation working groups the same as action groups or something different? Or does the working group only contain board members? I'm missing here the transperancy from .NET Foundation, as I'm also signed up to some action groups, but beside being invited to some GitHub teams, there was never any communication how they are supposed to work and communicate.

On the proposal it also mentions:

reaching out to working group by mail

Does this mean that working groups / actions groups can be contacted by email? Where are these mails going to and where are the actual addresses listed?

[richlander]

There is discussion about "dual license" models, and whether they can participate in the model. It all depends what the term "dual license" is intended to mean.

My definition of "dual license" has always been that the software is offered under multiple licenses and you get to pick the one that works for you. For example, you can license as LGPL and MIT. That works with this model, as long as one of the licenses is a permissive license.

I have learned a new definition of "dual license", which is that there are a set of characteristics, and you can get a partial set for free, or the complete set if you pay. License Zero is an example of such a license. It is not compatible with the ladder. I explain this in more detail @ https://github.com/dotnet-foundation/project-maturity-model/issues/16.

[jongalloway]

Thanks @haacked and @richlander for clarifying above.

Communication

This was absolutely intended as a request for comment. We put a lot of work into getting a good first draft together to have something concrete to comment on, and we did review with the .NET Foundation Advisory Council, .NET Foundation Board, and some pilot project leaders several times. We should have given a heads up to the project leaders, and missed that. We've had a bit going on, with putting this draft together, .NET Core 3 release, .NET Conf, 215 worldwide local events, etc., etc., but this is great feedback on better communication, and we'll learn from it. I'm sorry.

Issues in this repo

We've got a few mega "I have concerns..." issues that are turning into long, unfocused discussions. I think we probably need one general issue at the most, and would like to see some specific, actionable issues. For instance, in the original issue we've got a long bulleted list and no hope of actually arriving at a conclusion and closing this issue; a few specific issues titled things like "Remove SLA requirement" would be more actionable.

.NET Foundation Leadership Structure

The project leaders don't have a formal place in the .NET Foundation leadership structure, which led to the above communication issue. Of course they should, they're community leaders, invested in the .NET Foundation's success, affected by policy and service changes, etc.

The current structure includes a community elected board of directors, an executive director who reports to them and does the day to day work, two advisory groups (the Advisory Council and TSG / Corporate Sponsor group), and our new Action Groups. Before setting up the project leader slack, we didn't have a group discussion between project leaders, and they didn't operate as a body. Now that we have a cohesive group of project leaders, we need to figure out how to make that more official, have regular meetings, etc.

So from the project leader perspective, it may seem baffling that we didn't run this by all of you. From an organizational perspective, we reviewed with the official leadership structure several times. Again, this was a miss on our part, and I apologize.

Concerns on motivation

Perhaps since this was a surprise and required a lot of reading to fully understand, coupled with some general distrust of Microsoft and open source that we'll hopefully outgrow before we settle Mars, there have been some concerns as to why we're doing this and whose idea it was.

This is an attempt to solve some obvious problems in the .NET open source community. We had some board members campaign and win a board seat with statements like "I’d like the .NET ecosystem to thrive with projects people consider and take dependencies on to be much wider than the narrow “Microsoft-blessed” scope that people and companies generally consider." At our first board meeting, I asked the board to come up with some specific areas and action groups, and this was one of the seven that made the list. I had other things I'd expected to see on the list, but this wasn't one of them.

Separately, I've had discussions with .NET PM's for years about how we could actually ship non-Microsoft libraries in new project templates, include them in documentation, etc., in a way that meets the quality level that many require from Microsoft products, e.g. security reporting standards, servicability, assurance that the packages don't include injected malware, assurance we don't have "leftpad" like package deletion issues, etc. There have been multiple attempts to try to come up with policies and programs to more effectively integrate with community packages. @richlander brought up some very thoughtful ideas based on what he's seen both in working on the .NET team and from actually helping set up the .NET Foundation 5 years ago. Part of the goal was to turn Microsoft .NET team internal requirements and systems and policies into something other projects could opt into.

So we worked together to come up with a first draft. It was always with the plan that this had to be completely owned and run by the .NET Foundation, not Microsoft or any other company, as an independent, community run organization. We also planned to publish it as a solid first draft, but something we'd build in the open. Getting to that first draft included a PR with 200 comments. We wanted to get our thoughts down in a cohesive way to present to the community, rather than just a couple bullet points.

Speaking for myself, I've personally been frustrated over the years that a lot of .NET development shops, presentations, tutorials, documentation, training, etc. sticks very closely to what Microsoft ships because it's safe - it might not be the best, but it's widely distributed, got some assurance it'll be around and supported for a while, will get security fixes, will survive if the project lead runs off to join the circus, etc. As a developer, I want it all: I want a durable ecosystem, and I want a lot more than just one company churns out. So in the months leading up to this RFC, I was motivated to spend nights and weekends (in addition to daylight hours) working on this, because I think it's a good start on an idea that solves some real problems and gets us to the next level.

[haacked]

There hasn't been a "Welcome to the DNF, this is how we do business." that I've seen.

Yeah, that's because we're still figuring out things as we go along. 😛

So when we find out when the general public does, it seems odd.

Yeah, we can do better here. Right now, there's not a good way to submit a PR that's only visible to .NET Foundation members. We could in theory have a Discussion (in GitHub) that is only visible to members. That might have been a good approach for us to consider.

I wasn't aware this was the request. I saw this and thought it was the directive.

In the README, the first two words after the heading are "This proposal." I think we probably should have put "PROPOSAL" in the heading too.

Then it is ambiguous. I think some of us are still learning to work with the new Microsoft and trying not to let our previous encounters affect our current interactions.

I get this. It's easy to see the .NET Foundation as an instrument of Microsoft, but it really is an independent entity. Hopefully the fact that Google and Amazon (among others) are now sponsors makes that clear. Their participation creates a nice check and balance.

I think the important thing is now that we've clarified this is a proposal, we need to work through the various concerns and address the goals of the proposal.

As I understand it, the goal is to provide a framework for any entity to understand the level of risk in depending on a library. In particular, different entities will have different levels of concerns when deciding to take a dependency. Large companies like Google, Microsoft, and Amazon often have well defined concerns when taking a dependency. Even within these companies, different products require different levels of rigor depending on the scenario. For example, an internal service might have a lower bar than a product that others will also take a dependency on.

The point of this proposal is to try and distill these concerns into a ladder to help any entity make an informed decision. It won't be perfect, but it should be useful and helpful. We'll definitely need feedback from Amazon, Google, Smaller companies, Project Maintainers, .NET Developers, etc to put together something that's useful.

And if we can't shape it into something useful, then we can always decide not to do it or do part of it or do something different. All those options are on the table. 😄

[haacked]

Speaking for myself, I've personally been frustrated over the years that a lot of .NET development shops, presentations, tutorials, documentation, training, etc. sticks very closely to what Microsoft ships because it's safe

Ok, last comment then I gotta run. This right here speaks to the motivation. When I worked on ASP.NET MVC at Microsoft, we shipped a project template that included jQuery. It was the first time Microsoft (afaik) shipped 3rd party OSS that was covered by Microsoft's full 10 year support.

I had customers come up to thank me and tell me they weren't allowed to use jQuery until it was included "in the box" by Microsoft. That was a pretty common sentiment. I felt bad for them, but I also tried not to judge their companies. Trusting Microsoft is a useful short-hand for them.

What I'd like to see is people come up to us and say, "I wasn't allowed to use the FooBarBaz NuGet package before, but I can now because it's L3 on the DNF ladder." The goal is to expose .NET OSS projects to a much larger audience (those who are not using it today) by giving them some means of quickly evaluating risk.

[glennawatson]

I'm going to summarise what our biggest problem in terms of ReactiveUI project.

Getting more invested into the DNF policy requirements is a risk for our project just due to the past teething problems and concerns that it might still exists. Not going to repeat the broken record, just some stuff took a while to communicate up and get fixed, which stopped our ability for a while to effectively run the project.

So that's why we're being pretty heavy at providing a lot of github issues around policies, since it'd help the foundation by having those anyway, but also reduce our concerns/risks at being able to manage the project.

I also just wanted to make sure you realise that we see you guys actively working on highlighted issues as well though and appreciate it.

[jongalloway]

@glennawatson Specific to the project support requests on ReactiveUI project: if you have an existing issue, can you please add it here? That includes anything with CI/CD, etc. that you mentioned above. You've got two general questions listed, but not a specific project issue. I want to make sure we get things fixed for you.

[benaadams]

How I think about this at a broad brush level is:

1. The lowest levels are "how can the .NET Foundation help your projects"
2. The mid level is "how can your project help other .NET projects and the .NET community"
3. The higher(st) level is "how can we cut through corporate governance policies and enable projects to be adopted by companies with strict rules"

Or as I said on twitter https://twitter.com/ben_a_adams/status/1176688409278107650

At the .NET Foundation we want the .NET ecosystem to thrive, and people and companies to more often consider wider dependencies than "Microsoft-blessed” projects. To help facilitate this we are proposing a .NET Foundation Maturity Model: https://dotnetfoundation.org/blog/2019/09/23/growing-the-net-ecosystem-through-shared-models-of-quality-confidence-and-support

Regardless of the level a project is at; the most important thing to us is the projects, we want them to succeed.

This is a proposal to address one aspect of that; it is not a ruling. We are seeking feedback and running a pilot to learn how it works in practice.

From the feedback and pilot the proposal will evolve. Please keep giving feedback, we are listening.

[RLittlesII]

@jongalloway Thank you for the feedback. I did not intend for this to turn into a long rant. I genuinely had concerns and felt I should address them. I agree with your feedback about actionable items. I will find the actionable concerns and address them in separate issues.

Speaking for myself, I've personally been frustrated over the years that a lot of .NET development shops, presentations, tutorials, documentation, training, etc. sticks very closely to what Microsoft ships because it's safe - it might not be the best, but it's widely distributed, got some assurance it'll be around and supported for a while, will get security fixes, will survive if the project lead runs off to join the circus, etc. As a developer, I want it all: I want a durable ecosystem, and I want a lot more than just one company churns out.

I had this conversation with a developer a few weeks ago, and is candidly the reason I am here. Someone went off to join the circus and asked me to help make sure the work he'd done didn't fall over.

So in the months leading up to this RFC, I was motivated to spend nights and weekends (in addition to daylight hours) working on this, because I think it's a good start on an idea that solves some real problems and gets us to the next level.

For the record: I appreciate the effort and work that went into this. These were just concerns and points for discussion so that I can feel comfortable rallying behind what the DNF is proposing.

I would treat this issue as a question (I didn't have the access to add the label myself). Now that my concerns have been addressed and my questions have been answered I feel this can be closed. It served it's purpose, which was to document what I felt I was reading about the proposed project maturity model.

[haacked]

Thanks @RLittlesII. I'll go ahead and close this.

I really appreciate the feedback. I think it's clear now that the original announcement felt "heavy handed" as @jongalloway pointed out. As a result, a lot of the feedback had the tone of "Oh here we go again! This is a new fiat that we have to protest."

Hopefully by now it's clear that this really is a proposal that requires input. It would not make any sense for the board to push something through that our most important constituents oppose. Again, I don't expect everyone will be happy with every detail, but I do expect our most important projects will want to adopt it in the end. And if they don't we've failed and should scrap it.

My hope is together we'll shape it into something most of us are excited about.

[jongalloway]

I do want to make sure that we don't lose any specific issues. There are some sub-issues raised during the discussion, and I think they're all tracked. But @RLittlesII or anyone else on the thread, if there's something that was discussed above and isn't tracked, please add an issue or pull request.