dotnet-foundation / project-maturity-model

Proposal/RFC for new .NET library development model.
MIT License
39 stars 8 forks source link

Microsoft must not be in the business of telling users which open source is "good" or trustworthy #32

Closed Aaronontheweb closed 5 years ago

Aaronontheweb commented 5 years ago

After reading @jongalloway's reply here https://github.com/dotnet-foundation/project-maturity-model/issues/14#issuecomment-535720864 I wanted to create this issue to raise a fundamental problem that exists with the .NET Foundation in its current form that makes some portions of the ladder proposal distasteful, in my eyes as both the proprietor of an independent .NET OSS business, a project contributor, and as someone who deeply cares about the success of the .NET OSS ecosystem as a whole (it's why I started my company.)

Microsoft and the .NET Foundation are effectively joined at the hip. Its day to day operations and run by largely by Microsoft employees and its interests are fully aligned with Microsoft's. While the goals and intentions of the .NET Foundation are precisely intended to help foster a healthy third party ecosystem around .NET (Microsoft wants this too,) the fact remains that it's largely Microsoft's show and is viewed by users and .NET Foundation members as such. Let's acknowledge this perception as fact and move onto why it's an issue.

Microsoft's Influence on the Behavior of .NET Users

Microsoft has singular influence on the behavior and perception of the .NET community, being the creator of the platform; the developer of the most popular technologies and frameworks on it; and through tremendously high levels of effort: a solid track record of high quality work.

This singular influence comes with what is now a widely acknowledged problem that Jon and the .NET Foundation board are attempting to address through the maturity ladder, per his comments I linked earlier:

We had some board members campaign and win a board seat with statements like "I’d like the .NET ecosystem to thrive with projects people consider and take dependencies on to be much wider than the narrow “Microsoft-blessed” scope that people and companies generally consider." At our first board meeting, I asked the board to come up with some specific areas and action groups, and this was one of the seven that made the list. I had other things I'd expected to see on the list, but this wasn't one of them.

I applaud this and I am certain the lion's share of the .NET Foundation membership does too. Creating more "package publisher" diversity in the .NET ecosystem through normalizing the adoption of non-Microsoft technologies is essential to accelerating the rate of innovation in .NET as a whole, because Microsoft can only invent and maintain so much.

However, where this proposal goes sideways is it's a roadmap for how to become a Microsoft-blessed project, with the .NET Foundation (Microsoft) acting as the certification authority. This is the opposite of the stated goal - it keeps Microsoft in the position of centralinfluence over the .NET ecosystem, especially if the .NET Foundation's maturity ladder gets coupled with pure Microsoft channels like NuGet.org's user interface . There really would be no effective distinction between the two organizations.

Ultimately what I believe both I and the .NET Foundation leadership, members, and projects want is an rapidly growing ecosystem with projects that adopt good practices, are well-maintained, and regularly updated. That approach, necessarily, must be decentralized in order for rapid growth and innovation to occur.

The current maturity ladder is a centralized, top-down, and heavy-handed approach - because it defines fairly strict quality control rules that are a reflection of how Microsoft's own organizations operate. And if it's implemented in its current form, I'm afraid it will create monoculture among how .NET OSS projects are operated (i.e. they all have to be in the .NET Foundation to be a high quality project, must all use a permissive license, etc) and may further stifle the ability of new projects that don't have a long track record to get off the ground.

Just to drive the point home about monoculture, a specific example from Tier 3 of the project maturity ladder:

  • Practices
    • Complies with .NET Foundation continuation policies.
    • Stable packages depend on libraries that are at level #2 or higher.
    • Signs packages (Note: this refers to digitally signing NuGet packages, not strong naming).
    • Uses static analysis tools to validate pedigree and safety.
    • Applies .NET API design guidelines.
    • Seeks and applies guidance from .NET Foundation design and architecture group.
    • Updates package references regularly to accommodate servicing updates in dependencies.

This requirement effectively eliminates an entire OSS business model: offering certified builds to enterprise users. Most small users (i.e. customers that won't pay for OSS) are fine with the "buyer beware" culture of OSS. Big companies, who have much larger exposure, are willing to pay a company that contributes to / maintains the project for certified builds that are signed and more heavily scrutinized - that's not going to be the case any longer if an OSS project needs to be tier 3 or higher for BigCo to adopt it, as much of that value is now expected for free under this model.

Running a sustainable OSS project (i.e. capable of funding itself, has natural business incentives to provide high quality work to users) is hard enough and I don't believe the .NET Foundation ever intended to be in the business of telling those projects how they can and can't make money, but that's what introducing an explicit grading system and pushing that down to users will accomplish.

Additionally, I don't believe the .NET Foundation is properly resourced to even execute such an effort in its current form. It's a lot of work for a small team and will get worse the more projects being needing infrastructure - there's a whole issue opened to address this already, so I won't rehash it further.

Solution: Bottom-up Approach to Maturity

The .NET Foundation has its heart in the right place - widening the range of "acceptable" OSS beyond what is produced in Redmond. I want that too. Everyone does, most of all the developers who aren't able to use that OSS today because of the historical conservatism of the .NET ecosystem as a whole when it comes to OSS.

The current proposal puts the .NET Foundation as a centralized certification authority for the entire .NET ecosystem. That's heavy-handed and will ultimately undermine the .NET ecosystem's ability to become "bigger than Microsoft."

What I propose instead is a bottom-up approach:

  1. Never use the maturity ladder as a public grading system of any kind. No lists of "tier 4" or "tier 3" projects. And certainly don't integrate that language into NuGet.
  2. If NuGet and other Microsoft interfaces into third party packages (i.e. Github) want to offer some kind of hint as to how well a project is run, I recommend using something similar to Github Pulse - show that the project has commits from multiple users, automated CI checks, new releases, responses to issues, responds to recent security incidents, etc. The reason why this is fine is because all of that activity is 100% under the control of the project developers themselves - no one had to seek out permission from the .NET Foundation to be rated as a "good" project. This decouples the .NET Foundation from being the arbiter of what is good and what is not. Instead, the project leads themselves decide if they want to put the time and effort into doing those things and act accordingly.
  3. Projects do not need to be part of the .NET Foundation to achieve everything in point 2; projects like NServiceBus, ServiceStack, RavenDb, EventStore, and so on are trustworthy because those developers can't pay the bills if they don't act accordingly. That should be good enough.
  4. Use the maturity model as a roadmap for guiding projects towards best practices, not a process to be strictly followed. If one project doesn't want to sign its builds because its maintainers want to offer that as a paid service, let them and their users (consumers and contributors both) decide if that's ok.
  5. Leave the forking policies et al in place for .NET Foundation projects, but leave that at a .NET Foundation member project consideration - not something that applies to "any trustworthy .NET OSS" project.

If a project dies or gets abandoned, the objective numbers in point 2 will decline and maybe that project will be forked by users who still have skin in the game. Let it happen - that's allowing the ecosystem to run itself.

I believe in the .NET Foundation's mission and that's why we joined it and I very much want it to succeed. That's why I felt the need to criticize it and say what might be a painful truth: that it's still Microsoft's show and this proposal only makes it further so. It's in the best interests of the entire .NET ecosystem, Microsoft and the .NET Foundation included, to make .NET bigger than Microsoft. I hope you will consider my recommendation in the spirit of trying to help us all make that happen.

manigandham commented 5 years ago

Agree with this completely.

This maturity model just seems like going from Microsoft making official projects to Microsoft getting others to make official projects. It really sounds like some kind of disconnected corporate decision.

The fact that there are official projects and everything else is the one of the major problems in the .NET ecosystem. The answer is to offer support and tools to let people build and publish faster, and get those projects more visibility and usage so everyone is comfortable using them. Not an official seal that adds more friction and further divides the community.

Public OSS projects are naturally selected by users based on their activity and reputation. It's also what allows rapid development as projects grow, evolve, migrate and die to be replaced by others. This should be embraced and encouraged as much as possible by recognizing and amplifying those existing community signals.

We don't need this maturity model at all.

jongalloway commented 5 years ago

This is good feedback, and appreciated. I mean that sincerely. :)

Some rough thoughts because turning this into full prose would take hours:

Going back to this counter-proposal, my main question is what are the key objectives?

A maturity model in isolation may improve overall ecosystem quality, but it's not going to get Microsoft to ship or support it. The .NET Foundation has no say there, and realistically I have no say there. We can say, hey, here's this maturity model, and they'll say, great, good job I guess. Going back to the above browser standard metaphor, why write a standard that your main browser vendor doesn't care about?

isaacabraham commented 5 years ago

@jongalloway isn't that part of the issue: that designing a maturity model that fits the existing expectations of the largest vendor (ie MS) is just going to perpetuate the same issues ie MS writing software that fits the bar because it's designed around their expectations in the first place, but the rest of the community not getting involved as much. Might this not also be about MS changing their expectations of what they are able to accept and take a dependency on?

As a side note, in the F# there is the fsprojects github org / project which was used in this sort of way. Perhaps @dsyme would be interested in sharing some experiences of how that has worked. It's much more lightweight, and to be honest I would say it never really affected my decision to take a dependency on an OSS project one way or the other (indeed, I'm trying to think how many F# OSS projects would get close to the top of the ladder, and yet simultaneously it has a successfully OSS model that has naturally evolved over time).

JamesRandall commented 5 years ago

Reflecting on my own reaction to this and the comments above perhaps a big takeaway from this is the need for the Foundation to better communicate and/or demonstrate its independence from Microsoft and improving this before undertaking projects such as this.

Its the first time the DNF has proposed something that I've both been aware of and feels likely to impact me (as a contributor) and I've certainly got a degree of suspicion - not about individuals, but about the Foundation itself. It feels like a vehicle for Microsoft and big interests (to be crystal clear: DNF members may argue this is not the case and it may well not be, however I'm talking about perception).

I realise this might be a bit chicken and egg but an organisation perceived as being so closely aligned with Microsoft introducing a viral process for assessing, and visibly highlighting the maturity of OSS projects (and in the process add pressure to contributors) is going to attract a good degree of suspicion and distrust. NuGet has been discussed as carrying the badges which would be a massive community change and maturity with badges is going to be seen as endorsement of quality (whether that is the case or not - perception, again). Going back to the perception of the DNF: that feels akin to a land grab.

Perhaps also seek to bring more contributors on board as Foundation members - as a maintainer of a (compared to many others communicating here) small OSS project I've looked at membership of the DNF a number of times but its really not clear what the value is beyond a fairly vague couple of paragraphs and reading through the comments around this issue I'm even less clear: even if this had been run by .NET Project Leaders I as a normal member would have, it seems, had the same ability to influence it as I do now.

If developers like myself were more aligned with and positive about the DNF then that's going to make introducing significant change to the ecosystem an awful lot easier.

Perhaps address some of the other related issues first (infrastructure and comms challenges have been discussed, support / funding for contributors, and others) and swing back round to this at a later date.

JamesNK commented 5 years ago

Does anyone disagree that it is worth encouraging open source projects to:

And so on. These are all good things. We should be encouraging open source projects to do these things.

If you have individual points you disagree with, i.e. you think they are pushing an agenda that is harmful to the .NET OSS community, then you should create or contribute to issues that discuss those individual points. An uber complain thread isn't a productive way to make changes.

The last 9 months I've been collaborating with the CNFC on gRPC and it is an awesome organization, and driving force behind open source, particularly in the Go eco-system. I really wish .NET had something like the CNCF years ago.

Look at awesome this is: https://www.cncf.io/projects/

glennawatson commented 5 years ago

@JamesNK To be fair to these guys they have been pretty concise with their feedback they don't believe the .NET Foundation is the right certifying organisation. Eg the issue's topic is Microsoft must not be in the business of telling users which open source is "good" or trustworthy.

jongalloway commented 5 years ago

@glennawatson .NET Foundation != Microsoft

That's the whole point here

glennawatson commented 5 years ago

Yes, but it's valid feedback you have that perception unfortunately at the moment, not quite sure if this proposal is the right place for that issue though.

JamesRandall commented 5 years ago

@jongalloway - would you agree there could be a perception problem and that that might impact the Foundations ability to achieve its goals?

glennawatson commented 5 years ago

I have created #42 and https://github.com/orgs/dotnet-foundation/teams/outreach/discussions/11 to track the independence issue given in the feedback above. Thanks guys.

manigandham commented 5 years ago

Thanks for the replies. I think it's going to be really hard to separate the .NET Foundation from Microsoft. I understand they're different but didn't know how independent the foundation actually was. I also question how many users are going to really know or care.

In the minds of many developers, .NET = Microsoft with sanctioned and official solutions to everything. There are still many who haven't gotten past it being locked into Visual Studio and Windows. It's this very perception that's part of the issue with community projects, and guidelines to be certified just feels like corporate behavior which further blurs the lines.

A counter-proposal would be to turn the "maturity model" into guidelines only, for recommended ways to build and run .NET projects. Don't use it as an official status symbol, don't have tiers or certifications, and don't add integration in Nuget. Not every project needs or wants to meet all those requirements, and shouldn't have to. And the ones that don't shouldn't suffer for it.

@JamesNK Isn't the CNCF language and technology agnostic? What is keeping them from being a driving force in the .NET ecosystem?

JamesNK commented 5 years ago

CNCF is technology agnostic. My impression is a lot of the more popular projects are written in Go (e.g. Kubernetes) but there are many other languages used (Envoy proxy is C++, gRPC has implementations in many languages, etc).

The .NET model is based on what the CNCF has:

Would you prefer that the .NET Foundation doesn't have its own model and encourages projects to join CNCF instead? The .NET Foundation could work to make CNCF a bigger force in .NET, and provide evangelism and visibility that CNCF projects are following good OSS practices. (I actually think that if NuGet is displaying metadata about a project's .NET Foundation maturity model to developers, it could also show metadata about CNCF membership. They're each good alternatives)

If you do prefer the CNCF over .NET Foundation, I'm curious why. Both are independent organizations with decisions made by board of directors.

Aaronontheweb commented 5 years ago

If you approach this problem space assuming that .NET Foundation is Scooby Doo mask Microsoft in disguise, there's no reason to involve the .NET Foundation. Just deal directly with Microsoft.

Likewise, if you approach our objections from the point of view of us being a bunch of OSS communist rabble rousers FUD-spreading against the ghost of Microsoft past rather than as people who share the same common goals and interests as what's stated in the .NET Foundation's mission and are criticizing something they love and are invested in, why care about our feedback at all?

I'm assuming because that's not actually how you see us and you don't want to create a situation where the .NET Foundation's interests and that of its projects don't align. So comments like this are unhelpful and defensive.

The issue of the lack of boundaries between the .NET Foundation and Microsoft is real, to highlight some examples I shared on Twitter:

  1. This proposal was announced during .NET Conf, a Microsoft-run event, which I'm assuming was intentional;
  2. NuGet, a Microsoft product and their official distribution channel for packages, is considering incorporating this into package labels;
  3. And the steps on this ladder are the culmination of MSFT product groups' answers to the question: "what would it take for us to ship something that has a third party dependency?"

Or, in your own words:

One of our top driving factors was to allow key stakeholders to take dependencies on, ship, and support, open source packages. Like it or not, one of the top distributors of .NET packages right now is Microsoft. So we worked with Microsoft's .NET team to make sure our quality bars met the bars they use internally to ship and support the packages they ship.

And look, I get why the foundation would want to do this. I've gone out of my way to make clear that I believe the intentions are good as have just about everyone else who has spoken critically of the proposal.

Creating a standard on our own without consulting the main distributor of our packages is like creating a web standard without checking in with the Chrome dev team. You may end up with an elegant, but meaningless, standard.

This is, absolutely, the issue. NuGet should be in the position of distributing packages and making objective data about that package available (license, downloads, recency of updates, whether they have automated CI or not, etc) to users and no further. NuGet isn't an app store and shouldn't be run like one. It shouldn't be in the position of subjectively reviewing or promoting packages above others for any reason other than search relevancy, recency, and qualities that are entirely under the control of the package maintainers.

And moreover, why should a single OSS foundation get to decide what makes for a trustworthy package for the main distribution channel of the .NET ecosystem? Does Confluent's .NET Kafka driver to get to be a level 4 trustworthy package, even though they're in the Apache Foundation? Ostensibly, they're doing all of the things needed to be at that level - so why not?

If you want to make .NET Foundation packages trustworthy for internal consumption at Microsoft, i.e. have this maturity ladder apply strictly to .NET Foundation member projects, that's a different pitch than what's been presented here. This ladder has been presented as the roadmap for all .NET packages to become "trustworthy" in the eyes of Microsoft and how Microsoft will promote those projects to its users. That's a problem. There shouldn't be any bleed-over from the .NET Foundation into NuGet and other "Microsoft" channels. Other projects that (1) aren't in any foundation and (2) are in other foundations can all still do the right things and be "trustworthy."

glennawatson commented 5 years ago

I think from my understanding is that the .NET foundation will just run a badge server, rather than getting NuGet modified.

The proposal is you can just put the badge into your standard NuGet description which you can already now.

JamesNK commented 5 years ago

The .NET Foundation can't force NuGet to display metadata about a project's membership in .NET Foundation. Any changes to NuGet are a choice for the NuGet product team to make based on the criteria of whether it is good for NuGet users. Personally I think knowing that a project is part of an organization is useful thing for developers to know when making a decision. Your opinion might be different.

However, if NuGet did add this metadata, it shouldn't be limited to .NET Foundation.

That would prevent concerns of .NET Foundation mono culture.

Aaronontheweb commented 5 years ago

And another possibility that I should raise in response to this:

One of our top driving factors was to allow key stakeholders to take dependencies on, ship, and support, open source packages. Like it or not, one of the top distributors of .NET packages right now is Microsoft. So we worked with Microsoft's .NET team to make sure our quality bars met the bars they use internally to ship and support the packages they ship.

Given that much of .NET OSS contributors' work done voluntarily and for free, can you see how asking them to:

  1. Adopt an expensive professional product management model;
  2. restrict their business / operating models to those that support permissive licenses only;
  3. Make themselves open to new ownership on their projects, via continuation policies and .NET Foundation membership; and
  4. Requiring a Microsoft-approved specific set of coding standards for their projects

All in order to satisfy the wealthiest and largest .NET consumer in the ecosystem, again, with labor that is unpaid. Do you how see how this would not be received well by members of the community who are already struggling to find ways to make their projects self-sustainable?

Aaronontheweb commented 5 years ago

@JamesNK

Any changes to NuGet are a choice for the NuGet product team to make based on the criteria of whether it is good for NuGet users.

It's a fait accompli that NuGet will do this if the ladder becomes an official Microsoft-accepted quality standard for third party OSS. That's why there shouldn't be a public labeling or grading of projects. Make the quality metrics you pull from Github / Gitlab / BitBucket available in NuGet metadata and leave the rest of the decision making up to the end-users. Allow the ecosystem to run organically, in other words. Don't put Microsoft's very, very big thumb on the scale for projects that are already mature and popular and create barriers to entry for new projects.

Aaronontheweb commented 5 years ago

@glennawatson From NuGet's PM https://github.com/dotnet-foundation/project-maturity-model/issues/6#issuecomment-534795139

Got it. We are discussing about quality indicators and it does fit in that. Will keep you posted as we make progress.

Conversation was about including that information in NuGet search results, specifically. Nothing is set in stone yet, hence why I felt it necessary to object here. I object to a README label too. Really, any kind of public trust rating of projects originating within the shadow of Microsoft, due to Microsoft's historical and singular influence over the platform.

I'm putting this out there not in the spirit of calling Microsoft bad or anything of the sort - only pointing out the cure (edit: official ranking of projects per the current draft of the maturity ladder) likely isn't the right treatment for the disease (companies adopting a very narrowly defined band of Microsoft-blessed works.)

glennawatson commented 5 years ago

BTW, might be worth putting the feedback in a separate issue re: badges.

This one has got pretty long winded with a lot of debates etc.

JamesNK commented 5 years ago

Activity isn't a good indicator for everything. A project might be busy but be run by toxic maintainers. Or the project could not use 2FA. Or be built on unsecured CI server. Or have no license. Or have no plan for what happens if the maintainer rage quits .NET to become a Ruby developer. Etc.

Today when people use a library from a known company like Microsoft, Google, Telerik they know that all those problems have been taken case of. A program that allows developers to trust OSS projects more from outside known companies, or "blessed" OSS (Newtonsoft.Json) is useful.

Aaronontheweb commented 5 years ago

@glennawatson that's because this is a complicated, multi-factor issue. It can't be easily sub-partitioned. Let it play itself out.

@JamesNK A counter-example. Akka.NET takes a dependency, DotNetty, that was developed by the Windows Azure IOT group (Microsoft) and is critical to our networking systems. We're having to move off of it because the project has been abandoned - that group is now doing its development in Rust as I understand it. I couldn't push a new version of that package even if I contributed to it (more than I already have) - I'd need to fork and take on the burden of maintaining it.

Even projects run by "trustworthy" organizations get abandoned. The maturity ladder will do nothing to prevent that - no one is going to step in and run an expensive, complex project just because the ladder has a continuation policy.

JamesNK commented 5 years ago

Even projects run by "trustworthy" organizations get abandoned. The maturity ladder will do nothing to prevent that - no one is going to step in and run an expensive, complex project just because the ladder has a continuation policy.

Sure, that's ok. We don't live in a perfect world. If a software tool is really important to you a continuation policy would at least give you the option of approaching the .NET Foundation to ask about taking over maintaining it.

As long as the .NET Foundation doesn't oversell what a continuation policy means then I think that is fine.

manigandham commented 5 years ago

@JamesNK

Would you prefer that the .NET Foundation doesn't have its own model and encourages projects to join CNCF instead?

No preference about CNCF, and as I understand it they're only interested in "cloud-native" projects anyway. The question was more about why they haven't invested much in .NET projects in general and what that may show.

I believe the answer is because of the general consensus that .NET solutions tend to use official packages over community projects, so large distributed projects similar to Envoy haven't taken off in .NET from a lack of users, funding, and support.

If so, then tiers of certification for .NET projects will further the divide between official and not, instead of trying to influence and fix the underlying choice to always go with official only. Then again, maybe issue #35 (identity the target consumer) needs to be settled first to see if any of this is solving a major problem at all.

glennawatson commented 5 years ago

One question @JamesNK why was the CNCF a dream to deal with in your opinion? What made the process easier for GRPC?

oskardudycz commented 5 years ago

I think that the main understanding bias comes from the way how this information was introduced and named. Maybe if it was named Enterprise Maturity Ladder then at least it would clear up some of the intensions. It's also a bit awkward when MS doing open source for ~3-4 years makes itself an authority. Don't take me wrong, I like the "new way" of the MS but it's still fresh. Even in our .NET community, there are a lot of people that are doing it few times longer.

Let's be clear, we're discussing here our internal issues, while most of the .NET outside world haven't even heard about the .NET Core, maybe they heard that you can run .NET on Linux, but it's just a curio for them. They still have a picture of the "bad old MS" in front. How the maturity ladder can help with that?

The other thing about this bias is that probably maintainers were surprised that the first big .NET Foundation initiative is so focused on making the certification from the top without discussing it widely.

Maybe my experience is somehow distracted as I'm also doing Open Source for over 3 years and I'm the maintainer of relatively popular library for only 1 year, but imho maturity ladder won't help in making it sustainable. @JamesNK brought the comparison to the Telerik, Google - MS can compete with that paid labor. How smaller libraries and maintainers that are doing their best building the community after hours can be compared to that? These are not the day to day problems of the non-MS maintainers. Will the maturity ladder help with the bus factor? Imho if .NET foundation focused on the training and proactive collaborations with maintainers it would be more beneficial. What trainings? How to get the sponsorship, how to make the OSS model sustainable, promoting them. So not to certificate but bringing recommendations and real help on doing it in a mature way. If we focused only on the MS and bigger players needs then we're risking that we'll end up with a semi-open source community.

So the bias is not with promoting code of conduct, good practices, CI, CD, proper behavior for the users, transparency. It is because non-MS related maintainers don't see many benefits of having that, while the MS benefits are obvious. How can badge help us with maintenance and sustainability?

That's also the reason why I fully agree with @Aaronontheweb proposal of doing this longer and wider bottom-up approach with discussing the real needs and making it beneficial for all parts. Because if it's done otherwise then it reminds of the "bad old MS", which is for sure not the intention of the .NET Foundation.

Let's not make more divisions of our already divided community and focus on more collaborations between all parts.

rubberduck203 commented 5 years ago

Like it or not, one of the top distributors of .NET packages right now is Microsoft.

I find this to be a dubious statement that depends on how you define “largest contributor”. Perhaps it’s a true statement, perhaps not. If it’s true, however, it has only recently become true along with MS’s “change of heart” and open sourcing of many things it would have built and supported anyway.

As an outsider and long time .Net open source user/contributor/maintainer looking in at the foundation right now, I get the sincere impression the foundation believes there was no OSS .Net community before MS open sourced .Net and the foundation was... er... founded.

I whole heartedly agree that attempting to place MS level of engineering guarantees and trust upon the community will do nothing but stifle it and move us backwards. Most of us simply do not have the resources to support this maturity model.

I would very much like to see a clear problem statement from the foundation about what problem you’re trying to solve here. I’ve read through the README and proposal and simply can’t parse it out. The best I can gather is “We want the OSS community to provide first party levels of trust and guarantees.”, but that’s not really a problem, it’s a solution.

I know you have the best of intentions at heart and this firestorm must be a very exhausting time, making you question volunteering your limited time. I know that because I’ve been in your shoes before. It’s because I’ve been in this situation that I’m concerned.

snickler commented 5 years ago

I see this from the angle of the "don't trust anything if it's something I don't recognize" type of Enterprise. I'm utilizing many OSS libraries in projects due to the fact that I can show their stability and usefulness.

Explaining why I'm using or recommending an open source solution to those who are strictly business-focused is one of the most irritating things a developer could ever go through. Most of these places see OSS as untrustworthy, prone to vulnerabilities, and less secure than XX Blah Company's $100K commercial product (which happens to incorporate many open source libraries....).

In my own personal experience, stating that Microsoft or some other well-known company is using some of the same libraries happened to be enough to allow my usage of them. Growing the perception of the .NET Foundation as a separate entity from Microsoft is a great thing, but I believe that it's necessary to still have talks with Microsoft and other companies about direction when necessary.

I don't know what others have experienced, but having a recognizable company name somewhere in the midst brings "comfort" to the ones responsible for giving the yea or nay decision on open-source library usage.

dustinmoris commented 5 years ago

NuGet isn't an app store and shouldn't be run like one. It shouldn't be in the position of subjectively reviewing or promoting packages above others for any reason other than search relevancy, recency, and qualities that are entirely under the control of the package maintainers.

THIS 100%!

Activity isn't a good indicator for everything. A project might be busy but be run by toxic maintainers. Or the project could not use 2FA. Or be built on unsecured CI server. Or have no license. Or have no plan for what happens if the maintainer rage quits .NET to become a Ruby developer. Etc.

Today when people use a library from a known company like Microsoft, Google, Telerik they know that all those problems have been taken case of. A program that allows developers to trust OSS projects more from outside known companies, or "blessed" OSS (Newtonsoft.Json) is useful.

This is a clear case of "I work for MSFT and I care about these things, therefore I want to force everyone else to care about this too".

A project might be busy but be run by toxic maintainers.

This is subjective. There is different code of conducts out there. Unless one wants to contribute to a project there is no reason why this should be a concern.

Or the project could not use 2FA

This is completely irrelevant though. Microsoft might not distribute a package which doesn't have 2FA on, but this is an internal issue and shouldb't be forced down on everyone else. I can turn on 2FA right now with a weak password and a public email address. It's not a metric to say anything about maturity or trustworthyness IMHO. I know of packges where the 2FA codes are shared across the entire dev team in a shared 1Password note. This is less secure than one package being maintained by one person without 2FA.

Or be built on unsecured CI server

This is just another one of those. Overall these things all fall into the "trust" territory. Trust is an individual thing. It seems like that Microsoft has established what it takes for them to trust another OSS project and now they want to force this down on everyone else, but trust doesn't work like this. You can't force third parties to trust other third parties based on your world views, because trust is hugely individual. I don't trust Blazor even though it's form Microsoft, and I trusted Json.NET before and after it became Microsoft. On the other hand I trust Razor, but I don't trust the latest JS frontend fad. Why? Because I trust based on adoption, not if something has 2FA, or a CI server which pleases Microsoft. Trust will always remain an indidiual thing, whether you like/admit this or not, so trying to establish a common "trust" policy is deeply flawed in it's core idea.

I 100% agree wth @Aaronontheweb. Display project metrics like GitHub Pulse and let each individual decide themselves which metrics matter to them in order to establish trust or not. You can't dictate trust.

Or have no plan for what happens if the maintainer rage quits .NET to become a Ruby developer.

What happens if a DNF maturity level 3 team rage quits? Who is going to pick up the pieces. Anyone can write something in Markdown and make empty promises. Unless the DNF itself promises that they will 100% fill the gap by hiring a new maintainer I don't think there is any guraantees which a maturity level ladder can do on this one.

EDIT:

I think a great thing for the DNF to do would be set clear guidelines like "we know that many orgnisations care about X and Y, we recommend doing Z so these metrics go up in your GitHub/NuGet stats" and "if you can sustain X, Y and Z at a level of ... for ... months/years, then your project also qualifies to be referenced and distributed as part of other official MS packages".

This would be a clear guideline and statement to help maintainers as well as a clear objective way of setting a bar for packages potentially getting consumed by MS packages.

JesperTreetop commented 5 years ago

If you approach this problem space assuming that .NET Foundation is Scooby Doo mask Microsoft in disguise, there's no reason to involve the .NET Foundation. Just deal directly with Microsoft.

Conversely, if we can (at least as a thought experiment) approach this assuming that the .NET Foundation is a separate entity, you also have to understand that the .NET Foundation can't make Microsoft do anything, and Microsoft can't make the .NET Foundation do anything. This is actually true, FWIW.

I think a major reason people are skittish about this is that people remember Java being capital-O Open (or even all-caps, bolded, 72 pt OPEN) under Sun, and then being swept up by Oracle who started taking people to court for reimplementing interfaces, or demanding license fees from companies using products with bundled runtimes. It's not that we think Microsoft would do this (I think most parts are far too sensible, although I worry about SQL Server pricing missing a decimal point sometimes), it's that closely connected foundations of this kind need to continually and continuously demonstrate their independence by actions and deeds to both be and be recognized as truly independent.

The goals of Microsoft and the goals of the .NET Foundation are very closely aligned right now, and there need be nothing fishy about that. But let's take this as an example: .NET Standard is created as a recommended omnipresent target. Within years, it is more or less discontinued, presumably since it would be strange to have it not be implemented on the single largest .NET runtime. Oh, and the other runtimes not affected by it will also be merged into one, based on a meeting that took place, as far as I can tell, only among Microsoft employees and without community participation.

I know that the .NET Foundation exists, know that time was spent getting community members put on its board, know from posts, comments in this thread and others and community standups that Jon is animated about making sure it is an independent voice that does not just serve one master. I know very little, but I still have more knowledge about it than most people who use .NET. I still think there's more to do - if the .NET Foundation is indeed going to steer .NET into the future - for it to handle more tasks. And if the .NET Foundation isn't going to do that, it should be named something else.

Why is this relevant to this conversation? Because if the .NET Foundation isn't where those tasks take place, falling into the "Scooby Doo mask Microsoft" perception is a rational thing to do if you're a third party trying to assess the situation, and maybe work out if you want to choose .NET Core for your new product, based on recent events. I am not assuming malice in any of the actions I mentioned, hard decisions maybe needed to be made and they were made by people trying to do the best for .NET in earnest and with no big "Microsoft master plan" at hand - but if the goal is to be community-run, more things have to involve the community, even when it's sometimes inconvenient or less than expedient for Microsoft.

.NET used to be a place where Microsoft wanted you to do things Microsoft's way, and thanks largely to change driven by Microsoft employees for the past ten years or so, Jon and various Scotts definitely included, we're on a course to leave that behind. With every year, new opportunities with new sacrifices and new payoff are required to follow that path further. I don't mean to say that Microsoft has been sitting on its hands, because it's clearly taken enormous adjustment, and even as I'm writing this I'm mindful that it can be taken as a slight to Microsoft employees who have only good intentions. But being part of the .NET community, I still can't help but bring this up in a place where it looks relevant enough to me.

haacked commented 5 years ago

I think a major reason people are skittish about this is that people remember Java being capital-O Open (or even all-caps, bolded, 72 pt OPEN) under Sun, and then being swept up by Oracle who started taking people to court for reimplementing interfaces, or demanding license fees from companies using products with bundled runtimes.

Keep in mind that occurred in part because Sun withdrew from the ECMA to formally standardize Java. And now Oracle Java is the de facto standard.

C# on the other hand is an ECMA standard. Also, Microsoft no longer "owns" the .NET code base. It belongs to the DNF. So another Oracle can't swoop in and take people to court for it.

The point I want to make here is this feels like a Genetic Fallacy.

is a fallacy of irrelevance that is based solely on someone's or something's history, origin, or source rather than its current meaning or context. This overlooks any difference to be found in the present situation, typically transferring the positive or negative esteem from the earlier context.

And not without reason, I might add. I understand it. I think it's good to bring these things up so that we can address them! For example, I believe what happened to Java cannot happen to .NET because of the steps put in place. If you disagree, let's open an issue and discuss the safeguards that would need to be in place to prevent that.

At the same time, I think it's important to understand how the present is different from the past and reason accordingly.

For example, there's a lot of concern that the .NET Foundation is not independent enough from Microsoft. Keep in mind that fierce MS competitors Google and Amazon are also sponsors of the .NET Foundation. What else would you want to see to be convinced that it's truly independent?

For me, I want to see even more sponsors. MS still provides the bulk of funding for the foundation. I hope to see their funding become a smaller piece of the whole pie (by growing the pie of course!). I think addressing the challenges we're trying to address with this proposal can help that.

Because I trust based on adoption, not if something has 2FA, or a CI server which pleases Microsoft.

Unfortunately that's exactly what attackers are counting on these days. It's likely that a phishing attack enabled attackers to inject a remote code execution backdoor in bootstrap-sass. People trusted that package because it was adopted by many. I've written a series of posts on the topic of package security: https://haacked.com/archive/2019/06/11/package-manager-security/

Nothing is foolproof in protecting consumers, but as these attacks continue to be successful (and perhaps there are many that have been that we just don't know about), I think it's important to set some basic guidelines around good security practices.

Aaronontheweb commented 5 years ago

What else would you want to see to be convinced that it's truly independent?

Throwing the current text of this proposal in the trash in its entirety and starting the entire process over would be a good start, as it would show that the interests of the project maintainers and developers - the people bearing the cost of the implementing these practices, actually matter and aren't secondary to Microsoft's preferences as a consumer of .NET OSS.

It would also give the DNF leadership a chance to demonstrate that they've learned from the communications / process foul-ups here and do it right this time. You can start the conversation by answering #35 and #44 with hard data, written objections to adopting specific third party packages, and the things I and others would need to justify implementing these processes.

oskardudycz commented 5 years ago

To be fair lack of the exact authority that was driving the Java world was not good for the whole system. There were a lot of different Java versions and organizations that couldn't agree on the language and environment roadmap. I really don't like the Oracle and their way of working, but base on comments from my colleagues from the Java World they provided the single source of truth. Which was good for Java etc. as it was going into stagnation (lack of important features like lambda stuff etc.).

I'm fine with having MS as the authority that's driving the .NET. I see the benefit of having the single big player that stands behind. That's fine, and it might be good (or at least successful in some way).

Although currently there is confusion if really .NET Foundation is an independent authority or not. This proposal makes the impression that's not. It would be fine if it was clearly stated (as it is in Java world and Oracle). I think that we need actions not just words and documents to prove independence.

The current proposal shows the benefits for the MS but is not showing (as @rubberduck203 mentioned) what exact problem is it trying to solve for the whole community and its day to day issues (like eg. sustainability - having a hard time getting the support and sponsorship while working after hours).

dustinmoris commented 5 years ago

Nothing is foolproof in protecting consumers, but as these attacks continue to be successful (and perhaps there are many that have been that we just don't know about), I think it's important to set some basic guidelines around good security practices.

Guidlines is great, but you are proposing to label things as secure if something meets YOUR definition of secure, and therefore ignoring all other aspects of what people consider secure and therefore indireclty labelling everything else outside your own views as insecure. If I go to NuGet and I only see .NET Foundation level X packages labelled secure then you created an environment that by definition discriminates everyone and everything which is not part of your boys club.

EDIT: I don't think anyone is against guidelines, but there is a line between helping and guiding and presurring things into a mono culture and discriminating.

haacked commented 5 years ago

If I go to NuGet and I only see .NET Foundation level X packages labelled secure

Yeah, I would not advocate for that either.

In my view, it would be prudent to develop a set of package manager security standards and then have NuGet surface them in some way. For example, maybe you want to be warned if the owners of a package don't have 2FA enabled. This would allow everyone to define THEIR own idea of what's secure.

However, as we know, most people won't go through the trouble to customize this, so I think it's important to have a set of default profiles that have a recommended set of practices and people can either choose from one of the default profiles, or create and share profiles that meet their needs.

Keep in mind, this is a thought experiment. I don't control NuGet's roadmap or future plans (at least, not anymore 😛 )

glennawatson commented 5 years ago

It'd be nice if nuget made more of a deal about signed packages. I'm sure there is a whole other debate about equality with smaller package makers disadvantaged due to the cost of signing certificates though.

dustinmoris commented 5 years ago

@haacked Just continuing your thought experiment... what if you replace "default profiles" with "properties of interest".

E.g. instead of NuGet displaying "No DNF level/Level 1/Level 2/Level 3" and people having to know what those levels mean, Nuget could display a list of properties which we collectively think would be of interest:

I'm sure some talented UI/UX talents could display this in a very digestable way and the whole thing would feel more opt-in than opt-out. If a repo ticks all boxes of Level 3 except one, then this package would not look like it's losing credibility because it's still level 2 which feels like a huge gap.

I think this is what some others here have indicated with a GitHub Pulse like model... at least that was my understanding.

EDIT:

Think of it like your frontend not checking for specific browsers, but browser features ;) Same concept sort of :P

rubberduck203 commented 5 years ago

Nuget could display a list of properties which we collectively think would be of interest: License model 2FA true/false Signed binaries true/false Avg. repo activity last 6 months etc.

Talking the NuGet team into supporting this would so far more for the community at large. As a maintainer, it’s obvious to me that these are good for my project. As a user, it’s obvious to me they’re an okay proxy indicator of quality. Everyone benefits without me trying to figure out why I would go through the process of having my project blessed at some level of “maturity”.

Displaying and encouraging the community to do these things in a decentralized manner is the “open source way” to go about it.

Aaronontheweb commented 5 years ago

Displaying and encouraging the community to do these things in a decentralized manner is the “open source way” to go about it.

In my view, it would be prudent to develop a set of package manager security standards and then have NuGet surface them in some way. For example, maybe you want to be warned if the owners of a package don't have 2FA enabled.

Hear hear. This is a great bottom-up way of encouraging good practices.

JesperTreetop commented 5 years ago

Keep in mind that occurred in part because Sun withdrew from the ECMA to formally standardize Java. And now Oracle Java is the de facto standard.

C# on the other hand is an ECMA standard. Also, Microsoft no longer "owns" the .NET code base. It belongs to the DNF. So another Oracle can't swoop in and take people to court for it.

[..] And not without reason, I might add. I understand it. I think it's good to bring these things up so that we can address them! For example, I believe what happened to Java cannot happen to .NET because of the steps put in place. If you disagree, let's open an issue and discuss the safeguards that would need to be in place to prevent that.

Absolutely - in terms of the direct translation, I think .NET is pretty well inoculated for what transpired there, and culture/lawyer-wise they also went easy on Mono for years when it was a third-party concern. My point was rather that people are aware that things can change - in the longer term what happened to Java (a sister/evil twin community of .NET) in a drastic reversal, but also knowing how Microsoft has changed itself, although for the better, and the impact it's had on .NET. On the list of top 50 positive changes to come to .NET in the past 10 years, I think the ECMA standard developments rank at most once, all due respect to the people driving that. And .NET was an ECMA standard even before the current wave of positive developments.

In 99.9% of the cases, the changes that are happening that are driven by Microsoft are unequivocally great. But if the point is to judge independence, it's also interesting to ask the theoretical question "would they have happened if Microsoft was not, for all practical purposes, still flying the plane?". I frankly don't know the answer to this because I can't recall a situation where it has ever come up, and I imagine people wanting .NET Foundation to be independent are having a hard time to judge too. I realize this may sound facile, but hopefully you understand the general point, which isn't even that something is wrong, but that some things that seem right enough are hard to prove conclusively.

For example, there's a lot of concern that the .NET Foundation is not independent enough from Microsoft. Keep in mind that fierce MS competitors Google and Amazon are also sponsors of the .NET Foundation. What else would you want to see to be convinced that it's truly independent?

For me, I want to see even more sponsors. MS still provides the bulk of funding for the foundation. I hope to see their funding become a smaller piece of the whole pie (by growing the pie of course!). I think addressing the challenges we're trying to address with this proposal can help that.

I fully agree with this, and I don't mean to denigrate the participation and attention of the sponsors either - I come at most of this from an outsider perspective, and normally I'd shy away but in the subject of "how does this look for people who aren't on the inside", I think it's a valid source of argument. As for Google and Amazon specifically, it doesn't surprise me since it's an everyone-runs-everything-everywhere world. If I'm not mistaken, Samsung has made contributions to .NET Core for Tizen and Intel has contributed intrinsics support, not to mention the C# features driven by out-of-Microsoft-members, and that's the stirrings of a language and platform built by a community and a village that could ultimately shut my argument down completely.

dsyme commented 5 years ago

@isaacabraham mentioned me above, so I'll write a few thoughts on this (I haven't read the thread)

My main concern about this is that I'm pretty certain that Microsoft teams will be treated differently in the process. There are several ways this happens, none of them intentional, rather incidental.

  1. Imagine a Microsoft team is launching "Gizmo for Gizmod". They are hacking out the code and the PM needs a launch plan. They decide to go opensource, but it's not yet clear how solid their management backing is - they need lots of users as proof for management, preferably enterprise users. What maturity level should the project get? Well, the Microsoft PM will be massively incentivized to make sure this project gets a high maturity level, early, as this will drive an adoption cycle. There will inevitably be pressure on the foundation to assign a high maturity level through both the official process and backdoor channels. Questions may be raised. The same is true for other enterprises and individual projects, whenever there is a chance of a functioning backdoor influence.

  2. There are other variations on the above - a company's CEO has a backroom chat with a senior Microsoft exec or .NET Foundation about maturity setting. Or a Microsoft Research project fails to get a good maturity level because it is somehow perceived differently. Or a personal connection results in a high maturity level.

  3. Even if there is a 100% watertight process, it needs an independent appeals process. And even when there is an appeals process, people might still make the accusation of unfairness. It's a nightmare.

  4. On the other end of the spectrum, I am really skeptical about the process of lowering maturity levels. Lets say a company (Microsoft or other - I'll use Acme now) has obviously stopped development on "Gizmo for Gizmod" - no checkins, issues unanswered. Who is seriously going to police the lowering of maturity levels? Who would risk setting off the firestorm of "Acme stops work on Gizmo" by lowering? Indeed would that risk a lawsuit - personal or corporate - from Acme? Or the endless haggling that might happen for a misplaced level change?

I believe the only way this will work without drama is through 100% automated level setting. But that's almost exactly the equivalent of automatic badges....

Anyway, that's my gut feeling. I'm not familiar with how Apache etc. approach this and I guess just copying their procedures may be sufficient. But the governance and independence of the process (and indeed the whole foundation) is totally critical.

dustinmoris commented 5 years ago

These are great points made by @dsyme! While the DNF might argue that 1./2. won't happen and that they can implement 3., I think 4. is almost 100% certain to happen sooner or later and that point alone is reason for not having any maturity levels and just an automated metric based system/display of metrics on the NuGet page, etc.

dsyme commented 5 years ago

@isaacabraham asked me to comment specifically on fsprojects - The F# Community Incubation Space.

This is way, way at the other end of the spectrum. It is a low-governance space where

  1. The main aim is to give a positive neutral space for early stage F# community projects (they can mature there as well - e.g. Paket - but no one really cares either way what the maturity trajectory is).

  2. Each individual repo is responsible for pretty much everything

  3. The "governance" is pretty much limited to assigning alternative maintainers should the project go dry. Sometimes projects also get uncerimoniously shoved into the "archive" when they are obviously long dead. If there are any concerns over these actions we would just reverse them to status quo.

  4. The reputation and purpose of the space is shaped by its historical successful projects (there are many of these - Visual F# Power Tools, Paket and others)

  5. Occasionally Sergey Tihon and I go on a spree where we do things like updating a few projects for .NET Standard.

It's certainly not ideal but it has allowed a lot of positive early stage work in the F# community with very little governance process. It's disreputable enough that no one particularly wants a mature project to be there (unless the project has enough reputation of its own - fsprojects is not a huge source of reputation on its own), and reputable enough to make people feel safe to move a project from their personal repo to the neutral community space (for the possible benefit of attracting collaborators)

Good examples of projects that have matured are Fabulous, Paket. There are many others.

I would advocate that a few "C# Community Incubation Space" orgs (also ASP.NET Community Incubation Space etc. etc. for sub-communities) would be good for .NET too. By their nature their can be multiple such things

glennawatson commented 5 years ago

To get in the program requires board approval.

The board is fairly diverse in terms of their current companies.

  1. Phil runs a consulting startup.
  2. Oren is charge of a consultancy.
  3. Jon works at Google UK
  4. Sara is over at Stack Overflow
  5. Ben runs a games development company.
  6. Iris runs a startup.
  7. Beth works at Microsoft.

So there are a lot of non-microsoft related people to convince here.

dsyme commented 5 years ago

To get in the program requires board approval.

If I were the .NET Foundation, I would certainly advise placing all processes related to this program in the hands of a delegated committee (which may contain one or more board members). The role of the board itself must absolutely be to govern the process, not administer the process.

For example, there will obviously need to be an appeals process, which obviously needs to be governed by the board. The board can't be judging appeals over its own decisions. But it can govern a separate appeals process.

glennawatson commented 5 years ago

Fair.

dsyme commented 5 years ago

@glennawatson It's one case where adding a level of indirection really is the right thing to do :)

glennawatson commented 5 years ago

Might be worth putting in a separate issue about the separate delegated committee btw. It's fairly independent actionable nugget.

dsyme commented 5 years ago

Might be worth putting in a separate issue about the separate delegated committee btw. It's fairly independent actionable nugget.

Oh I'll leave that up to the board :) I advised the F# Foundation on these things but in practice it requires a long term decision from the board.

haacked commented 5 years ago

Throwing the current text of this proposal in the trash in its entirety and starting the entire process over would be a good start, as it would show that the interests of the project maintainers and developers

Done. :smile: https://dotnetfoundation.org/blog/2019/09/30/rethinking-project-maturity-as-a-community-process

Aaronontheweb commented 5 years ago

@haacked sincerely, thanks for listening.

As I mentioned on the other thread, everyone who commented here should join the project support action group: https://github.com/orgs/dotnet-foundation/teams/project-support - since this is where future initiatives like this will be discussed out in the open in the future.