Closed VinodAnandan closed 2 years ago
This touches number of struggles we experienced with Keycloak submission process. I think it would be highly beneficial to incorporate ideas brought up by Vinod in this issue.
@VinodAnandan I think summarizing input from SIGs part of the process is a great idea.
Also, all the TOC meetings are public including the project backlog via a github board, so they are as open and transparent as can be. We have been opposed to time boxing as that gives the TOC less flexibility to prioritize and make decisions on the fly.
Just because your project doesn't get accepted doesn't mean the process isn't fair, the TOC has the sole authority to decide what projects it accepts. For projects that don't accepted, the TOC usually puts forth a public statement why (e.g., Kyma https://github.com/cncf/toc/pull/292#issuecomment-556393240)
re: conflicts of interest, TOC members are expected to act in the interest of CNCF and not their employers, their employers are listed on the TOC README
@caniszczyk, thank you for your response.
Yes, there is a certain level of transparency, thanks to everyone who is helping to achieve that. I do watch the TOC meeting recordings and follow the mailing list, that’s why I felt that not all opensource projects are treated fairly. I created this issue hoping that it will start some discussion and ideas for improvement.
In addition to the rejection of projects without a proper reason, there are many projects which are still waiting a year or nearly a year to enter to the sandbox. I don't know if this will help to attract more opensource projects.
In regards to the conflict of interest, we can always expect many things from others, but it’s always good to have additional measures to prevent it. Also, the TOC Chair expressed potential concerns on the CNCF TOC Meeting 2019-12-17, related to TOC members from one organization that are “sponsoring” a project from their company (“https://www.youtube.com/watch?v=I6jMrH3RAzM”)
I think we shouldn't expect one person to be an expert on all the domains and technologies, etc., that's why I am not a supporter of the "Sponsoring" concept. I think the "Sponsoring" model opens up possibilities for lobbying. It's also common that individual TOC members may get interested in particular technology stacks and projects etc, that doesn't mean that the projects and technology stacks that are not familiar to TOC members should be treated unfairly.
As @bdaw expressed the same concerns, could you please let us know which CNCF or TOC principles Keycloak is not meeting to join as a sandbox project? What are the areas they need to improve if they want to come back?
I personally am interested in more opensource projects joining CNCF. We need more sustainable opensource projects.
There is an existing PR in progress for improving and documenting the onboarding process better, including getting input from the SIGs and trying to set better expectations around timing.
One of the benefits of the SIGs is that we get to leverage expertise from a much broader group of people. The TOC still have final discretion and we would still require that a project gets sponsors, but sponsors will be making their decision taking into account input from the relevant SIG.
Also, the TOC Chair expressed potential concerns on the CNCF TOC Meeting 2019-12-17, related to TOC members from one organization that are “sponsoring” a project from their company (“https://www.youtube.com/watch?v=I6jMrH3RAzM”)
Just to be absolutely clear, I was not suggesting that this is an active problem - I have full confidence that every member of the TOC is doing their best to act in the CNCF community's best interest, and are recusing themselves in situations where their companies are involved. But by increasing the sponsor count we completely avoid the possibility altogether, which would be a good thing.
That PR is based on the "SPONSOR" model. I am requesting to reconsider that model. That model is contradicting to the CNCF and TOC principles.
Apologies, I didn't mean to say that it is an active problem, that's why I used the "potential" word. I believe the "SPONSOR" model is flawed that's why it made you think those possibilities. And the company is just one of many other factors when it comes to conflict of interest.
The SIG may spend a lot of time and effort on all their reviews similar to the project team. But still, the TOC members may not be enthusiastic. This may lead to everybody ( Project Team, SIG, Community) feeling that they have wasted their time and effort. Every project coming to join the CNCF family should be treated fairly. The TOC should consider the fact that they are willing to donate their project to the CNCF foundation and not to other foundations.
Quoting Chris "TOC members are expected to act in the interest of CNCF and not their employers". I also think that TOC members should act in the interest of CNCF, not in their personal or their employer's interest. The TOC members should uphold the CNCF and TOC principles.
I have seen different projects treated differently during their submission.
I am not against the following project joining CNCF and I believe more projects should join the CNCF family. I am just unhappy with the partiality.
For a recent submission, the TOC members got too excited and sponsored the project, without even any presentation and not completely reviewing the content of the pull request. Only after sponsoring, the TOC members have realized that the project is asking for an incubation maturity level and they thought it was a sandbox. I don't know what was the urgency to get this project sponsored, compared to the other ones which are waiting nearly a year and one even got rejected after not having a sponsor after a year. Now TOC has instructed the SIG-network to review it. I don't understand the purpose of this review. This is like a group of judges already made a judgment and then they're requesting the police officers to investigate it.
When Keycloak requested to join as a sandbox, the TOC was concerned about the governance and the team responded with their open governance and published ( https://github.com/keycloak/keycloak/blob/master/GOVERNANCE.md ). Even though the project team has only asked for Sandbox maturity, the TOC was considering it like an Incubator/Graduation project. The project answered the questions and TOC didn't ask any further questions and didn't mention which CNCF or TOC principle Keycloak didn't meet, it was just rejected saying no "SPONSOR" after waiting a year.
Keycloak is already used by CNCF member companies. I don't think the decision to reject the project without even accepting it in the sandbox level is in the CNCF community's best interest
Because Kyma was mentioned here in this thread, for the sake of transparency, I'd like to clarify that even though this comment states what is the reason for not accepting Kyma:
In #303 @VinodAnandan started a discussion that I think is more appropriate to bring back into this issue.
I would really appreciate it if you could also discuss other information shared along with the screenshot. How the TOC behaves when a project is sponsored by TOC’s company and how they behave when it comes to similar projects from other companies. Also how fast the TOC are motivated sponsoring projects that they have an interest in.
@VinodAnandan, you sent me an email with unsubstantiated claims about biased behaviour, which I would describe as "opinion" rather than "information". The only "concrete example" provided was a screenshot showing that an ex-member of the TOC had put a thumbs-up reaction on a sandbox submission PR. You are welcome to provide other evidence of wrong-doing but - as I've told you in a private discussion - I for one find these general and unsubstantiated accusations that we are acting for our own benefit pretty upsetting, and it distracts from time we could be spending actually looking at projects.
The TOC is supposed to exercise judgement and as far as I have seen, everyone tries to do that in as neutral way as they can. It would definitely be a good thing to have conflict of interest guidelines - @michelleN and @saad-ali volunteered to draft some under #270.
The TOC is under no obligation to accept projects, and does not have a set of criteria that, if passed, guarantee acceptance. It's entitled to use its discretion to judge which projects to assess, in what order and to what level of detail. If you disagree with the way that the TOC is set up to operate, concrete and specific suggestions for improvement are great! But vague and unsubstantiated insinuations are really not productive.
@derberg Kyma describes itself as a "selection of cloud-native projects glued together". This is explicitly in conflict with the principle that "The TOC does not pick a 'winning stack' - i.e., vertically integrated set of projects as a solution for multiple application problems." Given this conflict, there were zero members of the TOC (at that time) who felt it would be a productive use of time to pursue it further. We did suggest coming into the LF if the goal was to find a neutral holding ground for a community project - did you pursue that idea?
I appreciate your reply. I am not interested in any personal accusation or hurting anyone's feelings, we all are doing volunteer work and I believe that everyone's work should be respected.
I also think it's not bad to admit if something went wrong and make improvements instead of covering it up. I think it will help to avoid repeating it. I am being vague because I don't want to pinpoint people. But If you still can't understand and still believe everything was perfect I am happy to provide you with specific information required.
What was the urgency for the TOC to sponsor the "Contour" project in 5 minutes compared to other projects which were waiting for TOC sponsors for more than a year?
Why the TOC members are only concerned about untangling a project from sponsoring from a company like Redhat whereas if a similar project was from a TOC's company, the TOC members are not worried about untangling the project?
Why "establish conflict of interest guidelines" are considered only after 8 months after being raised and why it's still not a priority?
Why have you replied to me saying that "You do know that ** is no longer on the TOC"?
My understanding is that the "TOC members are expected to act in the interest of CNCF " is there any change with that?
Looking forward to your response,
Thanks,
Vinod
What was the urgency for the TOC to sponsor the "Contour" project in 5 minutes compared to other projects which were waiting for TOC sponsors for more than a year?
Contour is still with SIG Network for recommendation. In practice this one got a bit caught up in the change to the new SIG process, and two sponsors stepped up before we decided it should go through the SIG. But TOC members don't start from a blank slate, we can use our existing knowledge and experience. If as individuals we're already familiar with a project and believe it will be a good fit, why should we hold back on saying so?
Why the TOC members are only concerned about untangling a project from sponsoring from a company like Redhat whereas if a similar project was from a TOC's company, the TOC members are not worried about untangling the project?
CNCF adopted projects should be neutral and avoid vendor-specific references. We are worried about that regardless of the original source of the project. Please let us know if we have missed some references in other projects.
Why "establish conflict of interest guidelines" are considered only after 8 months after being raised and why it's still not a priority?
All of us are volunteers doing other jobs as well. We do our best. We prioritised getting the project submission process sorted out first. There is no reason why anyone else in the community could not also step forward and offer to draft this in the mean time.
Why have you replied to me saying that "You do know that ** is no longer on the TOC"?
Because you suggested that this person putting a thumbs-up on a PR was evidence that the TOC is behaving inappropriately.
Thank you for your reply.
Don't you recognise a potential chance of a conflict of interest that may arise when the TOC sponsors a project in 5 minutes which is backed by a vendor for whom the TOC member was working for?
Don't you believe that this action also results in unfair treatment for other projects which are waiting for TOC sponsors for more than a year?
Do you acknowledge that if the projects and technology stacks are not familiar to TOC members, they may not get prioritised or they may have an unfair treatment?
If the TOC doesn't have enough time for the conflict of interest guideline, why not reuse the existing one from sig-security?
@lizrice I'm no longer working for SAP and no longer involved in Kyma project. You have to check with Kyma team if they go forward with LF or not.
I added my comment to this issue only because I had a feeling that @caniszczyk's comment showed the way Kyma was handled as a good example. I just do not agree with it. I do not agree with rejecting the project basing on one sentence taken from the landing page instead of basing it on a detailed presentation from maintainers where we would for example show you part about "extending applications with serverless functions" (a sentence before the sentence that you quoted). I already shared my detailed feedback with CNCF in some kind of feedback form. I can share more details if you want, but is it needed here? what is the point?
@derberg The Kyma documentation (above all this entire page) makes it abundantly clear that the project is intended as an end-to-end cloud native "stack" tying together a wide variety of disparate components (service mesh, Prometheus/Grafana, authz, API gateway, etc.). There is thus ample information available to TOC members to decline the project on the basis of the CNCF's clearly stated principles.
If "winning stack" was a concern, why it wasn't a concern for harbor? Is this because it's backed by VMware, Aqua, etc?
Also, why is the TOC interested in bringing Istio to CNCF, which does the same ("winning stack")?
@VinodAnandan Harbor isn't a "stack" in any meaningful way. It's a container registry and thus a single component of a cloud native "stack." Istio is a service mesh and thus also not a stack. It's agnostic to API gateway, event bus, and much else.
"Harbor is an open source container image registry that secures images with role-based access control, scans images for vulnerabilities, and signs images as trusted. " - https://goharbor.io/
Harbor "vertically integrated set of projects as a solution" more information can be found below:
https://goharbor.io/docs/1.10/install-config/ (Harbor Components)
re: Istio, there are multiple service meshes in CNCF and linkerd has been there since the beginning, Istio was originally proposed to CNCF but was put hold on Google, SMI just same in a spec across service meshes which is supported by linkerd etc
re: @lucperkins made the point about Harbor, it solves a singular purpose of being a registry versus kyma, which is more of a platform. Also Quay is also being proposed, competing projects are OK and fine, it's not that they can always come in the same time like containerd/rkt did.
At the end of the day, the @cncf/toc has WIDE discretion on which projects they accept, curate and prioritize, this is a very different model than other open source organizations like say the ASF. I think your points about the TOC following some type of COI process is fair, traditionally there was an unspoken rule that TOC members wouldn't vote or sponsor projects they were affiliated with but it would be nice to codify something and it's something @amye can work on and propose to the TOC to consider in the future.
@caniszczyk , thanks for your reply. I would like to clarify that I like all open source projects including Contour, Harbor, Istio, Knative, Kyma, etc. I would love them all under CNCF as graduate projects. I am sorry if I have hurt any project maintainers or community members' feelings. It was never my intention, I was being challenged that I am being vague and there were no issues with the TOC sponsorship and prioritisation. I don't like that one open-source project got special treatment just because TOC members like it or if it's sponsored by TOC member's company. I think TOC members should also consider project merits and community interest.
As you mentioned platform, it reminds me of @kelseyhightower tweet "Kubernetes is a platform for building platforms."I also think it's better to revisit the impractical PRINCIPLES which can't be followed by current CNCF projects. So new projects don't feel it's unfair for them. The unspoken rules were broken long before and community members indirectly tried to inform this and it was being ignored. Thank you very much for considering COI for TOC. I hope it will be considered as a priority.
@lucperkins as you might notice I'm not questioning the decision of TOC (we knew we are not perfect but we were opened for discussion, recommendations and adjustments, as we understood sandbox purpose is exactly this), I'm just questioning the style of how the decision was made and communicated. If anyone from CNCF or TOC is interested in detailed explanation and timeline I'd love to share, but privately as I'm not feeling comfortable sharing specific names in public as I'm aware this is all not a verbal real-time discussion and people interpret words literally, pick on the words, and that might lead to misunderstanding of the intentions.
again, just to highlight, I'm no longe at SAP and Kyma
@VinodAnandan
I would like to address the points that you have raised above. You have spent a lot of time making claims, implications and half-implications about conflicts, unfairness, and more. You have not yet provided any evidence other than repeated insinuations. I would ask you to be explicit about your statements and back them up with concrete evidence. Or kindly withdraw them and desist from your derogatory tone which insults the substantial volunteer effort from all of the community and TOC. I remind you that nobody is paid to do this work except the LF-CNCF exec staff who are neutral. Everyone has some point of view or history and we have tried VERY HARD to make certain that the TOC operates with high integrity and without conflicts.
To your points:
1) Don't you recognise a potential chance of a conflict of interest that may arise when the TOC sponsors a project in 5 minutes which is backed by a vendor for whom the TOC member was working for?
Please be explicit - are you talking about Contour?
Is it your view that no projects should be proposed if they have any association with the TOC? This would rule out Kubernetes from the CNCF. When you talk about conflict do you understand that the TOC is drawn from competing companies? I just don't even understand where you get all this stuff from. It is profoundly insulting to the efforts people are putting in here.
2) Don't you believe that this action also results in unfair treatment for other projects which are waiting for TOC sponsors for more than a year?
But Contour has not jumped the queue - it has still to get out of the SIG stage and then the role of sponsors comes into play.
Unfortunately, we had a long hiatus and took no projects for months. This was because the TOC was overwhelmed. But this was "unfair to everyone" not to a selection of projects.
Do you have any idea what it feels like to be a volunteer dealing with dozens of "urgent" projects in your free time, when you have work and family to think about? We have now resolved this problem, I hope, with the SIGs. I explained all this already on the TOC list, and I don't know what you still harp on about it. Can you even begin to see things from anyone else's point of view?
3) Do you acknowledge that if the projects and technology stacks are not familiar to TOC members, they may not get prioritised or they may have an unfair treatment?
The TOC has gaps it is true. But, this is why we created the SIGs. Which SIGs have you offered to help with?
You may not be aware of this, but it is also the case that the TOC is expected to be opinionated. And not all the projects that get proposed are a good fit, or really appealing. Ask yourself if you think the CNCF would be improved if ALL projects immediately had the required number of sponsors. Is that what you want?
4) If the TOC doesn't have enough time for the conflict of interest guideline, why not reuse the existing one from sig-security? https://github.com/cncf/sig-security/blob/master/assessments/guide/security-reviewer.md#conflict-of-interest
The TOC deals with conflicts by the conflicted person declaring their conflict and recusing themselves. This happened in my case with Cortex, which was proposed by multiple companies including Weaveworks. You should also note that I was the first to offer to support Thanos, which is an alternative to Cortex.
alexis
Hi @derberg
I wanted to speak about Kyma since this is clearly a sore topic still. I am really sorry that the decision to push it back was hurtful in some way. Unfortunately in a world of very limited attention, time, and other resources, plus a lot of demand for that time, we make decisions based on what is in front of us. The only good news is that like with any form of 'investment', you can always try again with changes.
A few more things about Kyma.
1) I appreciate you feel that the TOC may have missed something important by not seeing a presentation. Please be assured that the whole TOC dedicated a substantial chunk of a valuable F2F talking about Kyma and we were all unanimous that it was not currently a fit for the CNCF. So having further presentations would waste SAP's time and our time and the community's time, and be unfair to other projects waiting to present.
2) No information has since been presented to make anyone change their minds.
3) The TOC is intended as an opinionated group that literally makes judgements. The process was followed entirely correctly, if anything favouring Kyma with extra time because people felt bad that we had to say no.
4) Comms about this could possibly have been better. I think the TOC was certainly clear. The post-Kubecon holiday break and other conferences may have gotten in the way. But in the end, the TOC is overwhelmed and volunteering. I think having SIGs will help with future cases. I apologise for any hurt caused.
5) We are not against "stacks" we are against blessing one stack, and we are against projects that don't solve real problems.
Kyma failed to get support because nobody could see what problem it solves. If you make a big collection of projects into one larger project, why is that specific collection useful? It is not enough to say 'this is what people might need to install'. If you say "we really see a big need to solve this problem (x) in an opinionated way, and that means you have to combine ABC projects with XYZ integrations, thus and so, and then layer PQRS new code on top.... then you are off the races. You have a new larger project because you are using code to solve a problem.
Otherwise it is just a 'distribution'. The issue of creating a distribution is very complex and tied up with how upgrades, patches, and testing work, which is something we don't think the foundation (CNCF) should own. It is fine to do an independent OSS project here.
alexis
I'll use my right to remain silent 😄 I always wanted to say it in some situation (influence of American cinematography)
Any response from me would continue discussion only about the past as I'm not aware of what is the future of Kyma, thus I won't advocate for it. I learned my lesson the hard way 😄
Yes, I am talking about Contour and only referring to the TOC sponsorship/support and not to the entire process. In case if you haven’t noticed the email and commits, the email sent on 7th Jan
“I know we’ve talked about not syncing on TOC meetings to get new projects in. As things stand, we have just submitted contours as a sandbox project and there is support from Alexis and Matt and me. What do we want to consider necessary before we move forward? Do we want to have SIG-netowk take a look or do we want to just "make it so"?
https://github.com/cncf/toc/pull/330/commits/b84b7a600433a51b20411e89d7aa2b17a8698c10
"Ah - my mistake. I know we talked about it. Chirs added the "sandbox" label so that confused me. I'll fix it.
Agree that if we are talking incubation we should definitely have SIG-network start the process"
Multiple CNCF sandbox submissions are waiting for the TOC sponsorship/support for more than a year. Some of them (e.g: ORY Hydra) even decided to withdraw their submission. Some of them (e.g: Kamus, Keycloak) don't have TOC sponsorship/support yet. And TOC still doesn't want to be open and transparent about it. But for Contour, the TOCs have decided to sponsor/support in less than 5 minutes. I still think that it was totally unnecessary as 1 TOC had a hard conflict and 2 TOC has soft conflict. Also, two of the TOC's term was about to expire. I don't believe that Contour has some special status to be considered as an urgent situation over other projects that are waiting for TOC's support.
As I am working for a CNCF end-user company and we use some of the submissions, we appreciate these open-source software to join vendor-neutral home-like CNCF. I have contacted multiple other CNCF end-user companies who also use them, unfortunately, none of us have a voice in TOC. That's why I have also offered my help to the TOC with any assessments/information required, I have reached out in the public mailing list as well as TOC's email addresses multiple times. Still, the TOC won't say what stops them not to sponsor/support or what changes the projects need to make to get them the TOC support/sponsorship. And multiple project representatives are reserved when it comes to reaching out to the TOC, that's why I decided to speak for them.
I don't have anything against Contour or any other opensource projects. In my career, I have only worked for the end-user community companies maybe that is the reason I like all opensource projects. I don’t know all the interest of the vendor firm that has backed the OSS. As most of the TOC members are working for vendor companies you may have a good understanding of it, and if there are any issues, it would be great if the TOC could be open and transparent about it. Every project should be treated fairly it shouldn't have a special status if TOC's company back the project or TOC has a special interest. In my view, all projects should be challenged in the same manner.
If TOC can't solve the issue with COI and have more than enough workload, then please empower the SIG instead of using them in an advisory capacity.
@VinodAnandan - would you be interested in drafting a conflict of interest statement for the TOC and taking over #270 ? I'd also be happy to work with you it. Feel free to ping me on slack as well.
@michelleN Sure, I am interested and happy to help, that's one of the reasons I keep pursuing the COI with TOC. Thank you very much for considering this as important.
At the same time, because this has been going on for so long and more than enough information has been provided, I think we should wait a little more time and see if there are any further replies.
Sure np. Whenever you're ready, feel free to open a pull request and put it under toc/process/conflict-of-interest.md (or where ever you see fit). An alternative would be to start a google doc and share it here.
@lizrice @monadic any further comments on my last reply which explains details of the COI with Contour submission? If you have any other comments, please let us know as soon as possible. I also think the COI shouldn't generally discourage the TOC to support/sponsor projects.
@VinodAnandan ping!
Closing as part of end of year cleanup, can be resubmitted if there's something outstanding!
I would like to thank and congratulate the amazing work done by the Technical Oversight Committee (TOC) and the volunteers from SIG. Your work is very much appreciated and it has a significant impact on the future of CNCF and open-source projects. Similar to that, many others spend their significant time and effort on different open-source projects. I think having more transparency with the TOC selection process will bring some level of fairness to them. Some of these are already popular and widely adopted open-source projects and the rejections from CNCF - even to accept a project for sandbox, have a great impact on their community as well as the open-source maintainers. Please at least provide some justification as to why it was rejected. Sandbox by definition itself is just a very early stage and there are mandatory reviews and processes to get promoted further. I feel like CNCF TOC is making it very difficult for open-source projects even to come to sandbox, compared to many other foundations. And it’s unfair especially for some cases where sandbox projects are treated with very strict measures compared to others.
It's really good to see that multiple TOC members are interested in transparency when it comes to the TOC nomination process. I think Joe and Alexis and other TOC members have expressed concerns about reaching out to the TOC individually and potentially lobbying them for sponsorship. Also, I think many TOC members have concerns about having two TOC members from the same company sponsoring their projects. I do think that these are all valid concerns that could be resolved if there is willingness to change the current TOC operation model ("Sponsor") and have more transparency in the TOC process, operations, and responsibilities. I think the process should be equal, transparent and time-bound to all projects to keep fairness. Some of the projects have to wait more than a year and they have answered all TOC’ questions, after that without further justification and explanation, there comes a rejection. Whereas for other projects within 1 day there are 3 TOC sponsors ready to support without any presentation, SIG reviews, etc.
I believe in the final review stage of the sandbox project, every TOC member should review the great work done by the SIG as well as the project team and they must clarify any doubts they have from both within a fixed timeline. All of the TOC members should add their review in the pull request like a Positive and/ or Areas that need improvement. Once there is a quorum, the TOC liaison should combine them and include a final summary that should be sent to the project team in case of rejection, this way they will know what areas they need to improve if they want to come back. I think this work done by TOC should be considered for attendance as well. I don't think showing up for every TOC meeting itself should be considered a full attendance. Also, including a note above the review if there are any potential conflicts of interest ( e.g: I work for this security vendor who may have products/services which conflict, any other relationships with the project, members, company, etc.). The acceptance criteria should be based on the merits and facts of the projects and not on individual TOC member's intuition.
I don't think that just having more TOC members will necessarily help with the TOC workload, CNCF, and open source projects if the proper responsibilities and transparency are not in place. I also don't think the TOC members are actually "sponsoring" anything for the open-source project. The sandbox acceptance process shouldn't be unfair, restrictive and slow.
"The CNCF will strive to adhere to the following principles" ( source: https://github.com/cncf/foundation/blob/master/charter.md#3-values )
(a) Fast is better than slow, (b) Open, (c) Fair, (d) Strong technical identity, (e) Clear boundaries, (f) Scalable, (g) Platform agnostic