Closed brianraymor closed 5 years ago
brianraymor
commented
5 years ago Responding to the review comment from @hannes-ucsc - The discussion for adding pipelines for each deployment in the withdrawn RFC is here. Consensus was not reached before the RFC was withdrawn. I should have noted this in the open questions section of the new RFC. I apologize for the oversight.
brianraymor
commented
5 years ago @gabsie - Cara and I appreciate that thoughtful reviews from our UX team. Regarding your questions:
So my question is: what happens if the functionality which is in the annual roadmap is not answering user needs? What happens with ongoing feedback which comes from usability testing?
Based on Adrienne's suggestions, we've documented that the Roadmap process will incorporate information from both the UX survey and any available UX Insights. But what is there are emerging issues similar to the Metadata confusion in Beta 1? I'd suggest that this could be addressed in multiple ways:
The open question is whether we would interrupt or cancel a Release to focus on an emerging problem or wait until the next Release. Even in Scrum, it's not recommended to cancel Sprints:
Sprint cancellations consume resources, since everyone regroups in another Sprint Planning to start another Sprint. Sprint cancellations are often traumatic to the Scrum Team, and are very uncommon.
CaraLMason
commented
5 years ago Hi, @tburdett ! My response turned out to be an unintentional novel, so please let me know if you want me to set up a brief discussion on your feedback. You’ve raised some really good points, which echo some similar feedback that we’ve heard from some in the community, so thank you for articulating your concerns. Below is my far too long response. Happy reading. :)
We believe that what we have proposed enables PLs to effectively and replicably generate an annual roadmap, as the fulfillment of Oversight's explicit direction, with a clear path forward to do so. You're correct on the length of time being long for the annual roadmap, and we're in agreement there. You, Adrienne, and others have given good feedback on that, and we've clarified in recent changes (Aug 7 commits) about the role of the user survey/UX input in the overall timeline. Additionally, we think that the length of time ensures that we are working to get the right user and technical input which is something that we definitely agree with you on is important.
To your point regarding "faster horses" (those Henry Ford references really do stand the test of time :)), that is also part of why we felt a longer roadmapping process would be beneficial. It would provide the time to ingest what users are saying, understand that the horses they are asking for might not be the best/right solution, think and iterate through "assembly line" type innovations are possible, and formulate a coherent vision for the next year for the DCP. We think that you've done a lot (read: all) of the leg work for the coherent vision for the DCP via the strategy doc, which would seem to us to be able to be translate well and easily into a roadmap. For this year, having you spend the amount of time and effort you did on the strategy doc was absolutely fantastic, and, for future years, we want to help ensure that, if you or anyone else doesn't have that kind of time or willingness to put a stake in the ground, we can have a roadmap that helps ensure the long-term longevity and growth of the DCP, to ultimately serve users more effectively.
Further, we are in absolute agreement that incorporating user feedback iteratively is important. That is why we are proposing that the roadmap be done as an RFC, so that, as user needs become apparent, we have a clear, consistent path to address those changes, in a way that keeps the internal and external community involved and gives everyone a central point/place to go to know what the latest version is. Having the roadmap exist as an RFC also gives PLs the ability to quickly update it, which ensures that we maintain close ties to feedback from the community. If you have specific language in mind around ensuring that the roadmap is updated iteratively and quickly from community feedback, we are very open to it!
Additionally, to the point of this RFC being a departure from current practices, we are in agreement with you. It is a departure from current practices because Oversight has asked for things (predictable, high-confidence releases, an annual roadmap, etc) that require a response that diverges from previous processes. More than that, we aimed to give the DCP a framework by which to execute as a collective. I think that we can agree that working on a project with this many distributed teams, cross-component work and dependencies, and technological uncertainties leads to an extraordinarily complex beast to wrangle, for which all of the principles of Agile, which was originally only intended to be used in teams that were physically co-located together, may not work. In light of that reality, we have sought a middle ground that gives teams a clear, consistent method to execute, which we have heard from all teams is a historical point of frustration, that we fully intend to iterate with the community on.
As with everything in the DCP, we intend for this process to change as the needs of our community and users change. This RFC is not intended to be terminus of discussion, but, rather, a framework to reconstruct (or deconstruct :)) over time, while keeping the ways that we, as a collective, execute clear for everyone. After all, when we create anything in the DCP, the process by which we do so is, normally, unconsciously manifold. This doc changes some of those complex processes, but, more than anything, spells out how we are already executing.
All of that to say, we believe that we are in alignment on the core aims of what we are trying to do and the method that we are getting there is slightly divergent. Again, I’m more than happy to set up time to understand more about your concerns! We want to ensure that we’ve fully heard your input!
I'm not going to comment on specific contents of this pull request, as I think the entire document is built on flawed foundations. I have several major overall concerns.
- This RFC makes several invalid assumptions about the relationship between the DCP and it's users. It assumes that our users know exactly what they want, that we have a perfect understanding of what our users want, and that we should build exactly what they are asking for. As Henry Ford said, “If I had asked people what they wanted, they would have said faster horses.”
- This RFC emphasises predictability over value. Compounding the assumptions above, it holds the ability to deliver exactly what was promised, exactly on time, over the ability to incorporate frequent feedback from users into our plans. This is an almost impossible goal (see e.g. CHAOS report)
- This RFC emphasises long, deterministic cycle times (e.g. annual roadmapping process that last 70 days, annual UX surveys) over short feedback loops. This would rob the DCP community of opportunities to incorporate continuous input into our aims, which in turn gives us low confidence that prioritised deliverables will meet the needs of the HCA. There are concessionary statements about how the roadmap can be updated, but there are significant barriers in the way of doing so, and this document clearly disincentivises this sort of behaviour.
- This RFC wishes to establish a process that fails on all four principles of the agile manifesto, prioritising process and tools (e.g. mandated use of Zenhub) over individuals and interactions, contract negotiation (see e.g. planning and executing a release) over customer collaboration, following a plan (e.g. annual roadmaps) over responding to change, and comprehensive documentation (RFCs) over working software. If the unstated intent is to adopt waterfall practices, it succeeds, but this document explicitly includes the assertion that it avoids radical departures from current community practice.
I won't approve this PR without very major revisions.
hannes-ucsc
commented
5 years ago Additionally, to the point of this RFC being a departure from current practices, we are in agreement with you. It is a departure from current practices because Oversight has asked for things (predictable, high-confidence releases, an annual roadmap, etc) that require a response that diverges from previous processes.
In my experience, this is an impossible requirement to meet. Leaders like Agile because it gives them transparency into the current progress plus the ability to detect failure and change course early. But they fail to meet the other end of the bargain, which is to accept the absence of long-term predictability.
More than that, we aimed to give the DCP a framework by which to execute as a collective. I think that we can agree that working on a project with this many distributed teams, cross-component work and dependencies, and technological uncertainties leads to an extraordinarily complex beast to wrangle, for which all of the principles of Agile, which was originally only intended to be used in teams that were physically co-located together, may not work. In light of that reality, we have sought a middle ground that gives teams a clear, consistent method to execute, which we have heard from all teams is a historical point of frustration, that we fully intend to iterate with the community on.
This middle ground may not exist. It seems counter-intuitive to assemble multiple agile teams into a traditional, "predictable" and waterfall-like process. Uncertainty only increases the more parts one adds to a system. But the biggest contributor to uncertainty here at the DCP is the fact that officially nothing changes without hard-fought consensus, while many things change unofficially without it, and therefore without direction.
diekhans
commented
5 years ago Oversight has asked for things (predictable, high-confidence releases, an annual roadmap, etc)
Release of software or data? I believe many of the problems with the DCP is the lack of alignment on the goal being to produce software or produce data.
tburdett
commented
5 years ago @CaraLMason thanks for the detailed, thoughtful and honest response. You haven't addressed my concerns - indeed, you have confirmed them - but I appreciate the time you've taken to outline your rationale.
We obviously have the same goal in mind - we all want the DCP to be successful, and we want to ensure our teams can build a really successful DCP as effectively and as efficiently as possible.
I suspect where we really disagree is on the challenges we face in meeting this goal. This RFC doesn't really give an indication of what problems you currently perceive, though. In the motivation section, you cite a single bullet from John Marioni's slides at the January face to face:
A lack of reproducible planning process that prioritizes deliverables and ensures confidence that they will meet the needs of HCA
...but "predictability" does not feature here. On the contrary, I would say the key point is "ensures confidence that they will meet the needs of HCA". Further, the previous two bullet points on this slide are:
- Translating broad scientific direction into actionable engineering requires tight iteration cycles around prioritization of deliverables
- It is vital to maximize contributions while enabling rapid iteration and progress: Clearly identifying roles, expectations and direction requires prioritization of key deliverables and a process for implementation
...which clearly speak to the need for fast iterations incorporating user/scientific feedback. I have also never seen the Oversight group give explicit direction - their intent (as stated in the same slide deck) is to ensure that "each team will become increasingly empowered to plan and implement with greater independence and confidence". This RFC significantly hampers the independence of teams without explaining what gains will be achieved in doing so, or how any such gains might be observed and quantified. Without this, it is impossible to hold this process to account.
Even if I shared the conviction that predictability was in and of itself a problem, the four sentences in the motivation section do not provide adequate justification for the detailed and far-reaching proposals in this document. Throughout this RFC, where new or modified processes are defined, there is scant indication of what value they will provide to the DCP community or what problems they are intended to address.
Having said all of this, I am not wedded to any particular process, and I support the autonomy of those who will have to run it to determine the best process for themselves. I also recognise that compromises are needed to ensure the teams that make up the DCP community can work together effectively. If this direction represents the consensus of the DCP community, I will be very surprised, but I will happily stand corrected. In this case, to address the four concerns I outlined in my previous response you will need to (respectively):
NoopDog
commented
5 years ago officially nothing changes without hard-fought consensus, while many things change unofficially without it, and therefore without direction
Well said @hannes-ucsc
brianraymor
commented
5 years ago @tburdett - regarding your last set of comments:
Ensure that any proposed processes actively encourage all DCP teams to incorporate early and continuous user feedback into their design processes, rather than creating significant barriers to doing so, effectively endorsing ignorance of user needs for up to a year at a time
Can you point me towards the section where the RFC endorses ignorance of user needs for up to a year? And elaborate further on the significant barriers? While there's an annual roadmap, the Release planning process occurs on a three month cadence and incorporates emerging UX Insights and other material. Gabby has suggested some excellent clarifications for this case. Also see my comments to Gabby about the earlier case of metadata confusion in Beta 1. If this happened again, I would expect that this would be prioritized in the next Release and all hands on deck. No one would be confused whether it was a priority and whether they should be working on it.
Prioritise outcomes with positive value to our users (what I would call "deliverables") over the predictability of our work outputs
These are never mutually exclusive in my experience. Engineering teams are always focused on the highest DCP priorities expressed by the Roadmap, so predictability is not prioritized over customer value.
I prefer accountability to predictability.
There's accountability to deliver customer value in a Release - with estimates set by the Technical Leads. Furthermore, as estimation improves, it will become easier for us to schedule links in a chain of dependencies with confidence. For downstream components, this is especially important. It certainly helps to schedule other critical work while waiting for a dependency to complete.
Ensure that the defined process encourages short release and feedback cycles. I should stress that this is not at odds with the idea of creating an annual roadmap (an initiative I fully support and indeed have embraced in my attempts to help define a long range strategy for the DCP), but it does seek to acknowledge that any annual roadmap is based on our current understanding and will - and should - change frequently.
Three months sounds about right to me, but the DCP is continuously providing value as completed Roadmap objectives appear in Production.
Based on feedback from Phil, Jonah, and Brian O'Connor, the roadmap will also be reviewed and updated on a quarterly basis with approval by Oversight. I'll be making this change tomorrow to Updating the Roadmap.
State very explicitly what advantages shifting to waterfall practices will bring, and the anticipated costs and risks of doing so
With respect ... pass. I do not want to be distracted by an endless philosophical argument about methodologies or agile. Be happy to have those conversations over beer. The DCP needs a shared approach for roadmaps, planning, and execution. We've prototyped part of this approach in Q2 and Q3. It's already clearly demonstrated areas for improvement as I've privately discussed with you during our last phone conversation. It's definitely not perfect, but I'm sure that we can refine as we iterate.
NoopDog
commented
5 years ago I fell this RFC defines a fairly light process that describes the current practice that has evolved over the last few months and feel it would serve us to adopt this RFC with a few minor changes:
1) The one year roadmap does seem long to me. If we add a rolling quarterly review of the roadmap to update it for evolving priorities I fill this will make this RFC less waterfall.
2) We don't need to decide Zenhub columns here. We already have a set of columns that seem to work ok and if one team needs a local workflow they can create their own board now that Zenhub allows tickets to exist in multiple boards with different statuses.
I feel the process defined of course does not solve all of our process problems but does lay the groundwork for us to continue to improve our process. I feel the RFC describes a process that:
Provides a framework where we can iterate on improving our requirements process.
I would be happy to see this plan be formalized and then iterated upon then. We are already doing almost all of this and to me it seems to be quite an improvement over our previous process both in communicating what leadership what us to build and communicating to leadership our progress and challenges.
Cheers, D
brianraymor
commented
5 years ago @jahilton - Regarding your comments:
In general, I agree with concerns about assuming predictability, and ensuring that more adaptability to the scientific community be considered.
Please see this related response.
What do you specifically mean by assuming predictability?
Do you have concrete recommendations for ensuring that more adaptability to the scientific community be considered?
justincc
commented
5 years ago Additionally, to the point of this RFC being a departure from current practices, we are in agreement with you. It is a departure from current practices because Oversight has asked for things (predictable, high-confidence releases, an annual roadmap, etc) that require a response that diverges from previous processes.
In my experience, this is an impossible requirement to meet. Leaders like Agile because it gives them transparency into the current progress plus the ability to detect failure and change course early. But they fail to meet the other end of the bargain, which is to accept the absence of long-term predictability.
In my experience there is no long-term or even medium-term predictability when it comes to software, no matter how much planning you do. What matters is how quickly you can respond to change, hence agile. Is the proposed process one that makes it easier to respond to change or harder?
jahilton
commented
5 years ago @brianraymor my general concern is that we're being guided from within while we should be guided by the needs of the community. The Scientific Roadmap will only happen annually and will provide high-level 'direction of the field' rather than boots-on-the-ground needs. I am concerned that the annual user survey is too infrequent and may not yield enough responses to give us an accurate representation (I may be wrong there). I would be interested to hear if UX has ideas on more frequent user surveys, surveys narrowed to specific DCP features, and/or more explicit involvement throughout the Planning/Roadmapping process.
gabsie
commented
5 years ago @brianraymor my general concern is that we're being guided from within while we should be guided by the needs of the community. The Scientific Roadmap will only happen annually and will provide high-level 'direction of the field' rather than boots-on-the-ground needs. I am concerned that the annual user survey is too infrequent and may not yield enough responses to give us an accurate representation (I may be wrong there). I would be interested to hear if UX has ideas on more frequent user surveys, surveys narrowed to specific DCP features, and/or more explicit involvement throughout the Planning/Roadmapping process.
Hi, @jahilton - I had an extensive comment and review on this topic, specifically addressing your worries. Brian made some changes and entered comments.. https://github.com/HumanCellAtlas/dcp-community/pull/92#pullrequestreview-273223733
The thing which I think might help us probably move the discussion along, is to see an demo/example/prototype of this annual roadmap based on current state of affairs (hopefully it doesn't require 60 days to create this prototype). That might help the teams understand how it can help/not help, if it looks like its workable or not, if it's constraining, flexible. Because I think as a prescription or a template at the moment it has generated a lot of very similar reactions.
brianraymor
commented
5 years ago @justincc - Regarding your note:
In my experience there is no long-term or even medium-term predictability when it comes to software, no matter how much planning you do.
We should compare anecdotes in Boston. I've had the opposite experiences as both a Developer and PM. Microsoft has hard delivery windows with their OEMs for example.
What matters is how quickly you can respond to change, hence agile. Is the proposed process one that makes it easier to respond to change or harder?
It depends on how you respond. Definitely want to avoid "When in trouble or in doubt, run in circles, scream and shout". Without agreeing with your "what matters ..", my response is Easier - because the question is like "Which is faster - a small pony or an (imaginary) unicorn?" I honestly do not know what this process is being compared against, because there is no other DCP wide process proposed or practiced.
I attempted to address similar concerns in my response to Gabby.
Agile. The agile manifesto is a set of principles. Principles are not processes. Could we agree to stop using "agile" as a mantra or ward and be specific about concrete improvements or alternatives to this process? Even the original authors feel that the word "agile" has become meaningless and have harsh words for how "agile" is being practiced:
But in the 14 years since then, we‘ve lost our way. The word “agile” has become sloganized; meaningless at best, jingoist at worst. We have large swaths of people doing “flaccid agile,” a half-hearted attempt at following a few select software development practices, poorly
As I suggested in the previous incarnation of this RFC, be prepared to pick up the pen as we say in editing circles and propose better ideas.
justincc
commented
5 years ago As I suggested in the previous incarnation of this RFC, be prepared to pick up the pen as we say in editing circles and propose better ideas.
Sure. I'm usually too busy but this weekend is a Bank holiday (Mon holiday in UK) and I'm on holiday Tuesday. So I'll crank out a document with alternative ideas for at least the development parts. If you don't see it by end of next week that's because I didn't actually have time or I realized that the ideas aren't actually any good in this context. The aim will definitely not be to replace anything but to advance an alternative approach for criticism which may offer elements for synthesis.
tburdett
commented
5 years ago I shall leave discussions of agile vs. waterfall aside - I would agree that they're not particularly useful. In my initial review, I simply sought to point out that this is a "radical departure from current community practice" as most DCP community members strive to pursue agile practices. Please remove this statement. I'd also ask you to consider a wider review - for example, given the quarterly Release cadence, why would features be pushed to production on a weekly cadence? It seems inconsistent to have shorter deployment cycles than release/feedback cycles. Why not allow dedicated integration and QA time towards the end of a Release, removing pressure from engineering teams and mitigating against production instability?
The DCP needs a shared approach for roadmaps, planning, and execution.
Leaving aside my own personal judgements, this RFC does not convince me of this case. However, I have been a vocal critic, and if the process is adopted I am therefore prepared to play the entirely healthy role of holding this process to account. In order for me to do so, though, I'll repeat my earlier request: you MUST clearly state the intended goals of any suggested processes and what assessments one can use to determine their success. The current acceptance criteria simply states:
If the process fails to deliver product increments with high confidence after three quarters with an annual Roadmap, then more radical alternatives SHOULD be pursued that satisfy the requirements from Oversight
This is not sufficient to hold the process to account, and arguably no better than the previous baseline. "Confidence" and "product increments" aren't defined. In my judgement, no community processes or practices, defined or otherwise, have yet failed to deliver forward progress. Practices used early in DCP development (2017 and early 2018) identified specific targets that were hit reliably, but resulted in misaligned sets of features. How will you convince me (as your dedicated skeptic) that this process is working and not adding unnecessary overhead?
As a specific example: one of my goals would be to ensure we fail fast and learn from our mistakes, avoiding a lot of wasted development effort. The metadata confusion that emerged during beta testing is a good illustration of where we fell short of achieving this goal. How do you intend to demonstrate that this process does not repeat those failings?
brianraymor
commented
5 years ago @tburdett - responding to your note quickly before you disapparate on vacation.
I simply sought to point out that this is a "radical departure from current community practice" as most DCP community members strive to pursue agile practices. Please remove this statement.
Fair. This was a copy-n-paste from the previous Scrum-mier RFC. Addressed in 7869c2644b290095c33bc536027f0eace2c41861
Why not allow dedicated integration and QA time towards the end of a Release, removing pressure from engineering teams and mitigating against production instability?
I've previously added Integration to an update for buffering capacity that includes emerging requirements from DevSecOps - 3223f163996d6903cf16031a1c790e51bab31d44 - the failure to reserve buffer was one the case studies that I shared with you during our earlier phone conversation.
There's also a pending item in the top-level summary comment of this RFC where I'm tracking work related to comments:
It's unclear what you mean by pressure on engineering teams. As I quoted, individuals doing work say how long. Key to a plan is leadership sets time constraint then work scales. What is the source of this pressure or are you speaking in the abstract?
The DCP needs a shared approach for roadmaps, planning, and execution.
Leaving aside my own personal judgements, this RFC does not convince me of this case.
The requirement for a shared approach is from Oversight. And as noted in their August 20 meeting notes forwarded to DCP PM by Jonah:
"In addition to finalizing the roadmap, there is a need to settle on a singular process for planning. The planning and execution process that the teams used in Q2 and Q3 demonstrated that it can produce needed velocity on end-to-end themes. We should proceed with the planning RFC as a starting point that can be evolved as improvements are identified."
I've added a tracking note in the top-level summary comment to update the Acceptance Criteria section. Since Oversight will be reviewing Release progress with the Project Leads, it seems like Oversight could also offer guidance if this process is failing to address the needs of the community.
As a specific example: one of my goals would be to ensure we fail fast and learn from our mistakes, avoiding a lot of wasted development effort. The metadata confusion that emerged during beta testing is a good illustration of where we fell short of achieving this goal. How do you intend to demonstrate that this process does not repeat those failings?
I believe that this was addressed in my response to Gabby.
brianraymor
commented
5 years ago @gabsie - regarding your comment
The thing which I think might help us probably move the discussion along, is to see an demo/example/prototype of this annual roadmap based on current state of affairs (hopefully it doesn't require 60 days to create this prototype). That might help the teams understand how it can help/not help, if it looks like its workable or not, if it's constraining, flexible. Because I think as a prescription or a template at the moment it has generated a lot of very similar reactions.
Based on the Oversight request for a roadmap by August 30 (which will not use this RFC process), we'll have the example soon. As discussed on the August 22 DCP PM call, Cara and I are also eliminating the annual model and re-writing the roadmap process to be incremental based on quarterly updates to this ur-Product Roadmap.
tburdett
commented
5 years ago It's unclear what you mean by pressure on engineering teams.
There are many procedures in place for handling weekly software production deployments, including nominations of weekly release managers, integration test monitoring responsibilities, scale testing, and SOPs to prevent the ingestion of data during weekly software releases. I've observed that when weekly releases don't go smoothly, it requires a large investment of unplanned work. But maybe one of the tech leads should comment here. I note that there hasn't been input from many of the tech leads and that there isn't an approving review from a tech lead at the time of writing.
The requirement for a shared approach is from Oversight
Nonetheless, this isn't the only shared process possible, so I still expect a justification of why this process should be adopted. You have decided to write this RFC, so you must have goals in mind that you believe will be accomplished by a successful shared approach to planning and roadmapping. Please declare them, so there is at least some chance of evaluating success. I'd like to see a convincing argument that, even if I disagree with, I can see the rationale for. This is currently a diktat.
I've added a tracking note in the top-level summary comment to update the Acceptance Criteria section.
This is extremely welcome, thanks.
Since Oversight will be reviewing Release progress with the Project Leads, it seems like Oversight could also offer guidance if this process is failing to address the needs of the community.
Additional input is obviously welcome. Oversight did not write this RFC, though. Please outline the manner in which this process is expected to address the needs of the community. It is impossible to identify potential improvements if the goals of this process aren't explicit. And conversely, with explicit goals, you'll have an opportunity to convince me that I was wrong to be skeptical when they are met!
I believe that this was addressed in my response to Gabby.
Your response describes what the proposed mechanism is, not why it exists or how to demonstrate it will address my concerns over wasted development effort. In the metadata example, more than eighteen months of design and development was invested in the creation of services that were not fit for the purpose of presenting experimental metadata to users. A key success criteria must be to ensure we don't repeat this.
Reading between the lines, one might infer that the ability to Update the Roadmap and Clarify the Next Release is meant to enable goals such as:
I'm really looking for anything (here and elsewhere) that can help the DCP establish performance indicators to ascertain whether this proposed process is successful. Without these, the only possible assessment of this process is dogmatic (probably the reason for the "waterfall vs. agile" debates).
brianraymor
commented
5 years ago @tburdett - regarding your note, I appreciate that you've clarified engineering pressure. This is why I've previously encouraged Technical Leads to include buffer in their estimates. I believe that I've responded to this concern in detail in my previous response to you.
I note that there hasn't been input from many of the tech leads and that there isn't an approving review from a tech lead at the time of writing.
Cara and I voluntarily offered an additional week for community review after polling the Technical Leads. Some Technical Leads who voted for the poll then failed to review or request additional time. Justin approved earlier, but then joined Hannes in his reasonable request for clarification about Pipelines. I believe that I have resolved that issue with recent experimentation and shared the results with Justin, Hannes, and Trevor. I need to update the PIpelines section for review.
Dave Rogers left a balanced and positive review:
I would be happy to see this plan be formalized and then iterated upon then.
and has requested some changes which will be addressed this week.
brianraymor
commented
5 years ago @hannes-ucsc, @justincc, and @NoopDog - does 9538ae58b7aff3c453489c1aead661166deec270 address your questions and concerns about pipelines.
hannes-ucsc
commented
5 years ago @hannes-ucsc, @justincc, and @NoopDog - does 9538ae5 address your questions and concerns about pipelines.
It does, thank you. The Azul/DB team is currently trying out our own workspace and it is going well.
There is currently some discussion among the tech leads about their own internal consensus building process. Since that is process is partially covered here as well, I would like to wait and see how that conversation pans out before I make a call on this RFC.
brianraymor
commented
5 years ago @hannes- regarding:
There is currently some discussion among the tech leads about their own internal consensus building process. Since that is process is partially covered here as well, I would like to wait and see how that conversation pans out before I make a call on this RFC
Cara and I are not planning to pause the progress of this RFC while Technical Architecture reviews proposals for a CTO role. Once the authors have addressed or accommodated the issues from the community review tracked in the top-level summary comment, we will proceed to Oversight review with DCP PM. Guidance from the Oversight Committee has been:
In addition to finalizing the roadmap, there is a need to settle on a singular process for planning. The planning and execution process that the teams used in Q2 and Q3 demonstrated that it can produce needed velocity on end-to-end themes. We should proceed with the planning RFC as a starting point that can be evolved as improvements are identified.
If Technical Architecture transitions to a CTO, then I would expect the RFC "can be evolved as improvements are identified" which is true for any RFC. They are intended to be living documents.
justincc
commented
5 years ago @hannes-ucsc, @justincc, and @NoopDog - does 9538ae5 address your questions and concerns about pipelines.
On a read it looks like this is allowing teams to customize their workflows as long as there are the 4 pipelines
with a mapping from custom pipelines to those. If that is the case then yes, this addresses my concerns. Thank you.
justincc
commented
5 years ago As you know, I've now published an alternatives RFC with my concerns about the general approach to planning and alternatives. That's an informational RFC so I'm not going to insist that it's incorporated, even if I had that power.
With that in mind, I can't give a positive approval to this RFC but I will remove my formal request for changes since I'm not going to insist upon them, especially to the point where people are sick of dragging this process out. However, I do still request that they are given serious consideration by both the authors of this RFC and the reviewers.
justincc
commented
5 years ago Okay, I can't work out how to remove the request for changes without active approval. Please regard that as my position despite the red icon.
~August 12: Last call for community review~ ~August 19: Extended last call for community review based on #tech-architecture poll~ ~August 20: Authors are responding to remaining questions and preparing the RFC for Oversight review.~ ~September 17: Last call for oversight review~ September 19: Approved on DCP PM call
Blocked on Project Leads (please clarify during Oversight Review)
Shepherd Writeup
Many improvements were suggested by the members of the community and incorporated, including but not limited to:
The community review period was extended to three weeks to offer further opportunities for review by Technical Leads and developers.
There was a tendency to compare this process with imaginary processes described as agile. This process is also not waterfall.
There was a repeated concerns that developer-based estimation was either extremely hard or impossible (Justin). (Sometimes, people used "predictable which is not referenced in the RFC rather than "estimation"). And it may be more difficult to estimate when developers are context-switching across multiple initiatives. The authors believe that using Spikes to collaboratively design and prototype unknown unknowns will help mitigate these concerns over time. (We would also note that during the Q2 and Q3 prototyping of this RFC, there was a tendency for Technical Leads not to negotiate scope or reject Objectives due to fit.)
Objections were strongly expressed by Tony in RFC comments without proposing any concrete alternatives. Tony shared his objections with the Oversight Committee in response to their guidance - "there is a need to settle on a singular process for planning. The planning and execution process that the teams used in Q2 and Q3 demonstrated that it can produce needed velocity on end-to-end themes. We should proceed with the planning RFC as a starting point that can be evolved as improvements are identified."
Tony also commented that the historical ad-hoc process resulted in "... more than eighteen months of design and development was invested in the creation of services that were not fit for the purpose of presenting experimental metadata to users. A key success criteria must be to ensure we don't repeat this" which seemed contradictory to the authors.
Approved
Neutral
Justin originally approved but is now neutral. He has approved the requested changes for pipelines. The authors have reviewed and commented on his Alternative Planning and suggest that Oversight reviewers also review. The Theme Teams concept is complementary to this RFC.
Hannes approved the requested changes for pipelines, but remains neutral because this RFC would not pause for resolution of his Chief Architect proposal.
No objections were raised by other Technical Leads.
Requested Changes
The authors have acknowledged but are not accommodating the changes requested below.
If you’re coordinating plans across two orgs, then timing is the only universal language — and it is a language of commitment. Two teams can only work together if they share high integrity timing — two functional teams or client/back end, etc.
I'm not going to comment on specific contents of this pull request, as I think the entire document is built on flawed foundations. I have several major overall concerns.
See also:
Response from Cara
Response from Tony
Response from Brian R
Response from Tony
Response from Brian R
Response from Tony
Response from Brian R