Front-end features and coordination with back-end team (discussion)

[bengtan]

Cross-quoting @toland from https://github.com/hippware/rn-chat/issues/2383#issuecomment-391438385

This is a bit meta; apologies for the OT comment.

Product/QA creates rn-chat tickets by default. And sometimes, it's useful for them to have a rn-chat ticket (even if it's a server side change) so they remember to QA/test it when it's deployed.

This policy is problematic for the server-side team (i.e., myself and Bernard). If everything goes onto the rn-chat board by default, then it is very difficult for us to separate the things that we need to pay attention to and be involved in from the day-to-day work of the front-end team. The signal-to-noise ratio of the rn-chat board is very low for us (I am sure that the situation is different for the front-end team, of course).

It seems like what we need is a third board to track high-level feature discussions. These discussions are often not visible to us and we tend to find out about them after the decisions have been made and the front-end team has already begun work. We should be involved from the beginning. With a separate board, we could track the discussions and give input into feature development, and the rn-chat and wocky boards can be reserved for the day-to-day work of the different teams.

[bengtan]

(Meta: Not sure if a github discussion is the best place to discuss this. Maybe a group voice call is better. But anyway, I'll start it off.)

The signal-to-noise ratio of the rn-chat board is very low for us

And I would agree that we don't want @toland and @bernardd to have to monitor the front-end board.

Up till now, we've been ad-hoc with communicating features and requirements from the front-end to the back-end. I did it quite a bit for a while by writing specifications for both front-end and back-end to develop against. Other than that, sometimes we'd just mention @toland or @bernardd on github or message them on Slack (In other words, ad-hoc commenting/messaging on an as-needed basis).

I agree that we should communicate features and requirements better (and earlier) to the back-end team.

I'm just not sure that one single process change is going to be sufficient. I think it's going to be ... complicated.

At the moment, communication features and requirements from Product/Management (PM) to the front-end team is also somewhat ad-hoc.

Sometimes we have front-end 'ticket estimation meetings' where a bunch of new tickets is introduced to the front-end team.

Sometimes we have front-end tickets which are newly (quietly?) created on github, but the front-end team doesn't look at them until it's time to implement, only to discover that it's ambiguous, or a bad idea, or a back-end change is needed.

Sometimes PM creates new front-end tickets, shelves it for something more important, returns to it a month or so later, and the front-end devs have forgotten the (unwritten) details, or forgotten the clarifying questions they were gonna ask.

I think there are a lot of places where communication from PM to the front-end team can fall through the cracks. These possibly need to be fixed first.

So what am I trying to say?

It seems like what we need is a third board to track high-level feature discussions.

I'm not objecting against having a third board, but, given that we haven't been able to define a process for communicating features and requirements from PM to the front-end team AND follow it consistently, we might have the same problem with this 'third board process' ie. we could make a policy out of it, but it might not be adhered-to consistently.

Again, I'm not saying that this 'third board' is a bad idea. I just think we're going to have problems with making everyone use it on a consistent basis.

[bengtan]

I have a different proposal. This doesn't necessarily replace the 'third board' idea. Maybe it's a compliment to the 'third board'.

Maybe when @zavreb and @thescurry discuss new features (and @irfirl designs the screens), they should involve @toland in the early stages, maybe during or before the screens are designed.

I feel that these are discussions that I would be involved in, if I was in LA. But I'm not. Neither is @toland. However, at least @toland is in a similar timezone.

Just throwing it out there.

[zavreb]

We were doing this earlier in the year with Product Signoffs, we did that for Categories, Invite Friends and Geofence, however geofence was an exception due to the fact that the requirements changed during the Sprint. Perhaps @toland is referring to smaller one-offs or for items that have already been built.

Maybe when @zavreb and @thescurry discuss new features (and @irfirl designs the screens), they should involve @toland in the early stages, maybe during or before the screens are designed.

[toland]

(Meta: Not sure if a github discussion is the best place to discuss this. Maybe a group voice call is better. But anyway, I'll start it off.)

I think a call is a good medium for this type of discussion. Even better if it is a regular call (every 2-4 weeks) to review our process and work on ways to improve.

I agree that we should communicate features and requirements better (and earlier) to the back-end team.

+1

I'm just not sure that one single process change is going to be sufficient. I think it's going to be ... complicated.

I agree 100%. Process is always complicated 😄

Process issues rarely have a single cause or a simple fix. Especially when most of the team is remote.

At the moment, communication features and requirements from Product/Management (PM) to the front-end team is also somewhat ad-hoc. [...] I think there are a lot of places where communication from PM to the front-end team can fall through the cracks. These possibly need to be fixed first.

I was with you until the word "first." I think both issues are symptoms of the same problem, and both need to be addressed for us to be successful.

From where I sit, it appears that there is a problem with the way folks think about the application. I am not trying to point fingers here or anything, just relaying the situation as it appears from my perspective. To me, it looks like product thinks of the front-end as The Application with the backend being a mysterious and poorly understood yet necessary contrivance. This view is subtly reinforced by the ways in which work is defined, documented and tracked (for example, placing all tickets on the front-end board "by default").

The reality is that both the front-end and back-end are parts of the same application. It just so happens that we deploy part of our application onto users' phones and another part onto AWS. We often speak about the QA team, the product team, the front-end team, and the back-end team. But we should be one team with members who have different responsibilities. When I was a manager at Rackspace, I would often put an image of a crew (rowing) team or a Greek trireme up on the screen to illustrate the point that we all need to be pulling in the same direction to make the boat go forward. Otherwise, the boat will meander around or, in the worst case, just go around in circles or stop dead. I am not sure we are all pulling in the same direction.

I think that this would be a far less significant problem if we were in the same physical space. But, we are not, so we have to work harder on the communication issues. And I think this is largely an issue with communication. Better technology or tools may help, but won't solve the core problem. A good process exists independently of the tools used, but the right tools make the process easier to execute.

As I have said before, I have Opinions about process. I try not to be too vocal about that, but I am going to make an exception this time because I think it is relevant to the discussion. This is going to be long. My apologies in advance.

My interest in process goes all the way back to my introduction to the industry in the early to mid-nineties. That era saw the rise of Big Process (or Big Design Up Front (BDUF); think Rational Unified Process and the like) and the seeds of what would become the Agile Software Development movement (then known as "lightweight methodologies"). The roots of BDUF are a very reasonable sounding theory: we find once we get into development that there are a lot of questions and corner cases, and it is very expensive to correct those late in the development cycle, so therefore, we need to do more design and signoffs early in the in process. This does not work at all well, and in the most extreme cases leads to "analysis paralysis." (Fun fact: my first official title was "Programmer/Analyst." I was expected to do requirements analysis and programming. It is noncontroversial now, but was a big deal back in the day.)

The Agile crowd "won" that argument, eventually. The key insight was that no matter how much design and analysis you do up front, you will always run into questions, corner cases, WTF? moments, and changes in plans. So, you only do a little bit; The Simplest Thing That Could Possibly Work. Then you get feedback on that thing, address the feedback, and do the next Simplest Thing That Could Possibly Work. This has not been without its problems. For one thing, management doesn't like that you can't give a delivery date at the beginning of the project (even if everyone knows that date is based on prior expectations, wishful thinking, and huffing the dry erase markers). So we came up with relative estimation, iterations (sprints), velocity, and burndown charts. These were a reasonable solution to the problem and were quickly adopted. Unfortunately, these practices are adopted without understanding the context they were developed in, the problems that they are meant to solve, or the way that they can be used to solve said problems. They have become dogma over the last 15 years.

If any single methodology has become the generic version of Agile, it is Scrum. Nowadays, when someone says "Agile" what they usually mean is "Scrum." Extreme Programming (XP) was the first Agile methodology to get widespread adoption, but it was almost entirely focused on engineering practices. Ken Schwaber and Jeff Sutherland took the management related aspects of XP and codified them as Scrum. Both XP and Scrum were designed with particular teams and types of teams in mind. Most teams today do not look like the large, corporate teams that Beck and Schwaber were speaking to. In the meantime, things like sprints, estimates, and velocity have become something that teams do without really thinking about why. And the most important part of either methodology, continuous process improvement, has fallen by the wayside.

Unless you have a very specific kind of team, structured in a very specific way, and with certain very specific goals; then neither Scrum nor XP will be right for you "out of the box." Same goes for Crystal Clear, SAFe, AUP, or whatever other pre-packaged process framework is the flavor of the month. The right process for your team will be unique to that team. It will need to change and evolve as the team changes and evolves. Most importantly, it must be intentional. You can't throw a bunch of smart people into a (chat) room and expect a coherent process to emerge. Someone needs to own the process and drive it forward. It needs to be a priority for the business. And you need to consciously make regular efforts to improve.

I know all of this because I have been in shops where the process was ad-hoc to non-existent. Those shops did not remain fun places to work for long. I have also been in shops that ran like a well-oiled machine. Mostly, I know this because I was given some "problem" teams with "problem" developers working on "problem" applications at Rackspace and told to make something of it. Eventually, that team grew, took on several important projects, was very well regarded within the company, and had the highest engagement scores of any development team in my department. Right before I left, there were just over 30 developers on the team, with 1/3 of them working in India and the rest working in the Rackspace offices in Austin and San Antonio.

I had trouble getting things off the ground. The process wasn't working; the team wasn't "clicking." I brought in a very talented Scrum Master, Nancy, to help sort this out. Eventually, I realized we needed to jettison all of the Agile Orthodoxy and think about what worked for us. Nancy was reluctant, but she trusts me and was willing to give it a try. After trying several things, we ended up with a very Kanban-ish process. We stopped estimating since pretty much everyone agreed that the estimates and velocity were useless. Instead, we used lead time, cycle time, WIP, throughput, and cumulative flow. Note that these are all empirical metrics and hard to game (unlike velocity). The most important was WIP and cycle time. My management loved me because I could say things like: "My team is at 65% capacity right now because product can't make up their minds about which big feature comes next. Are there any small PoCs or QoL features that we can add in? If not, I am going to have Nancy bring up some tech debt and the team can do some other polish work." Nobody else in my department could speak about capacity like this.

We also got rid of sprints. Sprints (or iterations, as the XP guys called them) are meant to be used as deadlines to keep the team focused. Originally in Scrum, they were intended to be 4-6 weeks long so that you could develop a demonstrable "increment" of the final system. XP used shorter iterations, usually 2-3 weeks, since 6 weeks is more than enough time for a team to get off into the weeds. Most teams adopt the shorter sprints, but that generally isn't enough time to do much that is useful. I did away with the sprints altogether and focused on flow. As they say, you don't run a marathon as a series of sprints. You want to adopt a pace that makes progress, but that you can also keep up indefinitely.

We still had regular meetings to groom the backlog and review the process. And we delivered running, tested code on a regular cadence. These two things are the primary benefits of sprints, but you don't actually need to "do" sprints to get those benefits.

So, after all of that, what would I recommend in this situation. Well, I am glad you asked (even though no one really did). There are a few key things that every process should adhere to:

1. Everyone must know what the process is and why those practices are being used.
2. The process must be clearly documented somewhere (see point 1).
3. Each practice must have a purpose (we don't do anything just because it is in a book).
4. The process must be measured (how do you know if it is working?).
5. The process must be regularly reviewed and improved.
6. The process must be as simple as possible, but no simpler.

Therefore, my recommendations are:

• Someone would own the process and hold regular process review meetings to discuss how things are going. The core practices that make up the process would be documented on the wiki.
• We would have one board[1]. We are just too small for more than that. Something more sophisticated than ZenHub[2] so that we can set WIP limits, add swimlanes, and track important metrics (like cycle time)[3].
• Product would start out with a "big idea" ticket, we could go through rounds of discussion and high-level design, then break out front and back end work tickets to do the actual implementation. The developers should be involved in that discussion, and should probably write the implementation tickets.
• Each feature would have a business owner. If there was confusion, or a corner case, the developer could get clarification relatively quickly from the business owner and keep going.
• Deployments would happen on a regular cadence (and preferably would be 100% automated).
• For the most part, product, front-end and back-end work would be more or less synchronized.

This would require more regular, and more consistent, communication between the LA office and the folks that are working remotely. We are very siloed, and that will have to change. It would also require folks to take more ownership.

This post has gone on for long enough, so I am going to stop now. I think I have said my piece.

[1]: I mentioned adding a third board above because that would be the path of least resistance. I don't think that it is the best move from a big-picture perspective. [2]: Don't get me wrong. I love Github, and Zenhub is a nice extension. But it just doesn't have several features that are dealbreakers for me. [3]: I really like TargetProcess for this.

[toland]

Maybe when @zavreb and @thescurry discuss new features (and @irfirl designs the screens), they should involve @toland in the early stages, maybe during or before the screens are designed.

I think this is a great idea and would be more than happy to participate in those discussions.

[bengtan]

@toland:

One quick response now, and a more considered answer later because I have to read all that stuff you wrote. :)

I was with you until the word "first."

When I wrote my comment above (which you replied to), I had to rewrite/heavily-edit it a few times. I had a chain of logic/reasoning in my head but it just wouldn't come out right in the writing. Maybe my thinking wasn't clear enough and what I wrote was only expressing myself about 90% instead of 100%.

Now that you point it out ... yeah, maybe that word 'first' doesn't belong in there.

(And possibly if I had correctly omitted it initially, I could have saved you a lot of typing :) )

[bengtan]

@toland, I just read your long post.

I think it's a great post and thanks for writing it. It's going to give us a lot of food for thought.

I don't think it's going to result in any changes immediately, but it does start the conversation off, and maybe (hopefully?) we'll make changes somewhere down the line.

But, yeah, definitely a discussion worth continuing.

[bengtan]

Purging due to redesign/board-restructure/out-of-date/etc.