search

betterscientificsoftware / bssw.io

Better Scientific Software Homepage
https://bssw.io
Other
140 stars 90 forks source link

The myth of developer productivity #1341

Open markcmiller86 opened 2 years ago

markcmiller86 commented 2 years ago

I really enjoyed this article, https://nortal.com/us/blog/the-myth-of-developer-productivity/

I am wondering what others get out of it.

I think it could be a good CC

bartlettroscoe commented 1 year ago

I scanned this blog article and it looks like it might be worth a CC article for some audience. If I still feel that way after reading over this article, I will post a CC article on this.

bartlettroscoe commented 1 year ago

I read the entire article:

and I think it is a good read and likely very beneficial to many people.

But, as always, I am tempted to write an original article to pull together several different related articles on this topic including:

and his follow-up 10 years later:

And related to this is the topic of Kanban and the resources:

What it just re-enforces for me that the core foundations of software engineering are not changing very fast (or much at all) over the last 20 years. I just keep reading the same views and topics over and over again (being dressed up as the state of the art). Nothing in this article is unique or new. It is just a different author spewing the same material (and not referring prior work as they should). But it is a good retelling and packaging of this material so I endorse it.

Now I need to decide if I am going to write a simplistic one-paragraph CC article or write a longer (and more interesting) review article that ties all of these resources together.

What should I do?

markcmiller86 commented 1 year ago

@bartlettroscoe I'd be up for collaborating on someting here if you are interested. I've been thinking a lot lately about productivity and measurement of it.

I also wonder if CS pundits have looked to all the right places for productivity measures. For example, maybe programming is more like being an editor or copy writer for a news paper or magazine or blog/web site. How is productivity measured in that industry? Does it have some of the same issues measuring productivity that we have? What about book authors, music composers, etc.?

I don't like the idea that we (CS experts) keep telling the world...you can't measure our productivity. Its too complicated...too many different activities are in play, etc. Trus us, we know what we're doing.

bartlettroscoe commented 1 year ago

@bartlettroscoe I'd be up for collaborating on [something] here if you are interested.

@markcmiller86, sure. I am up for a co-authored original article on this subject.

I also wonder if CS pundits have looked to all the right places for productivity measures. For example, maybe programming is more like being an editor or copy writer for a news paper or magazine or blog/web site. How is productivity measured in that industry? Does it have some of the same issues measuring productivity that we have? What about book authors, music composers, etc.?

I am not sure a magazine or newspaper writer is a good analogy because if someone says "give me a story about X tomorrow by 12 noon" even a decent writer can do that, every time, because the scope is very ill-defined. The produced article does not have to "run" and actually it can even be highly (subjectively) illogical and still be published. Those products are completely subjective. There is nothing subjective about a piece of software. It is as concrete a thing as anything created in the real world by traditional engineers.

I don't like the idea that we (CS experts) keep telling the world...you can't measure our productivity. Its too complicated...too many different activities are in play, etc. Trus us, we know what we're doing.

Actually, the author does argue that you make measure things and act on those measurements in a ways that are widely considered to improve productivity. Those quantities of interest are "lead time" and "cycle time" and there are many knobs that can be turned to influence those (which is Lean theory and Kanban obviously). The argument is that we know that many things hurt productivity so let's focus on removing those impediments and then trust developers to do the right thing (with a good iterative process).

At the core of this whole discussion is the foundations for Lean and Agile methodologies. These methodologies say "train people for what they need to know, define the initial scope, and then work with them (iteratively) to develop and refine the product with constant feedback to produce the best thing they can with the allotted budget and time constraints."

Other than deciding on promotions and performance compensation, what is the reason/goal for trying to measure individual developer productivity? What organizations should really care about is sustainable problem solving (of which much but not all will be writing software) by the teams they employ. And if we measure and track "waste" and try to reduce it, we have gone a long way to improving productivity.

bartlettroscoe commented 1 year ago

I would highly recommend the book Poppendieck, Mary and Tom. Implementing Lean Software Development. Addison Wesley, 2007. All of these little blog articles are just the tip of the iceberg.

markcmiller86 commented 1 year ago

Other than on deciding on promotions and performance compensation, what is the reason/goal for trying to measure individual developer productivity?

FWIW, I think promotion and compensation rationale is significantly secondary to an organization's ability to propose and plan and hit budget and delivery deadline targets. I have the belief that the better an organization understands software productivity in a quantitative (and qualitivative) way, the better that organization can compete in attracting funding. At least I think that is true in a enviornment that relies primarily on market to make its funding decisions 😉.

bartlettroscoe commented 1 year ago

FWIW, I think promotion and compensation rationale is significantly secondary to an organization's ability to propose and plan and hit budget and delivery deadline targets.

The issue of software estimation is a bit different than individual developer productivity. Developer productivity is just one part. The other major parts are scope and available time (some aspects of quality should not be considered to be variable). What has been made clear in the last 50 years of software development is that is folly to try to violate the iron triangle of software development. The problem is that really don't know the scope of a hard problem until you are right in the middle of the work (known as a "Wicked Problem" in Code Complete 2nd edition). I think many projects in CSE are wicked problems. Therefore, the scope must always be fungible if there are time constraints.

That brings us to the Lean realization is that detailed software estimation is a form of waste in a Lean/Agile projects.

markcmiller86 commented 1 year ago

I know I haven't looked at this in as much detail as I'd like but its my impression that of all human "construction projects", software development remains the most elusive in accurately estimating. Now, the word "software" applies to so many different kinds of projects, maybe that is the fundamental issue. Software needed to control the space shuttle is very different business than word processing software and so including both projects in any discussion about software estimation isn't necessarily appropriate. On the other hand, what about sky scraper projects throughout history and the world? Has humankind done a better job at estimating their costs and timelines? I think so. I don't know so. I just think so. And, I think we've done tons better with sky scraper estimation than with software estimation.

And, I also have very little faith in my ability to estimate programming tasks that I am assigned. One issue that just occurred to me on that though is the amount of juggling and context switching my life as a software engineer as been. Its rare I get a full day to focus on a programming task...even a half day. And, estimating effort when so much of my life is interrupt driven is counterproductive. I wonder if many others who are responsible for HPC coding tasks feel similarly?

bartlettroscoe commented 1 year ago

And, estimating effort when so much of my life is interrupt driven is counterproductive.

Task switching is a recognized huge cause of waste in software development. Both Kanban and Scrum are designed to reduce thrashing and task switching and improve focus.

bartlettroscoe commented 1 year ago

@markcmiller86, are you still interested to collaborate with me on a short review article on this topic given these references listed above? If so, I will write an initial draft that you can poke holes in?

bartlettroscoe commented 1 year ago

I will start this and push to the new branch https://github.com/betterscientificsoftware/bssw.io/tree/1341-dev-productivity.

markcmiller86 commented 1 year ago

Sure thing. I'll take a quick look at the content on the new branch

bartlettroscoe commented 1 year ago

Sure thing. I'll take a quick look at the content on the new branch

There is nothing on the branch yet. I just created it from 'master'.

rinkug commented 4 months ago

The issue got closed since I liked a wrong PR to it, So I have reopened it.