pages/7-20-24-programmers-deserve-consequences

[utterances-bot]

After CrowdStrike, Programmers Deserve Consequences.

https://mccue.dev/pages/7-20-24-programmers-deserve-consequences

[jkesle]

lol big dawg, lets see you start writing code. no? don't have the chops? then zip that loud mouth and get back to this teeny bopper blogging you do

[bbottema]

Completely out of touch with reality. This is not how software engineering works. Also, the comparison to medical specialists is bonkers. This can only be rage bait.

[ask5]

Product Manager should be held accountable not engineers because they sign off on things.

[bowbahdoe]

@jkesle - uncalled for.

@andymartin Yeah, job security without wage manipulation or mass layoffs might be required to enforce any codes of practice...in practice. This isn't rage bait. I'm not going to delete it. I largely agree with most of the people shouting at me. Turns out things I write in a coffee shop on a Saturday aren't always rhetorically airtight. I'll own that.

@bbottema The comparison is not bonkers. We often do stuff that affects the lives of people directly but the entire field treats outages like natural disasters. This outage seems to have been caused by obvious negligence but we have no basis upon which to penalize that at any level aside from "they might lose customers."

@ask5 Yeah and thats kinda the issue, no?

[oliness]

Due to the Halting Problem, we can't prove that a program won't cause problems (this article points out that if you could you'd basically be omniscient - you'd be able to prove or disprove any mathematical conjecture). The best we can do is improve the intelligence of static analysis and increase the volume of testing. But even then, we can't find everything within a reasonable amount of time on large projects - you might have an invalid memory access that only occurs on very specific input that the fuzzer won't find unless it's run for millions of years.

In this case, it seems the bug was already there, just only triggered with a particular definition file. That definition file should have been tested better, given that it crashed everyone at once. I agree that we could mandate some minimum of testing, and harden the OS for kernel drivers - perhaps Windows can detect if it restarted due to a BSOD caused by a driver, and refuse to load that driver. Perfection is impossible however.

[jckmrrssy]

maybe you should run your musings through AI to check for “tone deafness” before releasing :)

Thank you for your edits - they clarify your position much more, though I’m still not sure I agree in full with your reasoning.

[red-g]

@oliness brings up some good points, but I would like to add that there are still software tools that can prove correctness (for instance SPARK or Agda). This would make development—at least up front—much more difficult and mathematical, but in a mission critical application that could be well worth the tradeoff.

Some kind of standard around provable correctness in those settings would probably beneficial. If reckless management or programmers started producing lower quality software, this could now be objectively observed, and thus held accountable for.

As for the halting problem: Correctness includes proving whether or not a procedure terminates. In formal verification, If it is difficult to prove whether or not a procedure terminates (or we know that it can diverge), such a procedure would be marked as possibly diverging. However, in practice we can almost always prove termination.

note: The CrowdStrike crash occurred in kernel code. Provable correctness is more tricky when it comes to lower level operations, as our computers are non-deterministic. But people much smarter than me have already tackled this problem.

[Jeremiah-Griffin]

The difference between medicine and software development is the vast majority of medical work can cause significant damage to a patient if done improperly. By contrast the majority of software development likely cannot. GCs, virtual memory, process isolation, browsers, etc all protect users from the risk of damage by incompetent application developers.

The remaining work the typical developer can screw up sufficiently to injure people is primarily in the processing of payments and secrets: both of which the market strongly incentivizes - and often implicitly mandates - the use of third party, verified, and standards compliant solutions.

I would say that for the majority of use cases the industry has substantially engineered its way out of situations where developer incompetence alone can wreak this sort of havok. To that end, it is unsurprising that it was a development environment where many of these advancements are unavailable or impossible to implement that developer incompetence did cause a catastrophe. Let's not lose sight of that.

Licensure is expensive to administer as program, expensive to apply, and expensive to maintain by the individual. That expense reduces the supply of software developers for innumerable reasons. Do we really want to cull potential millions of developers globally that will never meet a standard of licensure required for work they will never do?

This field is renowned for its accessibility to those without a formal education. Let's not dispense with that accessibility for all fields of development because one bad actor in one field did something wrong that for the vast majority of developers is technologically impossible to do.

The real solution is two fold.

First, as always, better tooling helps. Safer languages like zig and rust are increasingly being used for systems development. Static analysis tools for unsafe languages are improving every year. The capabilities based security models of next-gen embedded OSs and platforms like Fuschia and WASI go a long, long way as well.

The second element demands something of a rejection of one of your points:

Attempts to impose standards and restrictions largely fail.

This is a bit hard to quantify. In part because the statement is somewhat vague, in part because the word selection is non-committal. How often is "largely"? To what attempts are you referring? Are these standards and restrictions published specifications or the listless and elusive platitudes peddled in books the titles of which I think all of us can imagine.

Ultimately safety and security critical fields objectively do have have standards for testing and design of software: DO-178, DO-278, ISO - 15408, PCI DSS, and many, many more. Likewise, the adherance or not of these practices is frequently cited in civil and criminal cases in the defense or prosecution of incidents respectively.

The failing at the root of the issue here is not that a bunch of software developers in non-sensitive fields are unlicensed, it's that a company and its developers operating in an affirmatively sensitive field almost certainly didn't follow the extant standards and specifications they are expected to.

Licensure does not and is not intended to protect anyone from the acts of licensed persons who refuse to follow the standards and specifications of the licensing body.

In this alternate universe of software engineering licensure, instead of farming developers, managers, and executives who (presumably!!!) evaded industry standards and good practice for the creation of safe and secure software, crowdstrike would have farmed developers, managers, and executives who evaded the standards of a licensing body for the same. :shrug:

[Yongbeom-Kim]

I think that on top of everything everyone's said so far, the post misses out on the open-source aspect of software engineering. Anyone can become a software engineer, and everyone stands on the shoulders of giants.

You might write a database as a personal project, which someone at HashiCorp decides to use for their key services, which then Roblox decides to use to deploy their game with. Oops, there was an undiscovered issue with your project, and now Roblox is down for 72 hours.

When we write software, we are writing it on the shoulders of other 'giants', but I guarantee that a crap ton of these projects are written and updated by unpaid giants wiriting code in their free time. Seen this xkcd comic?

I understand what you're trying to go for here, but I think it is more likely that this kind of 'consequence' narrative willl just quash organisations the likes of Red Hat, Apache, etc., who all do so much for the CS/SWE community as a whole. Cloudstrike blames the outage on their testing software - which may or may not be actually true - but issues from software dependencies arise all the time, and can you really expect an organisation writing thousands of lines of code to read and review millions? And don't even get me started on updating these dependencies.

Why should there be uniform standards for software engineering? It is a field - unlike most others - that intersects with literally every other field, and each part of SWE that interacts with field X follows the relevant standards regarding field X. I cannot even begin to imagine how such a standard would be formed.

What exactly is wrong with the way the clooudstrike situation is being dealt with? It isn't as if cloudstrike isn't facing the legal repercussions that they should. The devs, team and management will probably face repercussions and cuts, even if simply from the financial cost of the ordeal alone.

To have the sort of licensing scheme you suggest would require a sort of upending of the entire way we write code, get developers, and do anything at all. The video you linked talks about things that would be amazing for C++ - but had it been on a conference about, say, web development, its contents would have been completely difference. If it cost an arm and a leg to become a software engineer in the first place, I would have never had this opportunity, so I cannot be on the same page as you. Today, organisations can still pick up certfications, and to maintiain them, are required to implement processes and train the relevant employees for these certifications - isn't this enough as it is?

[sophondetector]

""programmers deserve to live in the world where their actions are given weight and recognized as an engineering discipline with consequences for negligence all the way up the chain."

I'm sorry that is such a tepid, tedious, milquetoast take. The culture of software engineering is amoral, apathetic, mired in degenerate philosophy, and completely solipsistic.

[alexzk1]

Programmers are the hired workers. Point. If you insist on personal responsibility of them, then company should give them a cut of the profits and ask explicit if they want such responsibility. Then such a person becomes co-owner by definition.

[bowbahdoe]

@sophondetector I know I'm going to regret this, but what precisely do you mean by "degenerate philosophy?"

[origineering]

the programming profession clearly has a systemic problem; it is too much of an unregulated wild west with way too many gun slinging yahoos in it, but on the other side, we don't want the suits and lawyers coming into it and slowing the industry to a crawl — we want it civilized, accountable, without being over-regulated.

but, golly, the long chain of cluster-Fs, each of them so easily preventable that led to the needless tragedy of the CrowdStrike outage, where the cure was so much worse than the disease.

to see what i mean, here is a Y/T that gives some insider perspective to this: https://www.youtube.com/watch?v=7rx4U5TlaqE

i wish this was an isolated incident, but this is actually SNAFU at pretty well every company i have worked for that is in the IT sector. The greater the wealth and impact of the company, the more SNAFU is its normal everyday operations, and Agile only made code/product quality worse and more politicized.

sigh.

[origineering]

@sophondetector you wrote: re: ❝ “programmers deserve to live in the world where their actions are given weight and recognized as an engineering discipline with consequences for negligence all the way up the chain.”

I'm sorry that is such a tepid, tedious, milquetoast take. The culture of software engineering is amoral, apathetic, mired in degenerate philosophy, and completely solipsistic. ❞

kudos! so well expressed.

the statement: “a cynic is a disappointed optimist” (by George Carlin) seems to apply to both of us.