Add post about common trade-offs

[jryans]

On the FoC Slack, @akkartik mentions:

In general, the thing I find missing in the https://malleable.systems mission (and manifestos in general) is any mention of trade-offs. What are we willing to give up to achieve the stated goals? In this case study, malleability is only possible when we control the size and complexity of our software stacks. That might be a common compromise.

I think it would be good to review what common trade-offs across various approaches in a post on the blog.

[jryans]

@khinsen says:

Yes, trade-offs are an important point. In my guest blog post (https://malleable.systems/blog/2020/04/01/the-most-successful-malleable-system-in-history/) I point out one of them, which is responsibility for the overall system. If you let users mess around with it, you can't then let them sue the original authors if it breaks. And if you let people distribute their own add-ons, that makes three parties among which responsibility must be divided in some way.

Size and complexity are also an issue, but less so if you get away from the assumption of a software stack implying vertical dependencies. Replace this by a network of modular subsystems with reasonably stable APIs, and you can have malleability for the whole system and for each subsystem. Do it recursively and you can perhaps scale to really big and complex stuff (though I am obviously speculating here).

Sure, we have no technology for building systems out of modular subsystems. That remains to be done.

[themightychris]

Sure, we have no technology for building systems out of modular subsystems. That remains to be done.

I've been working on this problem for many years :-) Excited to connect!

My team has built and maintained a number of complex malleability-first systems for a while now, and are embarking on a next-generation rebuild of all the tooling to take advantage of the modern cloud-native ecosystem

I've become convinced that most of the tradeoffs are manageable, the challenges largely come from trying to shoehorn hyperscale-enterprise tooling and practices into ground-scale malleable systems. The design goals are just orthogonal, they are not driven by first principles

"Network of modular subsystems", if you're talking more about microservice architecture than just encapsulating modules, I've found to be a bit of an anti-pattern when it comes to user-malleable systems. They create a sort of reverse Conway's law, where you need to build an organization of critical mass to be able to manage the mass of independent subsystems.

Users need systems that are flat, or at least look flat to them

[khinsen]

@themightychris Good to know that someone has actually been thinking about this!

How exactly a network of modular subsystems is implemented depends very much on the application profile it is designed for. Encapsulated modules sounds fine to me. And you are completely right that Conway's law applies both ways.

Systems being flat at any given level is exactly my point. Most users would care mainly about one level, so it's all flat for them. But one user may want to hack the networking subsystem, whereas another one would be happy to leave that as a black box and hack on a higher level.

[themightychris]

Most users would care mainly about one level, so it's all flat for them. But one user may want to hack the networking subsystem, whereas another one would be happy to leave that as a black box and hack on a higher level.

Yep, that's a great way to put it. I sometimes talk about this as "putting the complexity/plumbing under the floor". Users should have an easily understood floor they can stand on without needing to know how all the facilities work before they start. But, the complexity should all be easy to access and service

Current "best" practices lead to a situation where a user wanting to run a reasonably complex off-the-shelf web application, change some CSS to theme it, and then see the result... requires learning 12 layers of technologies that aren't CSS before they can do that. The CSS source is buried all over a deep file tree, there's a tool and process to build the css, a tool and process to integrate the result into the application, and tool and process to update the build for the application, and tool an process to run the build, and tool and process to push the new build into an environment...etc

It needs to be 1) User hits run and then can use the application 2) User finds CSS source within a flat, logical source tree 3) User makes a change 4) User sees the result

A lot of the aforementioned complexity gets layered in to make things more consistent, reproducible, automated...but when it's not done "under the floor" the trade-off is a mountain of prereqs for what should be an easy task for a novice hacker

[akkartik]

Here's an evil twin's take on the Malleable Systems principles.

1. Software must be as easy to change as it is to use it.

I support making software harder to use if it will get us to this goal faster. Everybody eat your vegetables.

2. All layers, from the user interface through functionality to the data within, must support arbitrary recombination and reuse in new environments.

The easiest way to support this is by letting users edit source code. Recombination and reuse don't have to happen from behind some pane of glass. Allow your users to be like John McClane from the "Die Hard" movies, punching through walls and jumping through elevator shafts. Focus on giving them maps and safety nets.

3. Tools should strive to be easy to begin working with but still have lots of open-ended potential.

The most natural place to begin working with a tool is using it. Treat your existing users as potential hackers at the start of a funnel with a steep drop-off. Some of them will recombine what you give them, but you don't know who. How do you maintain motivation to continue supporting them under these conditions?

4. Retain ownership and control.

Existing extension systems like browsers and Emacs already do this? By the time somebody is extending your system they tend to know enough to exercise ownership. Maybe we just need to grow this set of people?

5. Freely shareable.

Again, it's unclear what systems are in the author's mind that are extensible but not shareable.

6. Modifying in the context of use.

This seems really hard. Should it be a high priority? I recommend going from (disempowered, beautiful) to (empowered, ugly) first, trading off one attribute (candy) for another (eat your vegetables) to strengthen fundamentals. Once you lay the ground work, then you can try to perform the energetic heavy lift to (empowered, beautiful).

7. Thoughtfully crafted.

This. This is the meat. Everything else feels like fluff, either really obvious or pie-in-the-sky hard. Don't make me wade through it to get here. That'll also give you more space to elaborate. How do we 'refine' our tools? Is this a matter of UX? Superficial reworking? Deeper surgery? Throwing everything out and starting over? A new methodology? A new social structure?

Overall, I think my disagreements mostly stem from an overwhelming sense of urgency. You're over-constraining your solution space by shooting simultaneously for useful social properties (empowerment) and useful individual aesthetic properties (Bret Victor-esque WYSIWYG context of use). We're far enough behind here that maybe we should aim lower. It doesn't convey the appropriate urgency to say that "software is trapped in silos". No, the problem is that we find ourselves in a world where everybody uses software, and software can do anything, and nobody knows what their software is doing. Instructions from too many masters are fractally intertwined. This can't end well. We need to get off this train as quickly as possible.

Signed, Kartik Agaram

[khinsen]

@themightychris Looks like we agree on what we want. There remains the minor technical detail of making it happen ;-)

The big problem with moving toward malleable systems is that malleability is not a priority in any mainstream technology. That leaves a couple of options:

1. Ignore mainstream technology (the Smalltalk way)
2. Put a layer on top of mainstream technology
3. Try to influence/nudge mainstream technology towards our goals

2. has a high risk of creating a mess, whereas 1. and 2. look like enormous efforts.

[themightychris]

To a lot of folks coming from spaces where malleability is a bad thing, malleable systems look like bad engineering.

I don't think we need to invent any major new tech, but we need to arrange and configure it differently, and it's going to look like bad engineering to a lot of people

I think WordPress is a really helpful case study here. It's not hard to find good engineers who will put down WordPress as bad and sloppy and aweful, yet nothing holds a candle to how much creativity and independence WordPress has unleashed the world over. That's not a coincidence. Malleable systems allow people to make messes. A system you can't make a mess with isn't malleable enough

[khinsen]

Good point. In a Twitter comment on my blog post, Karl Fogel pointed out this book chapter by Jim Blandy, whose very title makes a similar point.

And of course any personalized environment is a mess, seen from someone else's perspective. I'd post a picture of my desk as a proof if I weren't locked down at home ;-)

[jryans]

Thanks to everyone who's commented here so far, it's great to see different perspectives and thoughts on extra context that would be good to add. 😄

I'm guessing we might end up with several posts to add from what's being discussed here, as there are several overlapping themes, and it would help to convey things from many angles such as "trade-offs", "open problems to solve", etc.