capr / blag

:suspect: Blog/wiki of a grumpy old programmer
4 stars 0 forks source link

Elephants in the open source room #17

Open capr opened 3 years ago

capr commented 3 years ago

One pet peeve I have with open source is how often I find myself trying out a library or a piece of technology and in a matter of hours or days of using it I hit a major limitation that is not advertised anywhere on their project page/docs/FAQ, nowhere in sight, but it's the first thing you should know about it because it changes everything. It's gotten so bad lately that the first thing I do nowadays is check the project's mailing list and issue tracker even before trying out the thing, just to spare myself a probable disappointment.

I'll keep here my list of major facepalms:

Binaries on Linux

On Linux, if you build a C program on Ubuntu 20, the resulted binary will most likely not run on Ubuntu 18 if your program is anything more complex than "hello world". The reason for that is a feature of glibc called versioned symbls. This feature allows your distro to update the glibc underneath your programs without you having to recompile the programs. The filpside to that is that your programs are now tied to the version of glibc that they were built against and there's no way in gcc to ask for older symbol versions when building, IOW there's no way to create backwards compatible binaries on Linux at least not with gcc, so your only option is to build them on the oldest distro that you can support. The problem there is that the compiler on that distro will also be old, so your binary will be less optimized and possibly even buggy, that is, if your program even compiles at all. Nobody talks about this and yet people are confused as to why Linux has no market share on the destkop.

MySQL

Here's some stuff that will take you days to figure out but it should really be 5 minutes and you really wanna know this. MySQL has 3 forks: Oracle, MariaDB and Percona. Oracle gives you the database for free but backup costs money. It's like advertising a free chair but only 3 legs are free, the 4th one costs money. If you want free backup, use Percona's xtrabackup but make sure you use a version that is compatible with your version of MySQL. The real facepalm moment comes when you realize that with any of the current backup solutions, free and commercial, you can't have both partial backups and incremental backups at the same time. So if you want to host multiple apps on the same server each with its own database, you can't have separate incremental backups for each app. Running multiple MySQL instances is not a viable option because you will waste most of your RAM this way with many InnoDB pools instead of a single large one. So you either have to give up incremental backups and kill your RPO or backup all databases of all apps together and then when you need to restore a sigle database or table, restore all databases together on a different server and extract what you need from that.

CockroachDB

CoackroachDB uses the exact same approach with backups. It gives you full backups for free but incremental backups costs money.

Tarantool

Cannot add a column to an existing table unless that table is empty. I dare you to find that crucial piece of information on their website.

Most JS libraries

I always had a feeling that web developers (backend and frontend alike) are full of shit but boy do JS libraries take the cake.

Popper.js is a library to create popups in JS. It has 17k starts on github, it has its own cute website, the code is thousands of LOC and the author calls it an "engine", but can't actually do popups. I swear to God web developers today are just... anyway, see the problem is that the web's document model doesn't have a global z-index, so the only way to make a pop-up div to actually stay on top of everything else is to append it to the body element, and this library just doesn't do that. Can't fail harder than that.

Velocity.js is a library to do animations in JS. The author writes about how a successful open source project is 10% technology and 90% marketing and I think he's onto something. This library has tens of files tucked snuggly into a robust folder hierarchy, but despite looking at the code and reading the documentation for hours, I still have no idea why would I want to use this instead of CSS transitions or 4th grade math inside a requestAnimationFrame() call.

Markdown parsing libraries: markdown-it needs 8 KLOC to parse a stupid text format (my whole data-driven widget toolkit including the editable grid is currently at 20KLOC). Another library called micromark is a depressing dive into the mind of the typical web developer who discovered state machine parsing (plus you need npm and a bundling system just to get a js file out). Others are even worse.

Coroutine-based web servers

One facepalm that I remember from years ago was with Lua and coroutines, when I discovered that their coroutine-based web server called Xavante doesn't have a way to deal with blocking APIs, which makes it useless for just about anything interesting that you could do with a web server. Granted, there's no easy solution to that, you either re-implement every network-based API in Lua to work on coroutine-scheduled sockets (the way OpenResty does it with their DNS resolver, MySQL driver, etc.), or you make a hi-level threads+queues API for those blocking APIs to ride on. In fact, you have to do both because there's a lot of blocking APIs out there and you're gonna need them. Either way, you'd think this would be something interesting to talk about on their homepage, given how this changes everything, especially for someone new to this way of doing things.

C APIs that are not push-style and thus incompatible with coroutines

librsync (patch is callback-based) libmysql (async API is useless) libpng (can use the progressive API for loading but the output is callback-based) expat (callback-based)

...and loads more I can't remember...

LLVM and the C ABI

The C ABI is not in LLVM but in Clang, so if you want your new shiny language to call C (and you do) or pass structs by value, you gotta copy-paste that code (or target the new MLIR which promises to solve that but I haven't checked). I've discovered this because I couldn't understand why the Terra language only targets x64 (which wasn't advertised on their homepage either) because I totally expected the ABI details to be taken care of by LLVM. This is a problem for all LLVM-based languages, in case you're wondering why your favorite language only supports a few platforms.

LLVM IR semantics

The LLVM IR has no formal semantics which means that your language that targets LLVM (for realz, undef'n'all) has no formal semantics either. Csmith, a program that generates C99-conforming programs found 200 bugs in LLVM. There's KLLVM and Vellvm (for which Csmith found no bugs!) so in theory you could just use those to figure out some dark corners of the IR, but I haven't done that yet.

X11 Protocol

Check this out: https://stackoverflow.com/questions/32247059/x11-how-to-make-a-window-move-after-another (it was I who asked the question but I deleted my SO account in rage a long time ago). Just look at the comments and the answer and you can start facepalming yourself right away. Anyway the problem here is that the X11 protocol is asynchronous, but in order to do stuff like have a window follow another while that window is dragged, you have to mix queries with commands which leads to a TOCTOU real quck. In vain I tried to convince the dude on SO and some other dudes on the xorg mailing list that making your window manipulation API async is a dumb idea, but apparently they think it's one of the highlights of the technology. Also read this mailing list thread if you're bored.