Rewrite in C

[haampie]

Given complaints like #40, conflicting requests about always using system versions of dependencies, as well as people using libtree on old systems without std::filesystem #36, I think it doesn't hurt to rewrite libtree in C without any dependencies. That means anyone can pick their favorite build system, such as make or even gcc *.c.

I had already started doing so: https://github.com/haampie/libtree-in-c and hopefully it can be done over the Christmas holidays.

The C version also includes some more fancy reports when libraries are missing, as well as more feedback about rpaths (in what parent's rpath was the library found?)

[fzakaria]

If you'd like to keep it in C++ I can help create a Nix package for you: https://github.com/NixOS/nixpkgs This mechanism would allow any system that supports Nix (Linux, BSD & Darwin) to easily install your tool.

example:

❯ nix-shell -p hello
these paths will be fetched (0.05 MiB download, 0.24 MiB unpacked):
  /nix/store/xcmlbsqabmckx42p8w18ri5zq8v2iiav-stdenv-linux
  /nix/store/xcp9cav49dmsjbwdjlmkjxj10gkpx553-hello-2.10
copying path '/nix/store/xcp9cav49dmsjbwdjlmkjxj10gkpx553-hello-2.10' from 'https://cache.nixos.org'...
copying path '/nix/store/xcmlbsqabmckx42p8w18ri5zq8v2iiav-stdenv-linux' from 'https://cache.nixos.org'...

[nix-shell:~/code/ucsc/debian_histogram]$ hello
Hello, world!

[nix-shell:~/code/ucsc/debian_histogram]$ which hello
/nix/store/xcp9cav49dmsjbwdjlmkjxj10gkpx553-hello-2.10/bin/hello

[vsoch]

@haampie would this be possible to write in Go? Also what about having Python wrappers given C or C++? Could that be fairly easy to add?

[haampie]

@fzakaria that sounds great! If you want to package it, that'd be appreciated. But I also don't mind if you wait for the next release, as it changes the build system and language.

@vsoch It should be straightforward to write in Go since Go has an ELF parser in its standard library. As long as libtree itself can be statically compiled that's great, so that LD_LIBRARY_PATH does not influence libtree's own dependencies, and Go hits this checkbox. What I care most about though is ease of install, and I think a C compiler is as minimal as it gets in terms of requirements. Another point is speed; I used ELFIO in C++, and it was slow, because of way too many mmap's. I didn't check what Go does. But basically libtree only needs to read a few bytes from an ELF file, and then fopen/fseek/fread is much faster (which you can also do in Go, of course). Lastly I like that I can build tiny executables in C with musl libc (currently the static binary is ~75KB).

Others suggested Rust, but it has the same issues, and honestly I don't think libtree is such a vital tool it should benefit from guaranteed memory safety and whatever Rust promises.

[vsoch]

And what about Python wrappers?

I'd love to give a shot at the library in Go - I've used the elf and dwarf internals! Would you be able to write down the general process/steps for libtree for me to replicate (so I don't have to understand C++ which I'm not great at?) And it can be at a high level and I'll be strong when I need details to look at the code? it doesn't have to be any time soon!

[haampie]

See man ld.so

And what about Python wrappers?

Right now it just prints to stdout, so not sure how useful or what interface you expect. But you can just compile the C version with -shared and call the C functions?

[vsoch]

See man ld.so

Usually when someone asks for documentation about the code they aren't going to receive this kind of response well. I read this as "I don't want to help you."

Right now it just prints to stdout, so not sure how useful or what interface you expect. But you can just compile the C version with -shared and call the C functions?

The idea would be that a Python client could easily call functions (from within Python) to get either a string (or other) representation of the tree.

[haampie]

Usually when someone asks for documentation

man ld.so and man elf are the official documentation about the dynamic linker and elf files and libtree is an interpretation of those, so I'm not sure how else I can help you if you don't want to read 500 lines of C++ code.

The idea would be that a Python client could easily call functions (from within Python) to get either a string (or other) representation of the tree.

I think it's only worth creating Python wrappers if you need something fancier than string output, since otherwise you can use subprocess, right? Or is that bad practice? The fancier thing I can think of is having a visitor pattern so there's a callback to python upon entering/leaving a node in the tree, but I think for this type of use case you could also use pax-utils's lddtree.py script for dependency walking (although not 100% sure if they parse elf files or rely on the system dynamic linker).

[orbea]

What I care most about though is ease of install, and I think a C compiler is as minimal as it gets in terms of requirements.

I agree, in terms of getting this to work for as many people as possible with as few hurdles as possible using c99 or maybe c11 is the best bet and has the best chance of having no unusual dependencies that way.

Others suggested Rust, but it has the same issues, and honestly I don't think libtree is such a vital tool it should benefit from guaranteed memory safety and whatever Rust promises.

Fwiw rust is a great way to grief distro maintainers...

[haampie]

I've made it c99 after dropping the last two GNU C extensions (https://github.com/haampie/libtree-in-c/blob/890e687f6770bd5ecd6dc5552bb561c8ef481dea/Makefile#L5). Another interesting thing is that it can now be compiled as an actually portable executable or statically compiled with dietlibc (which makes the executable smaller than the source file...)

[haampie]

I've merged it to the master branch, so closing this.