Porting termux packages / reusing patches

[deliciouslytyped]

It might be interesting to look into reusing work that has gone into termux packages. (placeholder)

[deliciouslytyped]

(sidenote on motivation: having packages capable or running in a proot environment may be interesting beyond the android usecase? It would be nice to get some confirmation on this.)

Ok I'm probably shelving this for a while but the basic idea is to see if we can automatically ingest termux packages from https://github.com/termux/termux-packages and it's sub-repositories into an autogenerated overlay. termux-packages seems to be apt based, and I don't know if nixpkgs has any infra for helping with that, nor do I know my way around apt.

Some preliminary poking around suggests that the main interesting things will just be the patches. I haven't seen anything but I'm slightly worried about there being package specific packager issues, but we will probably have to handle that like always, with individual overrides. We also need a mapping file from termux package names to nix package names. I don't know if there are any cases of this, but I'm not sure yet what to do if there are any packages that correspond to multiple nix packages.

It looks like it may be necessary to manually review patches; many patches are about fixing hardcoded paths. However, for this at least, it may be sufficient to do have some automatic rewriting rules.

Furthermore it may be interesting to have some code that checks binaries for references to functions proot doesnt support, and maybe warn the user or packager. Obviously this would be a lower bound and would not catch everything. It may be largely sufficient to check the dynamic linker imports, and perhaps check for manual dynamic linker calls? Furthermore we should look into whether proot can do any logging for unsupported calls. Debugging the issue mentioned in the following paragraph may have been easier if there was a porting guide that mentioned to check something of the sort?

For opaque binary packages, either patches to proot that make things fail-nop instead of fail-fail may be interesting. This may also be handy if you just don't wan't to patch and recompile something. For example the setuid(getuid()) issue from https://github.com/t184256/nix-on-droid/issues/75. See also the LD_PRELOAD fix. termux-packages fixes this issue by completely removing the checks, which my fix is equivalent to. (I kind of wonder if this is CVE related or something). It may be interesting to just have some reconfigurable LD_PRELOAD library with various instrumentation options. (TODO: look for previous work) Besides the static packaging checks, there could also be runtime checks based on this - but since LD_PRELOAD uses the dynamic linker anyway(i.e. the static approach looks at the link tables), it's probably redundant and unnecessary?

I haven't looked for or checked any proot documentation yet.

So to start off I propose the following steps for minimum viability:

• [ ] Write a prototype that ingests a single package and generates an overlay for it
• [ ] Write a name mapping functionality and extend (just a simple map operation? :P) the previous to the whole package set
• [ ] Try to get the aarch64 cache guys to accept this package set / get this as some tier or other of supported in nixpkgs - this is part of minimum-viable because I don't want to be compiling this stuff on my phone.

Bonuses / uncategorized / not needed for minimum viability:

• [ ] find usecases for proot package support beyond android
• [ ] package-time checks for breaking function usage (i.e. there may or may not be some tooling for packaging)
• [ ] install-time checks for breaking function usage
• [ ] LD_PRELOAD and other instrumentation tooling
• [ ] proot failure logging
• [ ] porting guide
• [ ] (big?) write tests / check if existing nixos tests will work / look into porting the test system?

Single-ingest:

• [ ] path patch detection and remapping TODO

Package-set-ingest:

• [ ] recursive packages: termux-packages has some packages that install package sets
• [ ] name-mappings TODO

Proot help is at https://proot-me.github.io/#support .

[Gerschtli]

Thank you for your input! Thats an interesting idea to kind of automate the overlay creation but I see some things to look out for:

• IIRC @t184256 once mentioned that most of termux fixes concerns the bionic/android environment which in our case isn't a problem: We are running an aarch64 environment thanks to proot.
• Package version mismatches could be a problem: What if nixpkgs has an older version than termux and build instructions have changed?
• Concerning the cache: I think the best way for now would be to regularly build the patched packages (with a pinned version of nixpkgs which gets updated each time the build job has built every package successfully) and push them to nix-on-droid.cachix.org. This cache is currently used for proot builds and other packages and is used on each nix-on-droid installation.

So I think of a semi-automatic workflow where any contributor could easily port a termux package to an overlay. This could be supported by some utility scripts or whatever.

BTW: I love the idea of bringing tests to nix-on-droid!

[deliciouslytyped]

Thanks! All of that sounds good to me.

• I don't really see into the first point yet, but sounds valid.
• I suppose some more of the information we could track is whether things follow semver, and maybe hope for compat? :P - but yeah, sounds less sunshine and rainbows than I'd hoped. :P It's also possible that termux has the much older version? :P IDK where termux pulls or builds it's packages from.
• I've never used cachix but it sounds like a starting point.

[deliciouslytyped]

I asked someone to review this post and they brought to my attention:

As I said, termux fixes are not typically about running within proot, since termux normally does not do so. They are however, about running within the constraints of a single-user environment, so I would tweak the motivation section of your issue to say that. nix-on-droid like termux is running within one user, so it can't do user privilege separation the way nix wants to.

TODO: take advice.