search

edannenberg / kubler

A generic, extendable build orchestrator.
BSD 2-Clause "Simplified" License
158 stars 40 forks source link

Upstream bug: users and groups created during install of packages now respect ROOT #209

Open edannenberg opened 2 years ago

edannenberg commented 2 years ago

This commit breaks the build for uncached packages that create users/groups.

Problems:

@thesamesam Could this be reverted for now until all the edge cases are fleshed out? Right now it does more harm then good as its much harder to work around then with the old behavior.

thesamesam commented 2 years ago

Thanks! This sounds like ebuilds may have incorrect dependencies (it's an issue when installing from binpkgs too). Just got up though so let me have a think 🤔

Is there an example set of steps I can play with?

thesamesam commented 2 years ago

cc @Mord3rca

Mord3rca commented 2 years ago

Hey ! I see .. The only way to mitigate the fowners issue is to wrap the chown / fchown syscall... So, all commands will be affected at once. I did not try a full test build at work, but we have this kind of wrapper to avoid setting an unknown permission + we hook getpwname to return an internal list of users read from ${ROOT}/etc/passwd so commands like chown won't crash with a user which is not from the host. I'll see if this can correct the error. If so, maybe we can add the hooked function to libsandbox. I'll keep you informed.

thesamesam commented 2 years ago

We discussed this a bit in #gentoo-base on IRC and I think we need some logs to properly understand it. Could you file a bug on Gentoo's Bugzilla with that information? (inc. e.g an example ebuild, and some steps to repro).

Needing BROOT to contain things in order to do e.g. fowners is considered normal and something we've been doing for ages. I'm not against a change along the lines @Mord3rca suggests in principle, I think, but would need to see it in more detail?

Everything but point 2. should be handled by dependencies and point 2 seems like it should be handled by baselayout.

Mord3rca commented 2 years ago

@thesamesam Good news ! I've started a full rebuild (not of this, just to be clear) with my modification in eclasses + my hook for getpwnam / getgroupnam etc and it seems to do the trick for fchown. I'm starting to adapt my work for libsandbox so you can hopefully test it soon.

edannenberg commented 2 years ago

Thanks for the swift replies!

Is there an example set of steps I can play with?

Any ebuild that creates a user/group and then tries to fowners with the new user will do, provided the user doesn't exist on the host yet.

Simple test ebuild ``` # Copyright 1999-2017 Gentoo Foundation # Distributed under the terms of the GNU General Public License v2 EAPI=7 DESCRIPTION="A test" inherit user vcs-snapshot EGIT_COMMIT="be5b2e1" SRC_URI="https://github.com/edannenberg/kubler/archive/${EGIT_COMMIT}.tar.gz -> ${P}.tar.gz" KEYWORDS="alpha amd64 arm arm64 ia64 ppc sparc x86" IUSE="bar" SLOT="0" DEPEND="" RDEPEND="" pkg_setup() { enewgroup kubler enewuser kubler -1 -1 /usr/share/${PN} kubler } src_install() { insinto /usr/share/${PN} doins -r bin/ cmd/ engine/ lib/ template/ kubler.conf kubler.sh README.md COPYING fowners kubler:kubler /usr/share/${PN} } ```

We discussed this a bit in #gentoo-base on IRC and I think we need some logs to properly understand it. Could you file a bug on Gentoo's Bugzilla with that information? (inc. e.g an example ebuild, and some steps to repro).

Given the example ebuild above:

Original behavior:

$ ROOT=/foo emerge dev-util/foo
$ cat /etc/group
<snip>
kubler:x:249:
$ cat /etc/passwd
<snip>
kubler:x:999:249:added by portage for foo:/usr/share/foo:/bin/false
$ cat /foo/etc/passwd
cat: /foo/etc/passwd: No such file or directory
Current behavior ``` $ rm /packages/x86_64-pc-linux-gnu/dev-util/foo/foo-0.0.1-1.xpak $ ROOT=/foo emerge dev-util/foo >>> Emerging (1 of 1) dev-util/foo-0.0.1::kubler for /foo/ >>> Failed to emerge dev-util/foo-0.0.1 for /foo/, Log file: >>> '/var/tmp/portage/dev-util/foo-0.0.1/temp/build.log' >>> Jobs: 0 of 1 complete, 1 failed Load avg: 3.80, 5.65, 3.70 * Package: dev-util/foo-0.0.1 * Repository: kubler * Maintainer: erik.dannenberg@xtrade-gmbh.de * USE: abi_x86_64 amd64 elibc_glibc kernel_linux userland_GNU * FEATURES: preserve-libs sandbox userpriv usersandbox realpath: /foo/usr/lib: No such file or directory grep: /foo/etc/group: No such file or directory * Adding group 'kubler' to your system ... * - Groupid: next available groupadd: /foo//etc/group.153: No such file or directory groupadd: cannot lock /foo//etc/group; try again later. * ERROR: dev-util/foo-0.0.1::kubler failed (setup phase): * (no error message) * * Call stack: * ebuild.sh, line 127: Called pkg_setup * foo-0.0.1.ebuild, line 29: Called enewgroup 'kubler' * user.eclass, line 353: Called die * The specific snippet of code: * groupadd -r "${opts[@]}" "${egroup}" || die $ mkdir -p /foo/{etc,usr/lib} $ ROOT=/foo emerge dev-util/foo >>> Failed to emerge dev-util/foo-0.0.1 for /foo/, Log file: >>> '/var/tmp/portage/dev-util/foo-0.0.1/temp/build.log' >>> Jobs: 0 of 1 complete, 1 failed Load avg: 4.05, 5.74, 4.16 * Package: dev-util/foo-0.0.1o-0.0.1::kubler for /foo/ * Repository: kubler * Maintainer: erik.dannenberg@xtrade-gmbh.de * USE: abi_x86_64 amd64 elibc_glibc kernel_linux userland_GNU * FEATURES: preserve-libs sandbox userpriv usersandbox grep: /foo/etc/group: No such file or directory * Adding group 'kubler' to your system ... * - Groupid: next available grep: /foo/etc/passwd: No such file or directory * Adding user 'kubler' to your system ... grep: /foo/etc/passwd: No such file or directory * - Userid: 999 # Omitted the actual error that failed the build, the above errors are just from the # checks to determine the next free uid/gid and don't fail the build. # fowners don't finding the kubler user fails the build. $ cat /foo/etc/group kubler:x:999: $ cat /foo/etc/passwd kubler:!:999:999:added by portage for foo:/usr/share/foo:/bin/false ```

The user is created at ROOT now but as the user doesn't exist on the host fowners fails the build. Happy to file a bug if you still think that would help.

Needing BROOT to contain things in order to do e.g. fowners is considered normal and something we've been doing for ages.

Not sure I follow, could you elaborate please?

Everything but point 2. should be handled by dependencies and point 2 seems like it should be handled by baselayout.

Not sure what you mean with handled by dependencies. Point 3 was me being confused by portage's environment feature, so old binary packages still worked as before (TIL). The behavior is consistent, if you manage to create a new binary package that is. Workarounds:

Point 2 is a nice to have and I can work around that fairly easy by making sure the paths enewgroup/enewuser expect actually exist. Ideally the eclass would skip the check if ROOT doesn't have a passwd/group file.

Mord3rca commented 2 years ago

Hello, I'm making progress on the patch for adding shadow capabilities for libsandbox, however my patch do not work in the sandboxed context. I need to read more carefully the code to understand why. (It works when using LD_PRELOAD so .. It's something I guess) FYI, my branch can be found here: https://github.com/Mord3rca/sandbox/commits/libsandox/shadow_hook

@edannenberg regarding your workaround for the fowners situation, I guess you can do a more elegant solution. A lot of services (postfix, opendkim, ...) have the acct-user / acct-group related to their executing in both RDEPEND & BDEPEND, so you install the user in the host & target. This can be done only if you control the ebuild code of course.

Also, for your test build, be sure to install baselayout before everything else, if not, passwd / group files won't exist.

edannenberg commented 1 year ago

Sorry, I meant to reply to this sooner. Another option would be to support both behaviours via some ENV, the kubler patches should make this fairly easy to implement I think.