search

gentoo / dlang

[MIRROR] D programming language ebuild repository
https://gitweb.gentoo.org/repo/user/dlang.git
GNU General Public License v2.0
30 stars 25 forks source link

Automate gdmd syncing with gcc versions #119

Closed the-horo closed 1 year ago

the-horo commented 1 year ago

I've created a script to manage updating gdmd ebuilds and the gdc USE flags in dlang-compilers.eclass and use.desc based on the available gcc ebuilds in the main tree. Any feedback on it is appreciated.

mleise commented 1 year ago

Took the time to try it out today. Looks pretty awesome. You probably have tried to make it not depend on super user rights already? I think using chown 1000:1000 implying that the current user has that id is not ideal. Also cleaning up /var/tmp/portage after extracting all the GCC versions would be nice to have.

the-horo commented 1 year ago

You probably have tried to make it not depend on super user rights already?

The script does require unpacking, cleaning, and access to /var/tmp/portage/sys-devel/gcc which all require super user, all the other logic works fine without.

I think using chown 1000:1000 implying that the current user has that id is not ideal.

I added that in because I do most of the testing for this repo inside a chroot with only a root account and having files in the repo that my normal user can't access (i.e. overwrite) was annoying as I had to constantly chown them. However, you are right, this option is definitely not correct so I removed the offending lines.

Also cleaning up /var/tmp/portage after extracting all the GCC versions would be nice to have.

In the original script I forgot to reenable ebuild clean after I disabled it for faster iterations when testing it.

Other changes from the original script include:

  1. added a asroot function execute the few commands that require root
  2. started filtering hppa and sparc keywords since D isn't enabled for gcc on this arches (as per the comment in dlang-compilers.eclass and pkgcheck's output)
mleise commented 1 year ago

Other changes from the original script include:

1. added a `asroot` function execute the few commands that require root

2. started filtering `hppa` and `sparc` keywords since D isn't enabled for gcc on this arches (as per the comment in `dlang-compilers.eclass` and `pkgcheck`'s output)

Sounds good! I almost forgot about those keywords as well.

The script does require unpacking, cleaning, and access to /var/tmp/portage/sys-devel/gcc which all require super user, all the other logic works fine without.

Have you tried setting the PORTAGE_TMPDIR env variable temporarily to a user writable directory like /tmp or something during the ebuild commands? You can normally do all the ebuild commands except the actual installation from a user account that way. (If that's the only things that require root.) Also, would you like to become a maintainer with push access? You've been quite involved for a while now, not just pushing a new version of an ebuild but also improving the "guts" of the overlay.

the-horo commented 1 year ago

Have you tried setting the PORTAGE_TMPDIR env variable temporarily to a user writable directory like /tmp or something during the ebuild commands? You can normally do all the ebuild commands except the actual installation from a user account that way.

At first it seems to work but I still get an error which I think is ebuild complaining that it can't set up a proper build environment (with permissions, ownership etc.) because it requires root to do it:

bash: line 1: /var/cache/distfiles/.__portage_test_write__: Permission denied
!!! No write access to '/var/cache/distfiles'
 * gcc-11.4.0.tar.xz BLAKE2B SHA512 size ;-) ...                                                                        [ ok ]
 * gcc-11.4.0-patches-10.tar.xz BLAKE2B SHA512 size ;-) ...                                                             [ ok ]
 * gcc-11.4.0-musl-patches-2.tar.xz BLAKE2B SHA512 size ;-) ...                                                         [ ok ]
Traceback (most recent call last):
  File "/usr/lib/python3.11/site-packages/portage/util/__init__.py", line 1236, in apply_permissions
    os.chown(filename, uid, gid)
  File "/usr/lib/python3.11/site-packages/portage/__init__.py", line 280, in __call__
    rval = self._func(*wrapped_args, **wrapped_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
PermissionError: [Errno 1] Operation not permitted: b'/tmp/dlang-overlay/portage/sys-devel/gcc-11.4.0/distdir'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python-exec/python3.11/ebuild", line 406, in <module>
    a = portage.doebuild(
        ^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/portage/package/ebuild/doebuild.py", line 1403, in doebuild
    _prepare_fake_distdir(mysettings, alist)
  File "/usr/lib/python3.11/site-packages/portage/package/ebuild/prepare_build_dirs.py", line 503, in _prepare_fake_distdir
    portage.util.ensure_dirs(edpath, gid=portage_gid, mode=0o755)
  File "/usr/lib/python3.11/site-packages/portage/util/__init__.py", line 1616, in ensure_dirs
    perms_modified = apply_permissions(dir_path, **kwargs)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/portage/util/__init__.py", line 1243, in apply_permissions
    raise OperationNotPermitted(func_call)
portage.exception.OperationNotPermitted: chown('/tmp/dlang-overlay/portage/sys-devel/gcc-11.4.0/distdir', -1, 250)

So, the only upside to changing $PORTAGE_TMPDIR would be not polluting the default directory with the downside of the commands becoming asroot env PORTAGE_TMPDIR=/our/new/path ebuild ... since doas, at least, doesn't keep env variables.

Also, would you like to become a maintainer with push access?

I'd love that, however, I don't have any experience in contributing to open source projects outside of my commits here and I'm afraid, in my attempts to improve, I will get a few things wrong which I would rather not be caught by normal users. But I also won't get any experience if I don't contribute more so I don't think I should shy away from this. Anyway you got more expertise on this and my previous remarks are just me blabbering but I'm confident that I won't screw up trivial package bumps or stabilizations but changes to dlang.eclass or dmd.eclass I would like to have reviewed in PRs, if you think it is wise.

I assume the procedure is the same you described in https://github.com/gentoo/dlang/pull/120#issuecomment-1685200854.

the-horo commented 1 year ago

Changed to script to use quse instead of equery for less dependencies and a significant faster first part of the script.

mleise commented 1 year ago

I'd love that, however, I don't have any experience in contributing to open source projects outside of my commits here and I'm afraid, in my attempts to improve, I will get a few things wrong which I would rather not be caught by normal users. But I also won't get any experience if I don't contribute more so I don't think I should shy away from this. Anyway you got more expertise on this and my previous remarks are just me blabbering but I'm confident that I won't screw up trivial package bumps or stabilizations but changes to dlang.eclass or dmd.eclass I would like to have reviewed in PRs, if you think it is wise.

Sounds good. And don't worry about breaking things too much. Most packages break on their own after not being maintained for a while. Normally, in a larger project we'd always do PRs and have another contributor/colleague look over it. I think your proposal to do just package bumps without a PR is perfect for this repo.

I assume the procedure is the same you described in #120 (comment).

Yes, though it seems the assignee email cannot be set by you, so you may have to wait for a Gentoo dev to assign it.