deck get ...geyfra-demos.git fails on openSUSE

[rolandg333]

INFO] Deck version: 0.9.5

$ deck get https://github.com/gefyrahq/gefyra-demos.git

fails on openSuse with error message:

[CRITICAL] There was an error running deck: The command '/bin/sh -c addgroup -g ${GROUP_ID} -S tooler && adduser -u ${USER_ID} -S tooler -G tooler' returned a non-zero code: 1

'addgroup' and 'adduser' are commands not available on all kind of linux distributions. porting to 'groupadd' and 'useradd' might be a good solution ( an should be easy )

[Schille]

Hi @rolandg333 Thank you for bringing this up.

The tooler (a container image) component is built on each system when running for the first time. The code for the adduser/addgroup is this:

https://github.com/Getdeck/getdeck/blob/368b0c2debdc2f5092377f20ed309d19c807040c/getdeck/sources/tooler.py#L86-L89

Currently, there is a branch for Windows and Mac; all Linux distros fall in the else branch.

[rolandg333]

Hi Michael,

i already found the sources, and the place where i would change.

but how do i build the whole project, or test any change by myself.

On Fri, Oct 14, 2022 at 9:49 AM Michael Schilonka @.***> wrote:

Hi @rolandg333 https://github.com/rolandg333 Thank you for bringing this up.

The tooler (a container image) component is built on each system when running for the first time. The code for the adduser/addgroup is this:

https://github.com/Getdeck/getdeck/blob/main/getdeck/sources/tooler.py#L86-L89 Currently, there is a branch for Windows and Mac; all Linux distros fall in the else branch.

— Reply to this email directly, view it on GitHub https://github.com/Getdeck/getdeck/issues/85#issuecomment-1278621593, or unsubscribe https://github.com/notifications/unsubscribe-auth/AD3JMAGPAI4CRXFK4PGEYRLWDEF67ANCNFSM6AAAAAARE6KDFQ . You are receiving this because you were mentioned.Message ID: @.***>

[buschNT]

Hi @rolandg333 ,

currently, you would still have to test your changes manually. We are actively working on adding a better test setup and more tests, but this takes a little bit more time.

I develop using a venv. Then I run getdeck commands with the venv activated using python -m getdeck <COMMAND> and manually test changes. Or in case you are using VSCode, you can add the following to the configurations in your launch.json:

        {
            "name": "development",
            "type": "python",
            "request": "launch",
            "module": "getdeck",
            "justMyCode": true,
            "args": [
                "get",
                "<DECKFILE>"
            ]
        },

Sometimes you might have to add the "cwd": option, too, depending on where the deck file is located.

In order to test changes on the tooler, you have to delete your local tooler image in order to trigger a rebuild: https://github.com/Getdeck/getdeck/blob/368b0c2debdc2f5092377f20ed309d19c807040c/getdeck/sources/tooler.py#L27-L30

I created an issue regarding a development setup documentation: https://github.com/Getdeck/getdeck/issues/86

Hopefully my short development description helps, if not feel free to point out what's unclear.

[Z02X]

Mentioning this here so my issue #93 can be closed as duplicate. I believe the issue could be caused when the id's are 0, sutch as when run as root. I don't think it could be the commands missing.

[rolandg333]

the fix for #101 is not working on openSUSE. i face still the same error

i've added a debug line and found: [DEBUG] Tooler build_args: {'USER_ID': '1000', 'GROUP_ID': '100'}

because: $ id uid=1000(roland) gid=100(users) Gruppen=100(users),459(libvirt),460(qemu),463(docker),482(systemd-journal),487(kvm),496(wheel)

the docker image quay.io/getdeck/tooler is based on alpine linux; a group with gid 100 is already existing. the addgroup call fails.

when I override gid in

--- a/getdeck/sources/tooler.py
+++ b/getdeck/sources/tooler.py
@@ -79,6 +79,7 @@ def build_user_container(config: ClientConfiguration):
     if sys.platform != "win32":
         uid = os.getuid() or 1000
         gid = os.getgid() or 1000
+        gid = 1000
     else:
         uid = 1000
         gid = 1000

the tooler image gets created.

but i guess you try to pick up the uid and gid of the user on purpose ...

[Schille]

@rolandg333 thank you for your eager testing and your results. I will think about another solution then.

The point of mapping the user and group id is for the correct ownership of the generated files (in a mounted volume) later in the process. We want to write the files from within the docker container with the same user identity as the user calling deck.

[Schille]

I added a check to the image build process: if userid or groupid exist, they are not created. I assume the userid and groupid are used to map file permissions in the shared volumes to the host system. @rolandg333 are you able to test the code in branch fix/#85-1 if this is working for you? Please make sure to delete docker-tooler:0.10.1 or other Tooler images from your system. From a poetry env you could also run:

from getdeck.sources.tooler import *
from getdeck.configuration import *
build_user_container(default_configuration)

[Schille]

This is included in version 0.11.0 of deck.

[rolandg333]

hmm, now the group tooler does not get created; means the adduser call fails.

with this patch i get it working:

--- a/getdeck/sources/tooler.py
+++ b/getdeck/sources/tooler.py
@@ -87,8 +87,8 @@ def build_user_container(config: ClientConfiguration):
         user_group_add = "RUN addgroup -S tooler && adduser -S tooler -G tooler"
     else:
         user_group_add = (
-            "RUN cat /etc/group | grep ${GROUP_ID} || addgroup -g ${GROUP_ID} -S tooler && "
-            "cat /etc/passwd | grep ${USER_ID} || adduser -u ${USER_ID} -S tooler -G tooler"
+            "RUN getent group ${GROUP_ID} || addgroup -g ${GROUP_ID} -S tooler && "
+            "getent passwd ${USER_ID} || adduser -u ${USER_ID} -S tooler -G $(getent group ${GROUP_ID} | cut -d: -f1)"
         )

     Dockerfile = io.BytesIO(

[Schille]

Oh well. 🫣 I haven't had the group assignment in mind for the user creation. Your version looks much better. Will you create a PR for it? I would appreciate it.

[rolandg333]

how can i do this ? I cannot push on the branch.

[Schille]

You need to click the fork button an then create a Pull request from your repo onto this.

[rolandg333]

done

PR https://github.com/Getdeck/getdeck/pull/110

[Z02X]

I probably should have said something because I tested locally. But on some distros you would still get:

'deck-tooler:version' returned non-zero exit status 1: b'Error: stat /output/{helm chaart}/templates: permission denied\n\nUse --debug flag to render out invalid YAML

if you just use 1000.

[rolandg333]

which distro ?

[Schille]

@Z02X What distro are you talking about? Where did you get this error message from?

[Z02X]

My distro info:

NAME=Slackware
VERSION="15.0"
ID=slackware
VERSION_ID=15.0
PRETTY_NAME="Slackware 15.0 x86_64 (post 15.0 -current)"
ANSI_COLOR="0;34"
CPE_NAME="cpe:/o:slackware:slackware_linux:15.0"
HOME_URL="http://slackware.com/"
SUPPORT_URL="http://www.linuxquestions.org/questions/slackware-14/"
BUG_REPORT_URL="http://www.linuxquestions.org/questions/slackware-14/"
VERSION_CODENAME=current

I get around this by building my own image with:

RUN chown ${USER_ID}:${GROUP_ID} /sources
RUN chown ${USER_ID}:${GROUP_ID} /output

omitted. More of the output:

in image 'deck-tooler:0.11.0' returned non-zero exit status 1: b'Error: stat /output/portainer/templates: permission denied\n\nUse --debug flag to render out invalid YAML\n'

[Schille]

@Z02X what are the outputs of

import os
os.getuid()
os.getgid()

Do you run the Docker daemon in rootless mode? Do you run deck with a root user on your host? I'd say the permission errors are caused by faulty ownership on the shared volumes (/sources and /output). This seems to be a tricky issue. :thinking:

[rolandg333]

i installed Slackware 15 (on KVM).

using the adduser command, i create a new user with uid 1000 and the system has a group 'users' with gid 100

IMHO bug #85 is solved.

[rolandg333]

solved

[Schille]

Thanks a lot @rolandg333