siemens / kas

Setup tool for bitbake based projects
MIT License
392 stars 158 forks source link

KAS checkout not working in vscode devcontainer #101

Closed mschnelte closed 1 month ago

mschnelte commented 1 year ago

kas checkout is not working in vscode devcontainers because the needed environment variable

REMOTE_CONTAINERS_IPC

is not available to the git process.

The checkout fails with "Dev Containers CLI: RPC pipe not configured."

It seems that the env section of the yml file is not parsed during checkout. Hence adding REMOTE_CONTAINERS_IPC to this section is not sufficient.

Changing the kas/context.py line 109 to:

for key in ['http_proxy', 'https_proxy', 'ftp_proxy', 'no_proxy',
                    'SSH_AUTH_SOCK',
                    'BB_NUMBER_THREADS', 'PARALLEL_MAKE', "REMOTE_CONTAINERS_IPC"]:

solved that issue for me. However would it not be better to parse the env section of the yml file already before checkout?

jan-kiszka commented 1 year ago

The env variables are targeting the bitbake environment, that's why they do not help here. Also, we can't fully parse the env key before all repos have been checked out and all kas includes have been merged.

Your case (which no one requested so far) is more comparable to GIT_CREDENTIAL_HELPER & Co. Try to extend kas along that line.

mschnelte commented 1 year ago

Hi Jan,

tnx for your reply.

I would not know how to use GIT_CREDENTIAL_HELPER to support the devcontainer scenario. The vscode devcontainer extension is injecting a git credential helper into the container that is using the git credential helper of the host in order to avoid the need of setting up git inside the container.

The git credential helper inside the container needs the environment variable REMOTE_CONTAINERS_IPC in order to work. Hence this variable needs to be whitelisted. I think it falls into the same category as SSH_AUTH_SOCK.

jan-kiszka commented 8 months ago

I suppose it still needs someone to dive deeper into this. I'm not the right candidate as I'm not using vscode. Maybe there are more chances when taking this to the mailing list.

okaestne commented 7 months ago

I just had the same problem. A few things are necessary for this to work:

REMOTE_CONTAINERS_IPC needs to be passed into both, the kas env and to bitbake via BB_ENV_EXTRAWHITE. This can be easily be done in the kas.yaml:

env:
  # for devcontainer
  REMOTE_CONTAINERS_IPC: null

The second part is bitbake's fetch2 cleaned env for git, which needs to contain REMOTE_CONTAINERS_IPC as well.

This is only in master right now: https://git.yoctoproject.org/poky/commit/bitbake/lib/bb/fetch2/__init__.py?id=6cb490713ed

So right now, we need to patch bitbake or wait for the next release.

fmoessbauer commented 7 months ago

@okaestne We have a RFC for that patch on the kas ML for some time now. It would be great if someone could review and test it (ideally with the bitbake from master): https://groups.google.com/g/kas-devel/c/_eQsWitKflo/m/Q3ZhBkr7AQAJ

okaestne commented 2 months ago

I've just tested this patch on kas 4.5 and scarthgap. In my vscode devcontainer it copies the .gitconfig and the REMOTE_CONTAINERS_* env vars as desired and lets me clone private repos just fine.

fmoessbauer commented 2 months ago

Ehm... The patch is not applied yet. I'm currently rewriting it to rebase and to add the missing documentation. Please keep this issue open until applied.

okaestne commented 2 months ago

@fmoessbauer You mention codespaces in your commit. I tested with one of these today as well and found out that they provide the git credentials using a dead-simple git credential helper, that just echoes the parameters, including the GITHUB_TOKEN env var as password. So kas would need to pass that one through. This requires a patch for bitbake as well.

I would try to send a patch for bitbake first and a separate one for kas if it's been accepted, or what do you think?

fmoessbauer commented 2 months ago

@fmoessbauer You mention codespaces in your commit.

Well... I thought that this works the same way - but apparently I doesn't. I'll just remove this part of the commit message.

I would try to send a patch for bitbake first and a separate one for kas if it's been accepted, or what do you think?

That sounds like a plan. Anyways, thanks for testing!