siemens / kas

Setup tool for bitbake based projects
MIT License
353 stars 144 forks source link

cant pul repository from self signed http server #55

Closed chezgi closed 2 years ago

chezgi commented 3 years ago

when using local git server over self signed https server this error happend:

server certificate verification failed. CAfile: none CRLfile: none

also using export GIT_SSL_NO_VERIFY=1 not solved the problem.

jan-kiszka commented 3 years ago

Could you tell us a bit more details about what you are doing? E.g., are you facing issues with kas itself cloning from that server, or is bitbake stumbling during its fetches?

chezgi commented 3 years ago

in this sample:

header:
  version: 10
repos:
  meta-test:
      url: https://192.168.1.1/test/meta-test
      layers:
        meta-test-core:

we have a self signed git server at 192.168.1.1

this error happens when running:

kas checkout  myfile.yml
henning-schild commented 3 years ago

An easy way to get such a problem with a public repo would be

ssh localhost -R 4433:github.com:443
git clone https://localhost:4433/siemens/kas.git
export GIT_SSL_NO_VERIFY=""
git clone https://localhost:4433/siemens/kas.git

Did not yet dig deeper but that should give anyone a way to repro without needing an untrusted server ;)

jan-kiszka commented 3 years ago

Does bitbake respect GIT_SSL_NO_VERIFY="1"? Or does it run with such a setting by default (like it does for wget)? Then it would indeed by logical to respect it in kas as well.

chezgi commented 3 years ago

@jan-kiszka it is before running bitbake. it is in kas checkout phase. not build phase. it has nothing with bitbake.

jan-kiszka commented 3 years ago

Yes, I understand that your use case is a local layer repo server, not one with bitbake recipe artifacts. But I'm looking for solution patterns to apply them consistently. If bitbake already does things in one way and kas not yet, we can align the latter to the former. Otherwise, we risk to create inconsistent solutions for similar problems.

henning-schild commented 3 years ago

I tried it, kas and bitbake will fail to clone via https when remote trust can not be established. The trick for bitbake was the same ... change the name of the server with ssh port forwarding.

I did set GatewayPorts yesin my sshd and then renamed a repo i happened to have a recipe for ssh localhost -R 4433:salsa.debian.org:443 and changed "salsa.debian.org" in the SRC_URI to 172.17.0.1:4433 (that and the GatewayPorts) are because i am running in kas-container.

At the moment it seems that the "env" we can set in our yml does not have an effect on either bitbake nor kas, and it looks like bitbake does not seem to have any special provisioning for dealing with x509 trust.

chezgi commented 3 years ago

bitbake uses my home .gitconfig and .git-ceedentials but kas creates its home tmp dir and a simple .gitconfig.

in my system bitbake works with adding some configs to my .gitconfig file.

why kas doesn't use user's configs? if it used user's configs, then we don't need to add special configs for kas for every problem.

jan-kiszka commented 3 years ago

A primary goal of kas is reproducibility. If it pulls in things from the local user's home, it would no longer be able to ensure that the build will also work on a different machine. Your case would make a nice example for "works for me" - but no longer in CI e.g.

For that reason, all configuration should either be present in the kas.yaml files, or there should at least be an explicit statement "we are fine in pulling this or that from the local build env".

chezgi commented 3 years ago

all ci systems can maintain users configs with scripts. therefore using user's configs can be reproducible.

creating git configs is not responsibility of kas or any tool. this must be done with ci system itself.

for example we can mount our configs when using docker. or create our configs in bash script before running kas.

henning-schild commented 3 years ago

Given that both kas and bitbake fail to clone via https when trust can not be established i would conclude that we do have an issue here. One that most-likely deserves a solution.

We should probably refrain from discussions whether kas should use the users config, some people think so, many others are using kas just because of that "strict isolation". It is certainly a really bad idea to configure your git to ignore ssl only because one repo of your many projects is "broken". And to extend that hack to CI and expect others to include such a violent line in their .gitconfig. It should be possible but limited to that repo if possible, or maybe that "project" by setting the env variable which should be respected by kas and bitbake. In fact it should be respected by kas also for mercurial.

You closed the PR #54 and opened this issue instead. But really the wish was to bring it up on the mailinglist ... and you ignored that wish when opening the issue. Please bring that up on the mailinglist, maybe with a patch if you want. I am sure we will find a solution and will be able to add testcases for that corner-case ... to make it a proper feature.

jan-kiszka commented 2 years ago

We've enhanced certain things recently, e.g. added NETRC_FILE. Still something to do for this?

chezgi commented 2 years ago

i am not active on kas.

g4bwy commented 1 year ago

Hi,

see also my answer to #43