search

goharbor / harbor

An open source trusted cloud native registry project that stores, signs, and scans content.
https://goharbor.io
Apache License 2.0
23.95k stars 4.74k forks source link

Internal TLS not working when tutorial followed 2.0.0 #11934

Open kuburoman opened 4 years ago

kuburoman commented 4 years ago

Expected behavior and actual behavior: Expected is that Harbor started and is running properly Actual is that nginx-photon is constantly restarted and rest of pod reports errors.

Steps to reproduce the problem: Followed all the steps of tutorial to deploy Harbor with internal tls enabled. In steps there is written to run command docker run -v /:/hostfs goharbor/prepare:v2.0 gencert -p /path/to/internal/tls/cert but version should be v2.0.0. Installation does not show any errors.

Versions: Please specify the versions of following systems.

Additional context:

kuburoman commented 4 years ago

Without Internal tls Harbor starts as expected.

gregorkistler commented 4 years ago

Same here. Jobservice and proxy container ending in a restart loop.

reasonerjt commented 4 years ago

@kuburoman Did you use /path/to/internal/tls/cert in your command? I believe the doc implies it should be the path on your host to store the generated certificates.

Please attach the command you used to gen the cert and harbor.yml

ninjadq commented 4 years ago

/path/to/internal/tls/cert should the same as the value of internal_tls.dir in your harbor.yml, could you provide your harbor.yml file?

kuburoman commented 4 years ago

Command ./install.sh --with-notary --with-clair. If installation cant find certificates /opt/harbor/tls then it fails with error, but everything looks ok after installation, but every container has error about certificates.

Harbor.yml (Commented lines removed)

hostname: k8sharbor

http:
  port: 80

https:
  port: 443
  certificate: /opt/harbor/cert/k8sharbor.crt
  private_key: /opt/harbor/cert/k8sharbor.key

internal_tls:
  enabled: true
  dir: /opt/harbor/tls

harbor_admin_password: Harbor12345

database:
  password: Harbor12345
  max_idle_conns: 50
  max_open_conns: 100

data_volume: /srv/harbor

clair:
  updaters_interval: 12

trivy:
  ignore_unfixed: false
  skip_update: false
  insecure: false

jobservice:
  max_job_workers: 10

notification:
  webhook_job_max_retry: 10

chart:
  absolute_url: disabled

log:
  level: info
  local:
    rotate_count: 50
    rotate_size: 200M
    location: /srv/logs/harbor

_version: 2.0.0

proxy:
  http_proxy:
  https_proxy:
  no_proxy:
  components:
    - core
    - jobservice
    - clair
    - trivy
ninjadq commented 4 years ago

Based on your configs, you must run docker run -v /:/hostfs goharbor/prepare:v2.0 gencert -p /opt/harbor/tls before installation in order to generate certs for harbor components

kuburoman commented 4 years ago

Based on your configs, you must run docker run -v /:/hostfs goharbor/prepare:v2.0 gencert -p /opt/harbor/tls before installation in order to generate certs for harbor components

I ran that command before. Content of /opt/harbor/tls

clair_adapter.crt  harbor_db.csr           job_service.key    registry.crt
clair_adapter.csr  harbor_db.key           notary_server.crt  registry.csr
clair_adapter.key  harbor_internal_ca.crt  notary_server.csr  registryctl.crt
clair.crt          harbor_internal_ca.key  notary_server.key  registryctl.csr
clair.csr          harbor_internal_ca.srl  notary_signer.crt  registryctl.key
clair.key          chartmuseum.crt         notary_signer.csr  registry.key
core.crt           chartmuseum.csr         notary_signer.key  trivy_adapter.crt
core.csr           chartmuseum.key         proxy.crt          trivy_adapter.csr
core.key           job_service.crt         proxy.csr          trivy_adapter.key
harbor_db.crt      job_service.csr         proxy.key

Also image goharbor/prepare:v2.0 does not exist. I ran command: docker run -v /:/hostfs goharbor/prepare:v2.0.0 gencert -p /opt/harbor/tls

ninjadq commented 4 years ago

the goharbor/prepare:v2.0.0 is uploaded in docker-hub, if you can access the internet, docker will download it automatically. And this image also included in your offline tar file. If you can not access the internet, you need to load these images first via command docker load -i ./harbor*.tar.gz

kuburoman commented 4 years ago

the goharbor/prepare:v2.0.0 is uploaded in docker-hub, if you can access the internet, docker will download it automatically. And this image also included in your offline tar file. If you can not access the internet, you need to load these images first via command docker load -i ./harbor*.tar.gz

In documentation there is command with docker image goharbor/prepare:v2.0. That results in Error response from daemon: manifest for goharbor/prepare:v2.0 not found: manifest unknown: manifest unknown.

I ran command with docker image goharbor/prepare:v2.0.0 that exited correctly and generated certificates. After that I ran command ./install.sh --with-notary --with-clair that also exited correctly. But then i got error from container logs.

kuburoman commented 4 years ago

Also i used offline installer.

gregorkistler commented 4 years ago

Could solve it in my case. The container mounted directory /harbor_cust_cert/ only listed an empty core.crt file which seem to get generated on each docker-compose up. I've simply copied the harbor_internal_ca.crt file (generated via gencert) to the hard-coded directory make/common/config/shared/trust-certificates and redeployed containers.

Likely something on my end as I run Harbor on aarch64. As this does not work out-of-the-box I use my own script to build related Harbor images, build scanner binaries, alter some files and do some other stuff.

@kuburoman, as your proxy.log shows the same entries, could you check the content of make/common/config/shared/trust-certificates?

reasonerjt commented 4 years ago

@ninjadq any update?

ba-work commented 4 years ago

I hit the exact same issues.

Documentation clearly says goharbor/prepare:v2.0 but it should say goharbor/prepare:v2.0.0 (note the extra .0 at the end)

Evidence: https://hub.docker.com/r/goharbor/prepare/tags?page=1&name=2.0

Also had the same where harbor_internal_ca.crt was missing from ./common/config/shared/trust-certificates. Simple copy from the internal_tls location and redeploy fixed it on my end as well. This should probably be done as part of install.sh

Another thing I'd like to point out is that a first time install with internal_tls is logically impossible if you follow the instructions: You have to run goharbor/prepare to get certs before installing, but you have to run the installer to load the image for prepare (see the issue here?). Its easy enough to modify install.sh to work around this, but it this should probably just be handled in install.sh for the user. (the command is documented why not just run it for the user?).

Maybe internal TLS should be a --with-internal-tls flag, rather than a "edit the compose file" setting.

stale[bot] commented 4 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

ofek commented 4 years ago

@ninjadq Any update on this?

jr200 commented 3 years ago

I believe this issue still exists in harbor v2.2.0. After enabling internal_tls, and running install.sh, the nginx containers enters a restart loop due to the missing CA at common/config/shared/trust-certificates. Copying the file over solves this problem.

lqdflying commented 3 years ago

same as v2.3.0, Couldn't they just correct the bug so everyone don't need to search the solution from a closed issues?

I believe this issue still exists in harbor v2.2.0. After enabling internal_tls, and running install.sh, the nginx containers enters a restart loop due to the missing CA at common/config/shared/trust-certificates. Copying the file over solves this problem.

SistemasMicro commented 1 year ago

same as v2.8.0, Couldn't they just correct the bug so everyone don't need to search the solution from a closed issues?

vndroid commented 10 months ago

same as v2.8.0, Couldn't they just correct the bug so everyone don't need to search the solution from a closed issues?

Yeah, it's pretty stupid.

AugustasV commented 9 months ago

Still same problem

github-actions[bot] commented 7 months ago

This issue is being marked stale due to a period of inactivity. If this issue is still relevant, please comment or remove the stale label. Otherwise, this issue will close in 30 days.

vndroid commented 7 months ago

v2.9 still

luckylinux commented 6 months ago

Not sure if related (I was trying with Podman) but I even get a crash during install:

./prepare

prepare base dir is set to /home/podman/containers/local/harbor-online-installer-v2.10.2/harbor
Clearing the configuration file: /config/jobservice/env
Clearing the configuration file: /config/jobservice/config.yml
Clearing the configuration file: /config/log/rsyslog_docker.conf
Clearing the configuration file: /config/log/logrotate.conf
Clearing the configuration file: /config/nginx/nginx.conf
Clearing the configuration file: /config/registryctl/config.yml
Clearing the configuration file: /config/registryctl/env
Clearing the configuration file: /config/db/env
Clearing the configuration file: /config/portal/nginx.conf
Clearing the configuration file: /config/core/env
Clearing the configuration file: /config/core/app.conf
Clearing the configuration file: /config/registry/config.yml
Clearing the configuration file: /config/registry/passwd
Generated configuration file: /config/portal/nginx.conf
Generated configuration file: /config/log/logrotate.conf
Generated configuration file: /config/log/rsyslog_docker.conf
Generated configuration file: /config/nginx/nginx.conf
Generated configuration file: /config/core/env
Generated configuration file: /config/core/app.conf
Generated configuration file: /config/registry/config.yml
Generated configuration file: /config/registryctl/env
Generated configuration file: /config/registryctl/config.yml
Generated configuration file: /config/db/env
Generated configuration file: /config/jobservice/env
Generated configuration file: /config/jobservice/config.yml
Traceback (most recent call last):
  File "/usr/src/app/main.py", line 15, in <module>
    cli()
  File "/usr/lib/python3.11/site-packages/click/core.py", line 1137, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/click/core.py", line 1062, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/click/core.py", line 1668, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/click/core.py", line 763, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/app/commands/prepare.py", line 50, in prepare
    prepare_tls(config_dict)
  File "/usr/src/app/utils/internal_tls.py", line 4, in prepare_tls
    config_dict['internal_tls'].validate()
  File "/usr/src/app/models.py", line 102, in validate
    self._check(filename)
  File "/usr/src/app/models.py", line 84, in _check
    if not owner_can_read(path.stat().st_mode):
                          ^^^^^^^^^^^
  File "/usr/lib/python3.11/pathlib.py", line 1013, in stat
    return os.stat(self, follow_symlinks=follow_symlinks)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/data/secret/tls/portal.crt'
github-actions[bot] commented 4 months ago

This issue is being marked stale due to a period of inactivity. If this issue is still relevant, please comment or remove the stale label. Otherwise, this issue will close in 30 days.

github-actions[bot] commented 3 months ago

This issue was closed because it has been stalled for 30 days with no activity. If this issue is still relevant, please re-open a new issue.

WhoWouldaThunk commented 2 months ago

As others have said, installing it with internal_tls enabled: true doesn't work at all using the install.sh.

-p path should be the same as the one under interal_tls: dir: /wherever/you/want/certs 1) docker run -v /:/hostfs goharbor/prepare:v2.10.3 gencert -p /wherever/you/want/certs 2) sudo cp /wherever/you/want/certs/* /wherever_harbor_is/common/config/shared/trust-certificates 3) sudo ./install.sh --with-trivy

This works. So either the internal_tls dir: doesn't do what it's supposed to, or something somewhere is screwed up. But you have to copy the certs to the common/config/shared/trust-certificates folder so that harbor will run.

github-actions[bot] commented 3 weeks ago

This issue is being marked stale due to a period of inactivity. If this issue is still relevant, please comment or remove the stale label. Otherwise, this issue will close in 30 days.

AugustasV commented 3 weeks ago

As others have said, installing it with internal_tls enabled: true doesn't work at all using the install.sh.

-p path should be the same as the one under interal_tls: dir: /wherever/you/want/certs

  1. docker run -v /:/hostfs goharbor/prepare:v2.10.3 gencert -p /wherever/you/want/certs
  2. sudo cp /wherever/you/want/certs/* /wherever_harbor_is/common/config/shared/trust-certificates
  3. sudo ./install.sh --with-trivy

This works. So either the internal_tls dir: doesn't do what it's supposed to, or something somewhere is screwed up. But you have to copy the certs to the common/config/shared/trust-certificates folder so that harbor will run.

great insights, hope it will be implemented