Open kuburoman opened 4 years ago
Without Internal tls Harbor starts as expected.
Same here. Jobservice and proxy container ending in a restart loop.
@kuburoman
Did you use /path/to/internal/tls/cert
in your command? I believe the doc implies it should be the path on your host to store the generated certificates.
Please attach the command you used to gen the cert and harbor.yml
/path/to/internal/tls/cert
should the same as the value of internal_tls.dir
in your harbor.yml
, could you provide your harbor.yml file?
Command ./install.sh --with-notary --with-clair
.
If installation cant find certificates /opt/harbor/tls
then it fails with error, but everything looks ok after installation, but every container has error about certificates.
Harbor.yml (Commented lines removed)
hostname: k8sharbor
http:
port: 80
https:
port: 443
certificate: /opt/harbor/cert/k8sharbor.crt
private_key: /opt/harbor/cert/k8sharbor.key
internal_tls:
enabled: true
dir: /opt/harbor/tls
harbor_admin_password: Harbor12345
database:
password: Harbor12345
max_idle_conns: 50
max_open_conns: 100
data_volume: /srv/harbor
clair:
updaters_interval: 12
trivy:
ignore_unfixed: false
skip_update: false
insecure: false
jobservice:
max_job_workers: 10
notification:
webhook_job_max_retry: 10
chart:
absolute_url: disabled
log:
level: info
local:
rotate_count: 50
rotate_size: 200M
location: /srv/logs/harbor
_version: 2.0.0
proxy:
http_proxy:
https_proxy:
no_proxy:
components:
- core
- jobservice
- clair
- trivy
Based on your configs, you must run docker run -v /:/hostfs goharbor/prepare:v2.0 gencert -p /opt/harbor/tls
before installation in order to generate certs for harbor components
Based on your configs, you must run
docker run -v /:/hostfs goharbor/prepare:v2.0 gencert -p /opt/harbor/tls
before installation in order to generate certs for harbor components
I ran that command before. Content of /opt/harbor/tls
clair_adapter.crt harbor_db.csr job_service.key registry.crt
clair_adapter.csr harbor_db.key notary_server.crt registry.csr
clair_adapter.key harbor_internal_ca.crt notary_server.csr registryctl.crt
clair.crt harbor_internal_ca.key notary_server.key registryctl.csr
clair.csr harbor_internal_ca.srl notary_signer.crt registryctl.key
clair.key chartmuseum.crt notary_signer.csr registry.key
core.crt chartmuseum.csr notary_signer.key trivy_adapter.crt
core.csr chartmuseum.key proxy.crt trivy_adapter.csr
core.key job_service.crt proxy.csr trivy_adapter.key
harbor_db.crt job_service.csr proxy.key
Also image goharbor/prepare:v2.0
does not exist. I ran command: docker run -v /:/hostfs goharbor/prepare:v2.0.0 gencert -p /opt/harbor/tls
the goharbor/prepare:v2.0.0
is uploaded in docker-hub, if you can access the internet, docker will download it automatically. And this image also included in your offline tar file. If you can not access the internet, you need to load these images first via command docker load -i ./harbor*.tar.gz
the
goharbor/prepare:v2.0.0
is uploaded in docker-hub, if you can access the internet, docker will download it automatically. And this image also included in your offline tar file. If you can not access the internet, you need to load these images first via commanddocker load -i ./harbor*.tar.gz
In documentation there is command with docker image goharbor/prepare:v2.0
. That results in Error response from daemon: manifest for goharbor/prepare:v2.0 not found: manifest unknown: manifest unknown
.
I ran command with docker image goharbor/prepare:v2.0.0
that exited correctly and generated certificates. After that I ran command ./install.sh --with-notary --with-clair
that also exited correctly. But then i got error from container logs.
Also i used offline installer.
Could solve it in my case. The container mounted directory /harbor_cust_cert/
only listed an empty core.crt file which seem to get generated on each docker-compose up.
I've simply copied the harbor_internal_ca.crt
file (generated via gencert) to the hard-coded directory make/common/config/shared/trust-certificates
and redeployed containers.
Likely something on my end as I run Harbor on aarch64. As this does not work out-of-the-box I use my own script to build related Harbor images, build scanner binaries, alter some files and do some other stuff.
@kuburoman,
as your proxy.log shows the same entries, could you check the content of make/common/config/shared/trust-certificates
?
@ninjadq any update?
I hit the exact same issues.
Documentation clearly says goharbor/prepare:v2.0
but it should say goharbor/prepare:v2.0.0
(note the extra .0 at the end)
Evidence: https://hub.docker.com/r/goharbor/prepare/tags?page=1&name=2.0
Also had the same where harbor_internal_ca.crt
was missing from ./common/config/shared/trust-certificates
. Simple copy from the internal_tls
location and redeploy fixed it on my end as well. This should probably be done as part of install.sh
Another thing I'd like to point out is that a first time install with internal_tls
is logically impossible if you follow the instructions: You have to run goharbor/prepare
to get certs before installing, but you have to run the installer to load the image for prepare (see the issue here?). Its easy enough to modify install.sh
to work around this, but it this should probably just be handled in install.sh
for the user. (the command is documented why not just run it for the user?).
Maybe internal TLS should be a --with-internal-tls
flag, rather than a "edit the compose file" setting.
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
@ninjadq Any update on this?
I believe this issue still exists in harbor v2.2.0. After enabling internal_tls, and running install.sh, the nginx containers enters a restart loop due to the missing CA at common/config/shared/trust-certificates. Copying the file over solves this problem.
same as v2.3.0
, Couldn't they just correct the bug so everyone don't need to search the solution from a closed issues?
I believe this issue still exists in harbor v2.2.0. After enabling internal_tls, and running install.sh, the nginx containers enters a restart loop due to the missing CA at common/config/shared/trust-certificates. Copying the file over solves this problem.
same as v2.8.0, Couldn't they just correct the bug so everyone don't need to search the solution from a closed issues?
same as v2.8.0, Couldn't they just correct the bug so everyone don't need to search the solution from a closed issues?
Yeah, it's pretty stupid.
Still same problem
This issue is being marked stale due to a period of inactivity. If this issue is still relevant, please comment or remove the stale label. Otherwise, this issue will close in 30 days.
v2.9 still
Not sure if related (I was trying with Podman) but I even get a crash during install:
./prepare
prepare base dir is set to /home/podman/containers/local/harbor-online-installer-v2.10.2/harbor
Clearing the configuration file: /config/jobservice/env
Clearing the configuration file: /config/jobservice/config.yml
Clearing the configuration file: /config/log/rsyslog_docker.conf
Clearing the configuration file: /config/log/logrotate.conf
Clearing the configuration file: /config/nginx/nginx.conf
Clearing the configuration file: /config/registryctl/config.yml
Clearing the configuration file: /config/registryctl/env
Clearing the configuration file: /config/db/env
Clearing the configuration file: /config/portal/nginx.conf
Clearing the configuration file: /config/core/env
Clearing the configuration file: /config/core/app.conf
Clearing the configuration file: /config/registry/config.yml
Clearing the configuration file: /config/registry/passwd
Generated configuration file: /config/portal/nginx.conf
Generated configuration file: /config/log/logrotate.conf
Generated configuration file: /config/log/rsyslog_docker.conf
Generated configuration file: /config/nginx/nginx.conf
Generated configuration file: /config/core/env
Generated configuration file: /config/core/app.conf
Generated configuration file: /config/registry/config.yml
Generated configuration file: /config/registryctl/env
Generated configuration file: /config/registryctl/config.yml
Generated configuration file: /config/db/env
Generated configuration file: /config/jobservice/env
Generated configuration file: /config/jobservice/config.yml
Traceback (most recent call last):
File "/usr/src/app/main.py", line 15, in <module>
cli()
File "/usr/lib/python3.11/site-packages/click/core.py", line 1137, in __call__
return self.main(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/click/core.py", line 1062, in main
rv = self.invoke(ctx)
^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/click/core.py", line 1668, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/click/core.py", line 763, in invoke
return __callback(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/app/commands/prepare.py", line 50, in prepare
prepare_tls(config_dict)
File "/usr/src/app/utils/internal_tls.py", line 4, in prepare_tls
config_dict['internal_tls'].validate()
File "/usr/src/app/models.py", line 102, in validate
self._check(filename)
File "/usr/src/app/models.py", line 84, in _check
if not owner_can_read(path.stat().st_mode):
^^^^^^^^^^^
File "/usr/lib/python3.11/pathlib.py", line 1013, in stat
return os.stat(self, follow_symlinks=follow_symlinks)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/data/secret/tls/portal.crt'
This issue is being marked stale due to a period of inactivity. If this issue is still relevant, please comment or remove the stale label. Otherwise, this issue will close in 30 days.
This issue was closed because it has been stalled for 30 days with no activity. If this issue is still relevant, please re-open a new issue.
As others have said, installing it with internal_tls enabled: true doesn't work at all using the install.sh.
-p path should be the same as the one under interal_tls: dir: /wherever/you/want/certs 1) docker run -v /:/hostfs goharbor/prepare:v2.10.3 gencert -p /wherever/you/want/certs 2) sudo cp /wherever/you/want/certs/* /wherever_harbor_is/common/config/shared/trust-certificates 3) sudo ./install.sh --with-trivy
This works. So either the internal_tls dir: doesn't do what it's supposed to, or something somewhere is screwed up. But you have to copy the certs to the common/config/shared/trust-certificates folder so that harbor will run.
This issue is being marked stale due to a period of inactivity. If this issue is still relevant, please comment or remove the stale label. Otherwise, this issue will close in 30 days.
As others have said, installing it with internal_tls enabled: true doesn't work at all using the install.sh.
-p path should be the same as the one under interal_tls: dir: /wherever/you/want/certs
- docker run -v /:/hostfs goharbor/prepare:v2.10.3 gencert -p /wherever/you/want/certs
- sudo cp /wherever/you/want/certs/* /wherever_harbor_is/common/config/shared/trust-certificates
- sudo ./install.sh --with-trivy
This works. So either the internal_tls dir: doesn't do what it's supposed to, or something somewhere is screwed up. But you have to copy the certs to the common/config/shared/trust-certificates folder so that harbor will run.
great insights, hope it will be implemented
This still exists in 2.11.1. Thanks for the fix
Expected behavior and actual behavior: Expected is that Harbor started and is running properly Actual is that nginx-photon is constantly restarted and rest of pod reports errors.
Steps to reproduce the problem: Followed all the steps of tutorial to deploy Harbor with internal tls enabled. In steps there is written to run command
docker run -v /:/hostfs goharbor/prepare:v2.0 gencert -p /path/to/internal/tls/cert
but version should bev2.0.0
. Installation does not show any errors.Versions: Please specify the versions of following systems.
Additional context:
Log files:
proxy.log
core.log