vmware / harbor-boshrelease

CFF BOSH Release for Project Harbor
Apache License 2.0
36 stars 24 forks source link

Getting error while deploying the latest Bosh release and see below #29

Closed goprasad closed 6 years ago

goprasad commented 6 years ago

21:24:43 | Updating instance harbor-app: harbor-app/54ef00a2-4f54-49e2-9e61-77bbd0940e44 (0) (canary) (00:05:07) L Error: Action Failed get_task: Task 2b5c7659-8861-4f55-4fff-6886ae81fda6 result: 2 of 2 pre-start scripts failed. Failed Jobs: enable-bosh-dns, harbor.

21:29:50 | Error: Action Failed get_task: Task 2b5c7659-8861-4f55-4fff-6886ae81fda6 result: 2 of 2 pre-start scripts failed. Failed Jobs: enable-bosh-dns, harbor.

Started Tue Jul 24 21:15:24 UTC 2018 Finished Tue Jul 24 21:29:50 UTC 2018 Duration 00:14:26

Task 35 error

Updating deployment: Expected task '35' to succeed but state is 'error'

Exit code 1

goprasad commented 6 years ago

Appreciate your help in this regard and here more information on this error.

harbor.yml.txt and providing you the request info below.

Stemcell version: bosh-vsphere-esxi-ubuntu-trusty-go_agent 3468.54* ubuntu-trusty

Release version: harbor-container-registry 1.5.1-build.7* non-git

logs: logs are packaged from this folder '/var/vcap/sys/log' and attached the logs.tar.gz manifest used: harbor.yml.txt file attached

logs.tar.gz harbor.yml.txt

jessehu commented 6 years ago

@stonezdj to take a look please

jessehu commented 6 years ago

@goprasad cloud you please ssh to the harbor VM and check the error log in /var/vcap/sys/log/harbor/pre-start.err and pre-start.log ?

goprasad commented 6 years ago

https://github.com/vmware/harbor-boshrelease/files/2232507/logs.tar.gz /var/vcap/jobs/harbor/bin/pre-start: line 156: /var/vcap/jobs/bosh-dns/bin/wait: No such file or directory the logs.tar.gz got everything that you are looking for

jessehu commented 6 years ago

@goprasad did you follow https://github.com/vmware/harbor-boshrelease/blob/master/README.md#upload-cloud-config-and-runtime-config to add the boshdns addon to bosh director: bosh -n update-runtime-config manifests/runtime-config-bosh-dns.yml --name bosh-dns

goprasad commented 6 years ago

I did not the run the command(bosh -n update-runtime-config manifests/runtime-config-bosh-dns.yml --name bosh-dns) however harbor.yml file already got the references for bosh-dns https://github.com/vmware/harbor-boshrelease/files/2232509/harbor.yml.txt

goprasad commented 6 years ago

now I get this error after I added the (bosh -n update-runtime-config manifests/runtime-config-bosh-dns.yml --name bosh-dns)

is there way that i can generate the varibles while updating the runtime-config

04:40:47 | Preparing deployment: Preparing deployment (00:00:00) L Error: Failed to fetch variable '/bosh_dns_health_server_tls' from config server: Director is not configured with a config server

04:40:47 | Error: Failed to fetch variable '/bosh_dns_health_server_tls' from config server: Director is not configured with a config server

Started Tue Jul 31 04:40:47 UTC 2018 Finished Tue Jul 31 04:40:47 UTC 2018 Duration 00:00:00

goprasad commented 6 years ago

The above mentioned error has been resolved. now I see below error

Task 314 | 19:53:14 | Preparing deployment: Preparing deployment (00:00:01) Task 314 | 19:53:15 | Preparing package compilation: Finding packages to compile (00:00:00) Task 314 | 19:53:16 | Updating instance harbor-app: harbor-app/1e6ab393-46bb-4229-84a1-80605d7d5235 (0) (canary) (00:10:35) L Error: 'harbor-app/1e6ab393-46bb-4229-84a1-80605d7d5235 (0)' is not running after update. Review logs for failed jobs: harbor Task 314 | 20:03:51 | Error: 'harbor-app/1e6ab393-46bb-4229-84a1-80605d7d5235 (0)' is not running after update. Review logs for failed jobs: harbor

Bosh Release harbor details: ame Release(s) Stemcell(s) Team(s)
harbor-deployment bosh-dns/0.1.3 bosh-vsphere-esxi-ubuntu-trusty-go_agent/3468.54 -
harbor-container-registry/1.5.2-build.8

log details: I have packaged all the logs from this folder:/var/vcap/sys/log logs.zip

jessehu commented 6 years ago

@goprasad Per "Review logs for failed jobs: harbor", you can check log files in /var/vcap/sys/log/harbor/ . ctl.stdout.log says:

[Tue Jul 31 20:11:59 UTC 2018] Harbor service is not ready. Waiting for 5 seconds then check again.
[Tue Jul 31 20:12:04 UTC 2018] Error: Harbor Service failed to start in 180 seconds.

The output of the checking is in cron.log:

curl -s --cacert /var/vcap/jobs/harbor/config/ca.crt https://harbor-oma-eng.1dc.com/api/systeminfo
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python2.7/json/__init__.py", line 290, in load
    **kw)
  File "/usr/lib/python2.7/json/__init__.py", line 338, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python2.7/json/decoder.py", line 366, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python2.7/json/decoder.py", line 384, in raw_decode
    raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded
[Tue Jul 31 20:11:59 UTC 2018] Harbor status check: [FAILED]

You can 'bosh ssh' to the harbor node and run "curl --cacert /var/vcap/jobs/harbor/config/ca.crt https://harbor-oma-eng.1dc.com/api/systeminfo" to get detail error for us to debug.

stonezdj commented 6 years ago

The Harbor's CA should be same as the bosh director CA. I have ever seen the same error in other environments.

goprasad commented 6 years ago

Thank you Guys , its working fine..I had wrong IP set instead of using DNS host name

ghost commented 6 years ago

Glad to hear it @goprasad!