search

vmware / container-service-extension

Container Service for VMware vCloud Director
https://vmware.github.io/container-service-extension
Other
78 stars 52 forks source link

Config file yaml issue #311

Open ManivelRR opened 5 years ago

ManivelRR commented 5 years ago

Hi All,

Im new to CSE K8s.While installating CSE on photon OS,im getting the below error.Please suggest.

root@photon-machine [ ~ ]# cse version CSE, Container Service Extension for VMware vCloud Director, version 1.2.7 root@photon-machine [ ~ ]#

root@photon-machine [ ~ ]# cse install --config config.yaml

Traceback (most recent call last):

File "/usr/bin/cse", line 10, in 

sys.exit(cli())

File "/usr/lib/python3.6/site-packages/click/core.py", line 764, in call

return self.main(*args, **kwargs)

File "/usr/lib/python3.6/site-packages/click/core.py", line 717, in main

rv = self.invoke(ctx)

File "/usr/lib/python3.6/site-packages/click/core.py", line 1137, in invoke

return _process_result(sub_ctx.command.invoke(sub_ctx))

File "/usr/lib/python3.6/site-packages/click/core.py", line 956, in invoke

return ctx.invoke(self.callback, **ctx.params)

File "/usr/lib/python3.6/site-packages/click/core.py", line 555, in invoke

return callback(*args, **kwargs)

File "/usr/lib/python3.6/site-packages/click/decorators.py", line 17, in new_func

return f(get_current_context(), *args, **kwargs)

File "/usr/lib/python3.6/site-packages/container_service_extension/cse.py", line 239, in install

ext_install=ext_install)

File "/usr/lib/python3.6/site-packages/container_service_extension/configure_cse.py", line 644, in install_cse

config = get_validated_config(config_file_name)

File "/usr/lib/python3.6/site-packages/container_service_extension/configure_cse.py", line 328, in get_validated_config

config = yaml.safe_load(config_file)

File "/usr/lib/python3.6/site-packages/yaml/init.py", line 162, in safe_load

return load(stream, SafeLoader)

File "/usr/lib/python3.6/site-packages/yaml/init.py", line 114, in load

return loader.get_single_data()

File "/usr/lib/python3.6/site-packages/yaml/constructor.py", line 41, in get_single_data

node = self.get_single_node()

File "/usr/lib/python3.6/site-packages/yaml/composer.py", line 36, in get_single_node

document = self.compose_document()

File "/usr/lib/python3.6/site-packages/yaml/composer.py", line 55, in compose_document

node = self.compose_node(None, None)

File "/usr/lib/python3.6/site-packages/yaml/composer.py", line 84, in compose_node

node = self.compose_mapping_node(anchor)

File "/usr/lib/python3.6/site-packages/yaml/composer.py", line 127, in compose_mapping_node

while not self.check_event(MappingEndEvent):

File "/usr/lib/python3.6/site-packages/yaml/parser.py", line 98, in check_event

self.current_event = self.state()

File "/usr/lib/python3.6/site-packages/yaml/parser.py", line 439, in parse_block_mapping_key

"expected <block end>, but found %r" % token.id, token.start_mark)

yaml.parser.ParserError: while parsing a block mapping

in "config.yaml", line 1, column 1

expected , but found '-'

in "config.yaml", line 87, column 1

root@photon-machine [ ~ ]#

andrew-ni commented 5 years ago

Seems like the error is that config.yaml is not valid yaml. Can you verify that your config file is fine (maybe with a yaml parser or cse check --config config.yaml). If the problem persists, can you attach your config file (redacted) here?

ManivelRR commented 5 years ago

Seems like the error is that config.yaml is not valid yaml. Can you verify that your config file is fine (maybe with a yaml parser or cse check --config config.yaml). If the problem persists, can you attach your config file (redacted) here?

Hi Andrew,

root@photon-machine [ ~ ]# cse check --config config.yaml Traceback (most recent call last): File "/usr/bin/cse", line 10, in sys.exit(cli()) File "/usr/lib/python3.6/site-packages/click/core.py", line 764, in call return self.main(*args, kwargs) File "/usr/lib/python3.6/site-packages/click/core.py", line 716, in main with self.make_context(prog_name, args, extra) as ctx: File "/usr/lib/python3.6/site-packages/click/core.py", line 641, in make_conte xt self.parse_args(ctx, args) File "/usr/lib/python3.6/site-packages/click/core.py", line 1089, in parse_arg s rest = Command.parse_args(self, ctx, args) File "/usr/lib/python3.6/site-packages/click/core.py", line 940, in parse_args value, args = param.handle_parse_result(ctx, opts, args) File "/usr/lib/python3.6/site-packages/click/core.py", line 1476, in handle_pa rse_result value = invoke_param_callback( NameError: name 'invoke_param_callback' is not defined root@photon-machine [ ~ ]#

My YAML config file

amqp: exchange: cse-ext host: 10.128.7.36 password: Password@1234 port: 5672 prefix: vcd routing_key: cse ssl: false ssl_accept_all: false username: uladmin vhost: /

vcd: api_version: '29.0' host: 10.128.7.18 log: true password: Password@1234 port: 443 username: administrator verify: false

vcs:

service: enforce_authorization: false listeners: 5

broker: catalog: cse cse_msg_dir: /tmp/cse default_template: photon-v2 ip_allocation_mode: pool network: ManivelDV org: CSE storage_profile: '*' templates:

when i check my YAML file with http://www.yamllint.com/, it says it is valid.

Thanks, Manivel R

andrew-ni commented 5 years ago

Hi, can you provide a pip list/freeze (pip list or pip freeze)

Also, can you edit your previous post to include the config.yaml in a code block (surround the text content with backticks and 'yaml like this:

```yaml my config content pasted ```

ManivelRR commented 5 years ago

Hi Andrew,

Thanks for the reply.Please find the details of pip list,pip freeze and config.yaml file.

Thanks, Manivel R CSE.txt

andrew-ni commented 5 years ago

I used http://www.yamllint.com/ and copy/pasted your config file, and it seems that the config file is invalid yaml. Unsure if the file was actually starting and ending with ---, but if it was then that would have caused the error as well. The key 'mem' was indented out by one level. Make sure your config file looks exactly like this, or pass it through the yaml validator to double check for correctness. (Renamed issue from Kubernetes || CSE installation issue to Config file yaml issue)

If you're still having issues, let us know!

amqp:
  exchange: cse-ext
  host: 10.128.7.36
  password: Password@1234
  port: 5672
  prefix: vcd
  routing_key: cse
  ssl: false
  ssl_accept_all: false
  username: uladmin
  vhost: /

vcd:
  api_version: '29.0'
  host: 10.128.7.18
  log: true
  password: Password@1234
  port: 443
  username: administrator
  verify: false

vcs:
- name:  10.128.7.21
  password: Password@1234
  username: cseadmin@vsphere.local
  verify: false

service:
  enforce_authorization: false
  listeners: 5

broker:
  catalog: cse
  cse_msg_dir: /tmp/cse
  default_template: photon-v2
  ip_allocation_mode: pool
  network: ManivelDV
  org: CSE
  storage_profile: '*'
  templates:
  - admin_password: Password@1234
    catalog_item: photon-custom-hw11-2.0-304b817-k8s
    cleanup: true
    cpu: 2
    description: 'PhotonOS v2

      Docker 17.06.0-4

      Kubernetes 1.9.1

      weave 2.3.0'
    mem: 2048
    name: photon-v2
    sha256_ova: cb51e4b6d899c3588f961e73282709a0d054bb421787e140a1d80c24d4fd89e1
    source_ova: http://dl.bintray.com/vmware/photon/2.0/GA/ova/photon-custom-hw11-2.0-304b817.ova
    source_ova_name: photon-custom-hw11-2.0-304b817.ova
    temp_vapp: photon2-temp
  - admin_password: Password@1234
    catalog_item: ubuntu-16.04-server-cloudimg-amd64-k8s
    cleanup: true
    cpu: 2
    description: 'Ubuntu 16.04

      Docker 18.03.0~ce

      Kubernetes 1.10.1

      weave 2.3.0'
    mem: 2048
    name: ubuntu-16.04
    sha256_ova: 3c1bec8e2770af5b9b0462e20b7b24633666feedff43c099a6fb1330fcc869a9
    source_ova: https://cloud-images.ubuntu.com/releases/xenial/release-20180418/ubuntu-16.04-server-cloudimg-amd64.ova
    source_ova_name: ubuntu-16.04-server-cloudimg-amd64.ova
    temp_vapp: ubuntu1604-temp
  type: default
  vdc: CSE
ManivelRR commented 5 years ago

Hi Andrew,

It has been corrected and checked with yamllint website.It is showing as valid.However in CSE command line,it is still showing as invalid.

image

root@kubernetes [ ~ ]# cse check Validating config file 'config.yaml' Config file 'config.yaml' is invalid root@kubernetes [ ~ ]#

Thanks, Manivel R

andrew-ni commented 5 years ago

Can you please remove the dashes at the top and bottom of the config file.

yaml files are simply keys and their associated values. Those dashes at the top and bottom of your file should not exist in a yaml file and therefore are throwing off the python's yaml parser

ManivelRR commented 5 years ago

Thanks for your feedback.

Dashes are not there at the top and bottom.We removed already. We are using photon OS 2.0 currently.

Please let us know if there is any other supported OS with version for this CSE installation(like Centos or Ubuntu).I will try it over there and let you know the status.

Thanks, Manivel R

andrew-ni commented 5 years ago

CSE will run on any unix-based OS that is running Python 3.6.5+

It seems like the error you're experiencing is still somehow related to the config file. This is what a successful cse check looks like:

Validating config file '201.166.yaml'
Connected to AMQP server (bos1-vcd-sp-static-201-163.eng.vmware.com:5672)
InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised.
Connected to vCloud Director (IP)
Connected to vCenter Server 'vc1' as 'email' (host url)
Config file '201.166.yaml' is valid

Can you give us some more context on your python environment? The pip list you sent was for your system's default python (Python 2), whereas we need the information for the Python version that your CSE is using. Try pip3 list or pip3.6 list. Also, can you explain how you installed CSE on your system? (Should be something similar to running pip3 install container-service-extension)

Vijendrasi commented 5 years ago

I am also facing same issue with Photon OS 3.0, VCD version 9.5 and CSE version 1.2.7. Any idea how to fix it ? Or is there a bug in 1.2.7 version ? I have used CSE 1.2.5 last time and it works perfectly.

bennetms commented 2 years ago

Hi , we are trying to get CSE running in our environment. And trying to find out what the YAML settings mean.

amqp: ##Is this the mosquito MQTT, and is the exchange cse-ext the org vcd for this file? exchange: cse-ext host: 10.128.7.36 password: Password@1234 port: 5672 prefix: vcd ## Does this refer to the vcloud director prefix routing_key: cse ## does this route messages to the Container services service, and is ##the container service , seen as CSE by the messaging bus. ssl: false ssl_acceptall: false ## Does these two steps disable secure SSL (HTTPS) for transit data username: uladmin vhost: / ## why is the vhost needed , when there is already a host._

vcd: ## This is the VCloud Director api_version: '29.0' host: 10.128.7.18 log: true password: Password@1234 port: 443 username: administrator verify: false

vcs: ## Is this the record that tells the CSE to bind to a certain address and use a username for the CSE service?

  • name: 10.128.7.21 password: Password@1234 username: cseadmin@vsphere.local verify: false

service: ## Is this to tell CSE to not required authorized logins and to listen with 5 sockets? enforce_authorization: false listeners: 5

broker: ## Is this the CSE service, that is sent two scripts for two container types that will be hosted? catalog: cse ## Is this the VCloud catalog we are inserting these two definitions in? cse_msg_dir: /tmp/cse default_template: photon-v2 ip_allocation_mode: pool network: ManivelDV org: CSE storage_profile: '*' templates:

  • admin_password: Password@1234 catalog_item: photon-custom-hw11-2.0-304b817-k8s cleanup: true cpu: 2 description: 'PhotonOS v2

    Docker 17.06.0-4

    Kubernetes 1.9.1

    weave 2.3.0' mem: 2048 name: photon-v2 sha256_ova: cb51e4b6d899c3588f961e73282709a0d054bb421787e140a1d80c24d4fd89e1 source_ova: http://dl.bintray.com/vmware/photon/2.0/GA/ova/photon-custom-hw11-2.0-304b817.ova source_ova_name: photon-custom-hw11-2.0-304b817.ova temp_vapp: photon2-temp

  • admin_password: Password@1234 catalog_item: ubuntu-16.04-server-cloudimg-amd64-k8s cleanup: true cpu: 2 description: 'Ubuntu 16.04

    Docker 18.03.0~ce

    Kubernetes 1.10.1

    weave 2.3.0' mem: 2048 name: ubuntu-16.04 sha256_ova: 3c1bec8e2770af5b9b0462e20b7b24633666feedff43c099a6fb1330fcc869a9 source_ova: https://cloud-images.ubuntu.com/releases/xenial/release-20180418/ubuntu-16.04-server-cloudimg-amd64.ova source_ova_name: ubuntu-16.04-server-cloudimg-amd64.ova temp_vapp: ubuntu1604-temp type: default vdc: CSE

                                                                                                        _## Thank you._