manual kafka service add fails with protocol check

[HeartSaVioR]

WARN [22:27:48.346] [dw-33 - POST /api/v1/catalog/clusters/2/services/register/KAFKA] c.h.s.s.c.s.EnvironmentService - Protocol/Port information [SASL_PLAINTEXT://host1:port1,PLAINTEXT://host2:port2,SSL://host3:port3,SASL_SSL://host4:port4] for component KAFKABROKER doesn't seem to known format [([a-zA-Z]+)://[a-zA-Z0-9-\.]*:([0-9]+)]. skipping assignment...

Masked actual hosts and ports. It fails even with a single protocol.

[HeartSaVioR]

This is a bit complicated because of Ambari import. Ambari provides <protocol>://localhost:<port> so that was fairly easy, but not similar as above configuration.

[HeartSaVioR]

Thought about manual registration more, and I also found what I was missing: configuration file doesn't have default value hence and we can't get the default value anyway.

We need to define required fields for service and let user also input on registering. Since we add resource file for Hadoop / HBase / Hive in topology, we may not need to define required fields for those services, but need to define for others.

[HeartSaVioR]

Another issue is current implementation of Component. It contains host information as hosts, port, protocol, which can't handle above case. (different protocol and/or different port)

There're two ways of handle this case:

• allow multiple Components for same component (for example, NIMBUS)
  • hosts could be changed to host
• introduce a new entity for storing ComponentHost
  • move hosts, port, protocol to this entity
  • hosts could be changed to host

[HeartSaVioR]

Storm

• required
  • nimbus.seeds / nimbus.port => Nimbus component
  • ui.host / ui.port => UI component
• optional
  • nimbus.thrift.max_buffer_size
  • storm.thrift.transport
  • storm.principal.tolocal

Kafka

• required
  • zookeeper.connect
  • listeners -> Broker component
• optional
  • security.inter.broker.protocol

Druid

• required
  • druid.zk.service.host
• optional
  • druid.selectors.indexing.serviceName
  • druid.discovery.curator.path

[HeartSaVioR]

785

[HeartSaVioR]

I think #792 (which addresses #785) also covers this.

[HeartSaVioR]

Relevant issue addressed this. Closing.