using consul discovery

[winguse]

I got quite confused about how to use consul discovery.

Using Consul Agent to register the management service

I was first to register the management service by consul agent client, something like this:

        clientAgent.register(
          /* port = */ 8558,
          /* tcp = */ HostAndPort.fromParts("127.0.0.1", 8558),
          /* interval = */ 10 seconds,
          /* name = */ "my-service-name",
          /* id = */ "my-service-id",
          /* tags = */ List("system:my-service", "akka-management-port:8558").asJava,
          /* meta = */ Map.empty.asJava
        )

But as debugging, I find the address discovery-consul get is Empty. So, it failed to get correct HTTP REST connection.

And according to consul document:

ServiceAddress is the IP address of the service host — if empty, node address should be used

So I think this might be a bug.

Using Consul Catalog to register the management service

As I find it's using catalog agent to get catalog info, and I also check the test code. I try to switch to using catalog client:

            catalogClient
              .register(
                ImmutableCatalogRegistration
                  .builder()
                  .service(
                    ImmutableService
                      .builder()
                      .addTags("system:my-service", "akka-management-port:8558")
                      .address(address)
                      .id("my-service-id")
                      .service("my-service")
                      .port(8558)
                      .build()
                  )
                  .node(nodeName)
                  .address(address)
                  .build()
              )

but i find by using this way, the registered service will deregister after a while (I don't know why).

I read some document about consul, if I understand correctly, the recommended way is using agent instead of catalog. So, why we are using catalog here?

[TimMoore]

@mlosiewicz-pl, since you contributed the Consul discovery implementation, are you able to comment on this? Thanks!

[winguse]

update:

Regarding consul doc:

During this synchronization, the catalog is also checked for correctness. If any services or checks exist in the catalog that the agent is not aware of, they will be automatically removed to make the catalog reflect the proper set of services and health information for that agent. Consul treats the state of the agent as authoritative; if there are any differences between the agent and catalog view, the agent-local view will always be used.

It explains why the service being registered by catalog api will be missing after a while. If my understanding is correct, we should use catalog for registering the management service.

[winguse]

And for self join https://github.com/akka/akka-management/blob/3ed10d7c4f81bcaaf38f1ed9f380c82e5b856fe6/cluster-bootstrap/src/main/scala/akka/management/cluster/bootstrap/SelfAwareJoinDecider.scala#L68

It's doing plain text compare which may lead to problem:

Config akka.management.http.hostname can be a string of:

• 0.0.0.0
• 172.30.0.1 (one of the IP of either)
• node-name.some-domain (domain name, machine name)

And the value from consul can be:

• 172.30.0.1 (one of the IP of either)
• node-name.some-domain (domain name, machine name, when reading from service address)

I find we need to correctly config these two to get it work.