Register metadata failed by manual code in Spring Cloud Consul 2020

[HaojunRen]

If I set metadata by code as follows:

        applicationContext.getBeanFactory().addBeanPostProcessor(new InstantiationAwareBeanPostProcessorAdapter() {
            @Override
            public Object postProcessAfterInitialization(Object bean, String beanName) throws BeansException {
                if (bean instanceof ConsulDiscoveryProperties) {
                    ConsulDiscoveryProperties consulDiscoveryProperties = (ConsulDiscoveryProperties) bean;
                    Map<String, String> metadata = consulDiscoveryProperties.getMetadata();
            metadata.put("spring.application.type", "service");
                    ...
                }
            }
        });

The exception will throw out, is it an issue?

discovery 2021-01-06 15:06:24,935 INFO [main] o.s.c.c.s.ConsulServiceRegistry [ConsulServiceRegistry.java:65] - Registering service with consul: NewService{id='discovery-springcloud-example-a-1100', name='discovery-springcloud-example-a', tags=[], address='172.27.208.1', meta={group=example-service-group, version=1.0, region=dev, env=env1, zone=zone1, spring.boot.version=2.4.1, spring.application.name=discovery-springcloud-example-a, spring.application.type=service, spring.application.uuid=1ed4354e-e4be-4b82-93fe-5fe47dbd3f18, spring.application.discovery.plugin=Consul, spring.application.discovery.version=7.0.0-SNAPSHOT, spring.application.discovery.agent.version=1.0.0, spring.application.register.control.enabled=true, spring.application.discovery.control.enabled=true, spring.application.config.rest.control.enabled=true, spring.application.group.key=group, spring.application.context-path=/, secure=false}, port=1100, enableTagOverride=null, check=Check{script='null', dockerContainerID='null', shell='null', interval='10s', ttl='null', http='http://172.27.208.1:5100/actuator/health', method='null', header={}, tcp='null', timeout='null', deregisterCriticalServiceAfter='null', tlsSkipVerify=null, status='null', grpc='null', grpcUseTLS=null}, checks=null}
discovery 2021-01-06 15:06:24,981 ERROR [main] o.s.c.c.s.ConsulServiceRegistry [ConsulServiceRegistry.java:76] - Error registering service with consul: NewService{id='discovery-springcloud-example-a-1100', name='discovery-springcloud-example-a', tags=[], address='172.27.208.1', meta={group=example-service-group, version=1.0, region=dev, env=env1, zone=zone1, spring.boot.version=2.4.1, spring.application.name=discovery-springcloud-example-a, spring.application.type=service, spring.application.uuid=1ed4354e-e4be-4b82-93fe-5fe47dbd3f18, spring.application.discovery.plugin=Consul, spring.application.discovery.version=7.0.0-SNAPSHOT, spring.application.discovery.agent.version=1.0.0, spring.application.register.control.enabled=true, spring.application.discovery.control.enabled=true, spring.application.config.rest.control.enabled=true, spring.application.group.key=group, spring.application.context-path=/, secure=false}, port=1100, enableTagOverride=null, check=Check{script='null', dockerContainerID='null', shell='null', interval='10s', ttl='null', http='http://172.27.208.1:5100/actuator/health', method='null', header={}, tcp='null', timeout='null', deregisterCriticalServiceAfter='null', tlsSkipVerify=null, status='null', grpc='null', grpcUseTLS=null}, checks=null}
com.ecwid.consul.v1.OperationException: OperationException(statusCode=400, statusMessage='Bad Request', statusContent='Invalid Service Meta: Couldn't load metadata pair ('spring.application.type', 'service'): Key contains invalid characters')
    at com.ecwid.consul.v1.agent.AgentConsulClient.agentServiceRegister(AgentConsulClient.java:278)
    at com.ecwid.consul.v1.ConsulClient.agentServiceRegister(ConsulClient.java:310)
    at org.springframework.cloud.consul.serviceregistry.ConsulServiceRegistry.register(ConsulServiceRegistry.java:67)
    at com.nepxion.discovery.plugin.registercenter.consul.decorator.ConsulServiceRegistryDecorator.register(ConsulServiceRegistryDecorator.java:48)
    at org.springframework.cloud.consul.serviceregistry.ConsulServiceRegistry.register(ConsulServiceRegistry.java:43)
    at org.springframework.cloud.client.serviceregistry.AbstractAutoServiceRegistration.register(AbstractAutoServiceRegistration.java:232)
    at org.springframework.cloud.consul.serviceregistry.ConsulAutoServiceRegistration.register(ConsulAutoServiceRegistration.java:80)
    at org.springframework.cloud.client.serviceregistry.AbstractAutoServiceRegistration.start(AbstractAutoServiceRegistration.java:133)
    at org.springframework.cloud.consul.serviceregistry.ConsulAutoServiceRegistration.start(ConsulAutoServiceRegistration.java:70)
    at org.springframework.cloud.consul.serviceregistry.ConsulAutoServiceRegistrationListener.onApplicationEvent(ConsulAutoServiceRegistrationListener.java:60)
    at org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:172)
    at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:165)
    at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:139)
    at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:426)
    at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:383)
    at org.springframework.boot.web.servlet.context.WebServerStartStopLifecycle.start(WebServerStartStopLifecycle.java:46)
    at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:178)
    at org.springframework.context.support.DefaultLifecycleProcessor.access$200(DefaultLifecycleProcessor.java:54)
    at org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.start(DefaultLifecycleProcessor.java:356)
    at java.lang.Iterable.forEach(Iterable.java:75)
    at org.springframework.context.support.DefaultLifecycleProcessor.startBeans(DefaultLifecycleProcessor.java:155)
    at org.springframework.context.support.DefaultLifecycleProcessor.onRefresh(DefaultLifecycleProcessor.java:123)
    at org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:940)
    at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:591)
    at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:144)
    at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:767)
    at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:759)
    at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:426)
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:326)
    at org.springframework.boot.builder.SpringApplicationBuilder.run(SpringApplicationBuilder.java:144)
    at com.nepxion.discovery.plugin.example.service.DiscoveryApplicationA1.main(DiscoveryApplicationA1.java:33)

If that still use tags, works fine

List<String> tags = consulDiscoveryProperties.getTags();
tags.add("spring.application.type=service");
...

[HaojunRen]

The key is not allowed to contain "."

[HaojunRen]

Sorry,it is Consul limitation, but if we use MetaData key which contains dot(.), Spring Cloud can help me to resolve this issue? Thanks

[HaojunRen]

@spencergibb this issue difficult to us to resolve, it seems we must replace all "." to "-".

My suggestion is that Spring Cloud 2020 merges tags and current metadatas to final metadata. Example there are two configs spring.cloud.discovery.tag=a.b=1 spring.cloud.discovery.metadata.c=2

the special character metadata can be SET with tag, so the final MetaData Map is a.b=1, c=2.

if tag and metadata configs have the same key, use metadata config

BTE, refer to Consul issus links: https://github.com/hashicorp/consul/issues/8127 https://github.com/hashicorp/consul/issues/4422 https://github.com/codecentric/spring-boot-admin/issues/1456

[spencergibb]

Are your properties set by Spring Cloud consul or some other means?

[HaojunRen]

Yeah, it seems Consul sidecar does not want to change anything. To our business, MetaData keys are related with our many important outside systems, changing those keys are large workaround to us. so would you please do that enhancement in next 2020 version? That is Spring Cloud fan hope, Many many thanks!!

[HaojunRen]

Sorry, if i discribe not so cleared, i will try it again users can config tags as before like that spring.cloud.discovery.tag=a.b=1,x.y=2 so MetaData are {"a.b=1","x.y=2"}

if he add new format metadata like that spring.cloud.discovery.metadata.c=3

so the final metada is {"a.b=1","x.y=2","c=3"}

if he SET spring.cloud.discovery.tag=a.b=1,x.y=2,c=123 spring.cloud.discovery.metadata.c=3

the final metada still is {"a.b=1","x.y=2","c=3"}

[spencergibb]

What feature that you want to use depends on metadata?

[HaojunRen]

User has special character to config with tag, no special character to config with new MetaData fomat. Two operations have same result.

What feature that you want to use depends on metadata?

for example, using MetaData to do gray relase or blue-green release, or store customs info, like telephone, email etc, like spring.cloud.discovery.tag=service.version,service.owner.name=Tom,service.owner.email=xyz@google.com

[spencergibb]

If you just want access to tags via service instance and registration that is already available.

[HaojunRen]

if i config it as tags, getMetadata method return empty in service instance and registration. is that a bug? And those data can be found in getTags method

My Idea is getMetadata can get tags data

[spencergibb]

It is not a bug. Why do you need it in metadata?

[HaojunRen]

because in hoxton, we config those with tags, and can get those via getMetadata

[HaojunRen]

in 2020, we can accept to change those with metadata config, but our MetaData key have ".", Consul Server Side refuses to those. So I have no idea...

[spencergibb]

So you could change to use getTags() rather than metadata rather than migrating to use metadata

[HaojunRen]

According to your suggestion, here we have 2 register middlewares : Eureka and Consul. We provides a common module, it has follows logic:

Map<String, String> metadata = instance.getMetadata();

For Eureka it works

But for Consul it doesn't works if using getTags, because getTags is not a spring cloud common method

it should change to

ConsulServiceInstance consulServiceInstance = (ConsulServiceInstance) instance;
List<String> tags = consulServiceInstance.getTags();
Map<String, String> metadata = parseToMetadata(tags);

So the common module will not be so generic, we must change a lot of...

[HaojunRen]

Some business system maybe not use that common module, and I don't how much places shoud do following change

ConsulServiceInstance consulServiceInstance = (ConsulServiceInstance) instance;
List<String> tags = consulServiceInstance.getTags();
Map<String, String> metadata = parseToMetadata(tags);

[HaojunRen]

Would you please add a flag property to enable/disable merging tags to metadata?default value is false, so that use can control that logic by himself

[spencergibb]

https://github.com/spring-cloud/spring-cloud-consul/issues/699