Eureka First Bootstrap & retry defaulting to localhost

[MarkCorkery-Vennetics]

We've three Spring Boot applications:

• Eureka Service
• Config Server
• Simple Web Service making use of Eureka and Config Server

I've set up the services so that we use a Eureka First Bootstrap, i.e. the simple web application finds out about the config server from the eureka service.

When started separately (either locally or by starting them as individual docker images) everything is ok, i.e. start config server after discovery service is running, and the Simple web service is started once the config server is running.

When docker-compose is used to start the services, they obviously start at the same time and essentially race to get up and running. This isn't an issue as we've added failFast: true and retry values to the simple web service and also have the docker container restarting so that the simple web service will eventually restart at a time when the discovery service and config server are both running but this doesn't feel optimal.

The simple web service reattempts a number of times to connect to the discovery service. This is sensible and expected. The unexpected behaviour we noticed was the following:

• At the same time the simple web service attempts to contact the config server. Because it cannot contact the discovery service, it retries to connect to a config server on localhost, e.g. logs show retries going to http://localhost:8888.
• If you configure your retry to be sufficiently long, the simple web service will eventually successfully connect to the discovery service but the logs show it stills tries to establish communication to the config server by going to http://localhost:8888. Again, this wasn't ideal.

Three questions/observations:

• Is it a sensible strategy for the config client to fall back to trying localhost:8888 when it has been configured to use discovery to find the config server?
• When the eureka connections is established, should the retry mechanism not now switch to trying the config server endpoint as indicated by Eureka? Essentially putting in higher/longer retry intervals and periods for the config server connection is pointless in this case as it's never going to connect to it if it's looking at localhost so we're better just failing fast.
• Are there any properties that can override this behaviour? I've created a sample github repo that demonstrates this behaviour:

https://github.com/KramKroc/eurekafirstdiscovery/tree/master

[dsyer]

It worked right? I'm not entirely sure what you want to happen, but I'm also not really sure I know what did happen when it eventually connected. Is the app waiting and retrying in the wrong place?

The default URL for the config server is configured as spring.cloud.config.uri, so you can always change that if you want to, so it's something other than localhost, but I'm not sure that was really your point. Once the discovery client has an address for the config server it either switches over to that, in which case the app starts successfully and everyone is happy, or it doesn't, in which case it won't (so I didn't follow that bit of the question, since you said the app started, but you weren't happy)?

Discovery-first bootstrap is not really a mainstream feature, so it always has more bugs than the straightforward alternative (config first). I don't rule out ever fixing all the bugs, but it's never been used that much in anger, and it's really hard to write tests, so switching to config first would not be a bad idea IMO. Or if you want to use it heavily, maybe it's worth investing some time in fixing the issues and writing some tests.

[MarkCorkery-Vennetics]

Hi Dave,

thanks for getting back. It works eventually because the Docker Engine continues to restart the microservices until they reach a state where the discovery and config servers are up and processing requests.

This issue I have is highlighted in your second paragraph. When the discovery client eventually does connect successfully to the discovery service, the polling to the config server still tries localhost, rather than the discovered host. You can see this by pushing up the retry behaviour on the clients.

Would you expect the polling responsible for connect to the config server to switch from localhost to the discovered endpoint after a connection has been made to the discovery service. Basically is this a bug or is it a feature request :D

[cforce]

Can someone please publish at which delay /retry settings (upper & lower bound) this will eventually happen? I have the same problem. Actually it only happens one time directly after the start of the service, and pollutes the log somehow. Would be nice to make that clean here.

[dsyer]

It's not clear to me that it did ever happen. If you are trying anything please use snapshots though because there were known problems in Brixton.M4 (last stable).

[cforce]

Yes working on M4. Using snapshots goes to far for my use, but awaiting next Milestone or even Release. I read the Release is scheduled for Q1 this year. Do you have an estimate when next Milestrone will be released . Btw i just noticed after following starts the problems disappeared (without chnages) . Seems difficult to reproduce.

[dsantos]

Any solution for this behaviour?

I am having the same problem. If the client is starting, trying to fetch configurations from config server and the config server is not yet registered with consul, the client keeps retrying in http://localhost:8888, even if the config server is already started and registered with consul.

I am using spring-cloud-config version 1.1.0.RELEASE

[dsyer]

No, nothing happened yet. Normally people post back to github when they work on something like this. Feel free to start work on it if you have time.