search

spring-cloud / spring-cloud-netflix

Integration with Netflix OSS components
http://cloud.spring.io/spring-cloud-netflix/
Apache License 2.0
4.88k stars 2.44k forks source link

Heartbeat failed with response code 401 #291

Closed a-marcel closed 8 years ago

a-marcel commented 9 years ago

Hello

i cloned https://github.com/spring-cloud-samples/eureka.git and https://github.com/spring-cloud-samples/feign-eureka.git and compile that via maven.

i start eureka server first, after that the feign client.

after a few seconds i see in the eureka server console:

2015-04-01 11:42:52.309  INFO 18158 --- [nio-8761-exec-4] o.s.b.a.audit.listener.AuditListener     : AuditEvent [timestamp=Wed Apr 01 11:42:52 CEST 2015, principal=user, type=AUTHENTICATION_SUCCESS, data={details=org.springframework.security.web.authentication.WebAuthenticationDetails@957e: RemoteIpAddress: 127.0.0.1; SessionId: null}]
2015-04-01 11:43:02.411  INFO 18158 --- [nio-8761-exec-5] o.s.b.a.audit.listener.AuditListener     : AuditEvent [timestamp=Wed Apr 01 11:43:02 CEST 2015, principal=user, type=AUTHENTICATION_SUCCESS, data={details=org.springframework.security.web.authentication.WebAuthenticationDetails@957e: RemoteIpAddress: 127.0.0.1; SessionId: null}]
2015-04-01 11:43:02.412  WARN 18158 --- [nio-8761-exec-5] com.netflix.eureka.InstanceRegistry      : DS: Registry: lease doesn't exist, registering resource: HELLOSERVER - 127.0.0.1:HelloServer:7111
2015-04-01 11:43:02.413  WARN 18158 --- [nio-8761-exec-5] c.n.eureka.resources.InstanceResource    : Not Found (Renew): HELLOSERVER - 127.0.0.1:HelloServer:7111
2015-04-01 11:43:02.439  INFO 18158 --- [nio-8761-exec-6] o.s.b.a.audit.listener.AuditListener     : AuditEvent [timestamp=Wed Apr 01 11:43:02 CEST 2015, principal=user, type=AUTHENTICATION_SUCCESS, data={details=org.springframework.security.web.authentication.WebAuthenticationDetails@957e: RemoteIpAddress: 127.0.0.1; SessionId: null}]
2015-04-01 11:43:02.459  INFO 18158 --- [nio-8761-exec-6] com.netflix.eureka.InstanceRegistry      : Registered instance id 127.0.0.1:HelloServer:7111 with status UP
2015-04-01 11:43:02.764  INFO 18158 --- [nio-8761-exec-7] o.s.b.a.audit.listener.AuditListener     : AuditEvent [timestamp=Wed Apr 01 11:43:02 CEST 2015, principal=anonymousUser, type=AUTHORIZATION_FAILURE, data={message=Zugriff verweigert, type=org.springframework.security.access.AccessDeniedException}]
2015-04-01 11:43:02.790  WARN 18158 --- [egister-process] c.netflix.eureka.cluster.PeerEurekaNode  : The replication of HELLOSERVER/127.0.0.1:HelloServer:7111/Register failed with response code 401

This exception occours after 10 seconds again and again.

I debug just a little bit, and maybe there is a problem in one batch task of the eureka server. This one don't use the credenticals from the config.

My steps are:

git clone https://github.com/spring-cloud-samples/eureka.git
cd eureka/
mvn package
java -jar target/eureka-0.0.1-SNAPSHOT.jar
git clone https://github.com/spring-cloud-samples/feign-eureka.git
cd feign-eureka/server/
mvn package
java -jar target/feign-eureka-hello-server-0.0.1-SNAPSHOT.jar

thanks for you help

p.s: in my project, the same error happend. this is for reproducing the error.

dsyer commented 9 years ago

Is that the eureka server not being able to contact itself? Do you have a replica set up?

a-marcel commented 9 years ago

No, i use the Standalone Setup with no peers.

dsyer commented 9 years ago

So you would need to set 

eureka:
  client:
    registerWithEureka: false
    fetchRegistry: false

in the Erueka server?

a-marcel commented 9 years ago

Hello,

yes i've use this config

You can see it here: https://github.com/spring-cloud-samples/eureka/blob/master/src/main/resources/application.yml

server:
  port: 8761
security:
  user:
    password: ${eureka.password} # Don't use a default password in a real app

eureka:
  client:
    registerWithEureka: false
    fetchRegistry: false
  server:
    waitTimeInMsWhenSyncEmpty: 0
  password: ${SECURITY_USER_PASSWORD:password}

Thanks Marcel

spencergibb commented 9 years ago

Closed for lack of feedback. Please, reopen if there is something new to add.

KramKroc commented 8 years ago

@spencergibb Can you reopen this issue? I see the same issue with the latest code base (1.3.0). If you add spring security to the eureka server (even if it is standalone as documented above) and security details (username & password), this behaviour appears.

I noticed it did not happen with the https://github.com/spring-cloud-samples/eureka.git and https://github.com/spring-cloud-samples/feign-eureka.git examples so I forked them:

https://github.com/KramKroc/eureka.git https://github.com/KramKroc/feign-eureka.git

It turns out that the behaviour is present in the example when eureka.instance.metadataMap.instanceId is commented out in:

https://github.com/KramKroc/feign-eureka/blob/master/server/src/main/resources/application.yml

Not sure why so would love if you could explain why?

spencergibb commented 8 years ago

@KramKroc there is no spring cloud 1.3.0, do you mean boot? The instanceId is what uniquely identifies service instances to eureka. Not sure why that would cause an authentication failure.

KramKroc commented 8 years ago

@spencergibb yikes, yes, I did mean version 1.3.0 of Spring Boot! As I indicated, the only difference is the definition of that instanceId.

Without the definition of the instanceId

You see the 401 errors:

The Feign Server logs:

2015-12-18 17:52:52.046  INFO 16133 --- [pool-4-thread-1] com.netflix.discovery.DiscoveryClient    : DiscoveryClient_HELLOSERVER/Marks-MBP.home - Re-registering apps/HELLOSERVER
2015-12-18 17:52:52.046  INFO 16133 --- [pool-4-thread-1] com.netflix.discovery.DiscoveryClient    : DiscoveryClient_HELLOSERVER/Marks-MBP.home: registering service...
2015-12-18 17:52:52.180  INFO 16133 --- [pool-4-thread-1] com.netflix.discovery.DiscoveryClient    : DiscoveryClient_HELLOSERVER/Marks-MBP.home - registration status: 204
2015-12-18 17:53:12.011  INFO 16133 --- [pool-5-thread-1] com.netflix.discovery.DiscoveryClient    : Disable delta property : false
2015-12-18 17:53:12.012  INFO 16133 --- [pool-5-thread-1] com.netflix.discovery.DiscoveryClient    : Single vip registry refresh property : null
2015-12-18 17:53:12.012  INFO 16133 --- [pool-5-thread-1] com.netflix.discovery.DiscoveryClient    : Force full registry fetch : false
2015-12-18 17:53:12.012  INFO 16133 --- [pool-5-thread-1] com.netflix.discovery.DiscoveryClient    : Application is null : false
2015-12-18 17:53:12.012  INFO 16133 --- [pool-5-thread-1] com.netflix.discovery.DiscoveryClient    : Registered Applications size is zero : true
2015-12-18 17:53:12.012  INFO 16133 --- [pool-5-thread-1] com.netflix.discovery.DiscoveryClient    : Application version is -1: false
2015-12-18 17:53:12.017  INFO 16133 --- [pool-5-thread-1] com.netflix.discovery.DiscoveryClient    : Getting all instance registry info from the eureka server
2015-12-18 17:53:12.021  INFO 16133 --- [pool-5-thread-1] com.netflix.discovery.DiscoveryClient    : The response status is 200
2015-12-18 17:53:22.023  INFO 16133 --- [scoveryClient-0] com.netflix.discovery.DiscoveryClient    : DiscoveryClient_HELLOSERVER/Marks-MBP.home - retransmit instance info with status UP
2015-12-18 17:53:22.023  INFO 16133 --- [scoveryClient-0] com.netflix.discovery.DiscoveryClient    : DiscoveryClient_HELLOSERVER/Marks-MBP.home: registering service...
2015-12-18 17:53:22.031  INFO 16133 --- [scoveryClient-0] com.netflix.discovery.DiscoveryClient    : DiscoveryClient_HELLOSERVER/Marks-MBP.home - registration status: 204
2015-12-18 17:53:22.216  INFO 16133 --- [pool-4-thread-1] com.netflix.discovery.DiscoveryClient    : DiscoveryClient_HELLOSERVER/Marks-MBP.home - Re-registering apps/HELLOSERVER
2015-12-18 17:53:22.216  INFO 16133 --- [pool-4-thread-1] com.netflix.discovery.DiscoveryClient    : DiscoveryClient_HELLOSERVER/Marks-MBP.home: registering service...
2015-12-18 17:53:22.224  INFO 16133 --- [pool-4-thread-1] com.netflix.discovery.DiscoveryClient    : DiscoveryClient_HELLOSERVER/Marks-MBP.home - registration status: 204

The Eureka Service logs:

2015-12-18 17:52:52.043  WARN 16124 --- [nio-8761-exec-2] c.n.e.registry.AbstractInstanceRegistry  : DS: Registry: lease doesn't exist, registering resource: HELLOSERVER - Marks-MBP.home
2015-12-18 17:52:52.043  WARN 16124 --- [nio-8761-exec-2] c.n.eureka.resources.InstanceResource    : Not Found (Renew): HELLOSERVER - Marks-MBP.home
2015-12-18 17:52:52.177  INFO 16124 --- [nio-8761-exec-4] c.n.e.registry.AbstractInstanceRegistry  : Registered instance HELLOSERVER/Marks-MBP.home with status UP (replication=false)
2015-12-18 17:53:02.866 ERROR 16124 --- [et_localhost-18] c.n.e.cluster.ReplicationTaskProcessor   : Batch update failure with HTTP status code 401; discarding 1 replication tasks
2015-12-18 17:53:02.866  WARN 16124 --- [et_localhost-18] c.n.eureka.util.batcher.TaskExecutors    : Discarding 1 tasks of TaskBatchingWorker-target_localhost-18 due to permanent error
2015-12-18 17:53:12.711 ERROR 16124 --- [et_localhost-18] c.n.e.cluster.ReplicationTaskProcessor   : Batch update failure with HTTP status code 401; discarding 1 replication tasks
2015-12-18 17:53:12.711  WARN 16124 --- [et_localhost-18] c.n.eureka.util.batcher.TaskExecutors    : Discarding 1 tasks of TaskBatchingWorker-target_localhost-18 due to permanent error
2015-12-18 17:53:22.030  INFO 16124 --- [nio-8761-exec-1] c.n.e.registry.AbstractInstanceRegistry  : Registered instance HELLOSERVER/Marks-MBP.home with status UP (replication=false)
2015-12-18 17:53:22.222  INFO 16124 --- [nio-8761-exec-4] c.n.e.registry.AbstractInstanceRegistry  : Registered instance HELLOSERVER/Marks-MBP.home with status UP (replication=false)
2015-12-18 17:53:22.724 ERROR 16124 --- [et_localhost-18] c.n.e.cluster.ReplicationTaskProcessor   : Batch update failure with HTTP status code 401; discarding 1 replication tasks
2015-12-18 17:53:22.724  WARN 16124 --- [et_localhost-18] c.n.eureka.util.batcher.TaskExecutors    : Discarding 1 tasks of TaskBatchingWorker-target_localhost-18 due to permanent error
2015-12-18 17:53:32.750 ERROR 16124 --- [et_localhost-18] c.n.e.cluster.ReplicationTaskProcessor   : Batch update failure with HTTP status code 401; discarding 1 replication tasks
2015-12-18 17:53:32.750  WARN 16124 --- [et_localhost-18] c.n.eureka.util.batcher.TaskExecutors    : Discarding 1 tasks of TaskBatchingWorker-target_localhost-18 due to permanent error
2015-12-18 17:53:42.757 ERROR 16124 --- [et_localhost-18] c.n.e.cluster.ReplicationTaskProcessor   : Batch update failure with HTTP status code 401; discarding 1 replication tasks
2015-12-18 17:53:42.757  WARN 16124 --- [et_localhost-18] c.n.eureka.util.batcher.TaskExecutors    : Discarding 1 tasks of TaskBatchingWorker-target_localhost-18 due to permanent error
2015-12-18 17:53:52.764 ERROR 16124 --- [et_localhost-18] c.n.e.cluster.ReplicationTaskProcessor   : Batch update failure with HTTP status code 401; discarding 1 replication tasks
2015-12-18 17:53:52.764  WARN 16124 --- [et_localhost-18] c.n.eureka.util.batcher.TaskExecutors    : Discarding 1 tasks of TaskBatchingWorker-target_localhost-18 due to permanent error
2015-12-18 17:54:02.785 ERROR 16124 --- [et_localhost-18] c.n.e.cluster.ReplicationTaskProcessor   : Batch update failure with HTTP status code 401; discarding 1 replication tasks
2015-12-18 17:54:02.785  WARN 16124 --- [et_localhost-18] c.n.eureka.util.batcher.TaskExecutors    : Discarding 1 tasks of TaskBatchingWorker-target_localhost-18 due to permanent error
2015-12-18 17:54:12.792 ERROR 16124 --- [et_localhost-18] c.n.e.cluster.ReplicationTaskProcessor   : Batch update failure with HTTP status code 401; discarding 1 replication tasks
2015-12-18 17:54:12.793  WARN 16124 --- [et_localhost-18] c.n.eureka.util.batcher.TaskExecutors    : Discarding 1 tasks of TaskBatchingWorker-target_localhost-18 due to permanent error
2015-12-18 17:54:22.801 ERROR 16124 --- [et_localhost-10] c.n.e.cluster.ReplicationTaskProcessor   : Batch update failure with HTTP status code 401; discarding 1 replication tasks
2015-12-18 17:54:22.801  WARN 16124 --- [et_localhost-10] c.n.eureka.util.batcher.TaskExecutors    : Discarding 1 tasks of TaskBatchingWorker-target_localhost-10 due to permanent error

With a defined instanceId

You see no auth errors. But the registration behaviour is different too, in that the lease cannot be found. If I get a chance I'll try to debug this further but wondered if anything sprung out?

The Feign Server logs:

2015-12-18 18:06:50.902  INFO 16179 --- [pool-4-thread-1] com.netflix.discovery.DiscoveryClient    : DiscoveryClient_HELLOSERVER/Marks-MBP.home:HelloServer:7111 - Re-registering apps/HELLOSERVER
2015-12-18 18:06:50.902  INFO 16179 --- [pool-4-thread-1] com.netflix.discovery.DiscoveryClient    : DiscoveryClient_HELLOSERVER/Marks-MBP.home:HelloServer:7111: registering service...
2015-12-18 18:06:51.065  INFO 16179 --- [pool-4-thread-1] com.netflix.discovery.DiscoveryClient    : DiscoveryClient_HELLOSERVER/Marks-MBP.home:HelloServer:7111 - registration status: 204
2015-12-18 18:07:01.078  INFO 16179 --- [pool-4-thread-1] com.netflix.discovery.DiscoveryClient    : DiscoveryClient_HELLOSERVER/Marks-MBP.home:HelloServer:7111 - Re-registering apps/HELLOSERVER
2015-12-18 18:07:01.078  INFO 16179 --- [pool-4-thread-1] com.netflix.discovery.DiscoveryClient    : DiscoveryClient_HELLOSERVER/Marks-MBP.home:HelloServer:7111: registering service...
2015-12-18 18:07:01.087  INFO 16179 --- [pool-4-thread-1] com.netflix.discovery.DiscoveryClient    : DiscoveryClient_HELLOSERVER/Marks-MBP.home:HelloServer:7111 - registration status: 204
2015-12-18 18:07:10.874  INFO 16179 --- [pool-5-thread-1] com.netflix.discovery.DiscoveryClient    : Disable delta property : false
2015-12-18 18:07:10.874  INFO 16179 --- [pool-5-thread-1] com.netflix.discovery.DiscoveryClient    : Single vip registry refresh property : null
2015-12-18 18:07:10.874  INFO 16179 --- [pool-5-thread-1] com.netflix.discovery.DiscoveryClient    : Force full registry fetch : false
2015-12-18 18:07:10.875  INFO 16179 --- [pool-5-thread-1] com.netflix.discovery.DiscoveryClient    : Application is null : false
2015-12-18 18:07:10.875  INFO 16179 --- [pool-5-thread-1] com.netflix.discovery.DiscoveryClient    : Registered Applications size is zero : true
2015-12-18 18:07:10.877  INFO 16179 --- [pool-5-thread-1] com.netflix.discovery.DiscoveryClient    : Application version is -1: false
2015-12-18 18:07:10.882  INFO 16179 --- [pool-5-thread-1] com.netflix.discovery.DiscoveryClient    : Getting all instance registry info from the eureka server
2015-12-18 18:07:10.886  INFO 16179 --- [pool-5-thread-1] com.netflix.discovery.DiscoveryClient    : The response status is 200
2015-12-18 18:07:11.096  INFO 16179 --- [pool-4-thread-1] com.netflix.discovery.DiscoveryClient    : DiscoveryClient_HELLOSERVER/Marks-MBP.home:HelloServer:7111 - Re-registering apps/HELLOSERVER
2015-12-18 18:07:11.096  INFO 16179 --- [pool-4-thread-1] com.netflix.discovery.DiscoveryClient    : DiscoveryClient_HELLOSERVER/Marks-MBP.home:HelloServer:7111: registering service...
2015-12-18 18:07:11.104  INFO 16179 --- [pool-4-thread-1] com.netflix.discovery.DiscoveryClient    : DiscoveryClient_HELLOSERVER/Marks-MBP.home:HelloServer:7111 - registration status: 204
2015-12-18 18:07:20.879  INFO 16179 --- [scoveryClient-2] com.netflix.discovery.DiscoveryClient    : DiscoveryClient_HELLOSERVER/Marks-MBP.home:HelloServer:7111 - retransmit instance info with status UP
2015-12-18 18:07:20.879  INFO 16179 --- [scoveryClient-2] com.netflix.discovery.DiscoveryClient    : DiscoveryClient_HELLOSERVER/Marks-MBP.home:HelloServer:7111: registering service...
2015-12-18 18:07:20.888  INFO 16179 --- [scoveryClient-2] com.netflix.discovery.DiscoveryClient    : DiscoveryClient_HELLOSERVER/Marks-MBP.home:HelloServer:7111 - registration status: 204
2015-12-18 18:07:21.117  INFO 16179 --- [pool-4-thread-1] com.netflix.discovery.DiscoveryClient    : DiscoveryClient_HELLOSERVER/Marks-MBP.home:HelloServer:7111 - Re-registering apps/HELLOSERVER
2015-12-18 18:07:21.117  INFO 16179 --- [pool-4-thread-1] com.netflix.discovery.DiscoveryClient    : DiscoveryClient_HELLOSERVER/Marks-MBP.home:HelloServer:7111: registering service...
2015-12-18 18:07:21.127  INFO 16179 --- [pool-4-thread-1] com.netflix.discovery.DiscoveryClient    : DiscoveryClient_HELLOSERVER/Marks-MBP.home:HelloServer:7111 - registration status: 204
2015-12-18 18:07:31.140  INFO 16179 --- [pool-4-thread-1] com.netflix.discovery.DiscoveryClient    : DiscoveryClient_HELLOSERVER/Marks-MBP.home:HelloServer:7111 - Re-registering apps/HELLOSERVER
2015-12-18 18:07:31.140  INFO 16179 --- [pool-4-thread-1] com.netflix.discovery.DiscoveryClient    : DiscoveryClient_HELLOSERVER/Marks-MBP.home:HelloServer:7111: registering service...

Eureka Service logs:

2015-12-18 18:06:29.808  INFO 16171 --- [nio-8761-exec-2] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring FrameworkServlet 'dispatcherServlet'
2015-12-18 18:06:29.809  INFO 16171 --- [nio-8761-exec-2] o.s.web.servlet.DispatcherServlet        : FrameworkServlet 'dispatcherServlet': initialization started
2015-12-18 18:06:29.836  INFO 16171 --- [nio-8761-exec-2] o.s.web.servlet.DispatcherServlet        : FrameworkServlet 'dispatcherServlet': initialization completed in 27 ms
2015-12-18 18:06:50.898  WARN 16171 --- [nio-8761-exec-4] c.n.e.registry.AbstractInstanceRegistry  : DS: Registry: lease doesn't exist, registering resource: HELLOSERVER - Marks-MBP.home:HelloServer:7111
2015-12-18 18:06:50.899  WARN 16171 --- [nio-8761-exec-4] c.n.eureka.resources.InstanceResource    : Not Found (Renew): HELLOSERVER - Marks-MBP.home:HelloServer:7111
2015-12-18 18:06:51.063  INFO 16171 --- [nio-8761-exec-5] c.n.e.registry.AbstractInstanceRegistry  : Registered instance HELLOSERVER/Marks-MBP.home with status UP (replication=false)
2015-12-18 18:07:01.076  WARN 16171 --- [nio-8761-exec-8] c.n.e.registry.AbstractInstanceRegistry  : DS: Registry: lease doesn't exist, registering resource: HELLOSERVER - Marks-MBP.home:HelloServer:7111
2015-12-18 18:07:01.076  WARN 16171 --- [nio-8761-exec-8] c.n.eureka.resources.InstanceResource    : Not Found (Renew): HELLOSERVER - Marks-MBP.home:HelloServer:7111
2015-12-18 18:07:01.085  INFO 16171 --- [nio-8761-exec-9] c.n.e.registry.AbstractInstanceRegistry  : Registered instance HELLOSERVER/Marks-MBP.home with status UP (replication=false)
2015-12-18 18:07:11.095  WARN 16171 --- [nio-8761-exec-1] c.n.e.registry.AbstractInstanceRegistry  : DS: Registry: lease doesn't exist, registering resource: HELLOSERVER - Marks-MBP.home:HelloServer:7111
2015-12-18 18:07:11.095  WARN 16171 --- [nio-8761-exec-1] c.n.eureka.resources.InstanceResource    : Not Found (Renew): HELLOSERVER - Marks-MBP.home:HelloServer:7111
2015-12-18 18:07:11.103  INFO 16171 --- [nio-8761-exec-2] c.n.e.registry.AbstractInstanceRegistry  : Registered instance HELLOSERVER/Marks-MBP.home with status UP (replication=false)
2015-12-18 18:07:20.887  INFO 16171 --- [nio-8761-exec-3] c.n.e.registry.AbstractInstanceRegistry  : Registered instance HELLOSERVER/Marks-MBP.home with status UP (replication=false)
2015-12-18 18:07:21.113  WARN 16171 --- [nio-8761-exec-4] c.n.e.registry.AbstractInstanceRegistry  : DS: Registry: lease doesn't exist, registering resource: HELLOSERVER - Marks-MBP.home:HelloServer:7111
2015-12-18 18:07:21.114  WARN 16171 --- [nio-8761-exec-4] c.n.eureka.resources.InstanceResource    : Not Found (Renew): HELLOSERVER - Marks-MBP.home:HelloServer:7111
2015-12-18 18:07:21.124  INFO 16171 --- [nio-8761-exec-5] c.n.e.registry.AbstractInstanceRegistry  : Registered instance HELLOSERVER/Marks-MBP.home with status UP (replication=false)
2015-12-18 18:07:31.137  WARN 16171 --- [nio-8761-exec-8] c.n.e.registry.AbstractInstanceRegistry  : DS: Registry: lease doesn't exist, registering resource: HELLOSERVER - Marks-MBP.home:HelloServer:7111
2015-12-18 18:07:31.138  WARN 16171 --- [nio-8761-exec-8] c.n.eureka.resources.InstanceResource    : Not Found (Renew): HELLOSERVER - Marks-MBP.home:HelloServer:7111
2015-12-18 18:07:31.148  INFO 16171 --- [nio-8761-exec-9] c.n.e.registry.AbstractInstanceRegistry  : Registered instance HELLOSERVER/Marks-MBP.home with status UP (replication=false)
2015-12-18 18:07:41.160  WARN 16171 --- [nio-8761-exec-1] c.n.e.registry.AbstractInstanceRegistry  : DS: Registry: lease doesn't exist, registering resource: HELLOSERVER - Marks-MBP.home:HelloServer:7111
2015-12-18 18:07:41.160  WARN 16171 --- [nio-8761-exec-1] c.n.eureka.resources.InstanceResource    : Not Found (Renew): HELLOSERVER - Marks-MBP.home:HelloServer:7111
2015-12-18 18:07:41.170  INFO 16171 --- [nio-8761-exec-2] c.n.e.registry.AbstractInstanceRegistry  : Registered instance HELLOSERVER/Marks-MBP.home with status UP (replication=false)
2015-12-18 18:07:51.176  WARN 16171 --- [nio-8761-exec-3] c.n.e.registry.AbstractInstanceRegistry  : DS: Registry: lease doesn't exist, registering resource: HELLOSERVER - Marks-MBP.home:HelloServer:7111
2015-12-18 18:07:51.177  WARN 16171 --- [nio-8761-exec-3] c.n.eureka.resources.InstanceResource    : Not Found (Renew): HELLOSERVER - Marks-MBP.home:HelloServer:7111
2015-12-18 18:07:51.183  INFO 16171 --- [nio-8761-exec-4] c.n.e.registry.AbstractInstanceRegistry  : Registered instance HELLOSERVER/Marks-MBP.home with status UP (replication=false)
2015-12-18 18:07:52.919  WARN 16171 --- [nio-8761-exec-5] c.n.e.registry.AbstractInstanceRegistry  : DS: Registry: cancel failed because Lease is not registered for: HELLOSERVER/Marks-MBP.home:HelloServer:7111
2015-12-18 18:07:52.919  INFO 16171 --- [nio-8761-exec-5] c.n.eureka.resources.InstanceResource    : Not Found (Cancel): HELLOSERVER - Marks-MBP.home:HelloServer:7111
KramKroc commented 8 years ago

Actually, I just updated https://github.com/KramKroc/feign-eureka.git to Brixton.M3 of Spring Cloud and the constant reregistering has disappeared and we now consistently get the 401 error regardless of whether we have instanceId specified @spencergibb

spencergibb commented 8 years ago

By default eureka register's things with hostname as the instanceId. My guess is without setting the instanceId, eureka gets confused and is trying to peer with it. Eureka by default expects every instance registered to be on a different host. Setting instanceId is the way to go.

KramKroc commented 8 years ago

@spencergibb not sure if you saw my follow up about the behaviour in Brixton.M3? 401 is no consistently occurring regardless of instanceId being specified.

spencergibb commented 8 years ago

Can you try with eureka.instance.instanceId instead of eureka.instance.metadataMap.instanceId?

KramKroc commented 8 years ago

Tried, but still 401 errors as before. Checked latest code in if you need to clone to reproduce it.

spencergibb commented 8 years ago

@KramKroc I am unable to reproduce using those projects. I'm not running a config server on localhost are you?

KramKroc commented 8 years ago

Not running any config server @spencergibb And you've run my forked versions of the projects rather than the originals?

spencergibb commented 8 years ago

yes

KramKroc commented 8 years ago

Opps, turns I did have a config server instance running locally, but I nixed it, restarted eureka and feign and I still have the 401 errors in the eureka server logs.

KramKroc commented 8 years ago

Hi @spencergibb I tried this out on my windows PC over the weekend and like you I can't reproduce it at all. This is totally confusing as it happens each time on my mac. I'm going to see if any party in my office can try this out tomorrow morning and get back to you. Both machines are using the same version of Java and the latest code is committed to GitHub so should be running the same codeline. What OS did you happen to use when you tried this out?

dsyer commented 8 years ago

Is there something else running that you didn't know about (like the config server was)? All the logs tell us from the server is that there is a client that isn't authenticated. That wouldn't be very surprising given that unauthenticated is the default. The clients you already told us about are working, right? So there isn't really a problem here at all in that case.

I think we need to close this issue again and open a new one with crisper description if there is still actually a problem. There's too much noise here and it's not clear we are all talking about the same issue.

KramKroc commented 8 years ago

Hi @dsyer. The first thing I did after I realised that I had a config server from another project still running was shut it down and then grep for java processes. After confirming that nothing else was running, I started the eureka and feign-eureka server running again and still witnessed the behaviour, so it isn't a rogue application that is connecting to the server. There is nothing in the client (feign-eureka) logs indicating it is having communication issues with the server.

I reran with debug enabled for the eureka server and the snippet from the logs below shows possibly some more detail, i.e. there is an initial auth success followed by a auth failure for a batch update. As you said, the client and server seem to work OK, but surely the error indicates something unexpected is occurring?

I'm happy to close this issue and open a new one but the behaviour I'm seeing is inline with the original issue raised by @marcelalburg 

2015-12-21 08:59:58.936 DEBUG 729 --- [nio-8761-exec-5] o.s.b.c.web.OrderedRequestContextFilter  : Bound request context to thread: org.apache.catalina.connector.RequestFacade@390ca853
2015-12-21 08:59:58.936 DEBUG 729 --- [nio-8761-exec-5] o.s.b.a.audit.listener.AuditListener     : AuditEvent [timestamp=Mon Dec 21 08:59:58 GMT 2015, principal=user, type=AUTHENTICATION_SUCCESS, data={details=org.springframework.security.web.authentication.WebAuthenticationDetails@957e: RemoteIpAddress: 127.0.0.1; SessionId: null}]
2015-12-21 08:59:58.939 DEBUG 729 --- [nio-8761-exec-5] o.s.b.c.web.OrderedRequestContextFilter  : Cleared thread-bound request context: org.apache.catalina.connector.RequestFacade@390ca853
2015-12-21 08:59:59.449 DEBUG 729 --- [nio-8761-exec-6] o.s.b.c.web.OrderedRequestContextFilter  : Bound request context to thread: org.apache.catalina.connector.RequestFacade@390ca853
2015-12-21 08:59:59.450 DEBUG 729 --- [nio-8761-exec-6] o.s.b.a.audit.listener.AuditListener     : AuditEvent [timestamp=Mon Dec 21 08:59:59 GMT 2015, principal=anonymousUser, type=AUTHORIZATION_FAILURE, data={type=org.springframework.security.access.AccessDeniedException, message=Access is denied}]
2015-12-21 08:59:59.451 DEBUG 729 --- [nio-8761-exec-6] o.s.b.c.web.OrderedRequestContextFilter  : Cleared thread-bound request context: org.apache.catalina.connector.RequestFacade@390ca853
2015-12-21 08:59:59.451 DEBUG 729 --- [nio-8761-exec-6] o.s.b.a.e.mvc.EndpointHandlerMapping     : Looking up handler method for path /error
2015-12-21 08:59:59.453 DEBUG 729 --- [nio-8761-exec-6] o.s.b.a.e.mvc.EndpointHandlerMapping     : Did not find handler method for [/error]
2015-12-21 08:59:59.468 ERROR 729 --- [get_localhost-2] c.n.e.cluster.ReplicationTaskProcessor   : Batch update failure with HTTP status code 401; discarding 1 replication tasks
2015-12-21 08:59:59.468  WARN 729 --- [get_localhost-2] c.n.eureka.util.batcher.TaskExecutors    : Discarding 1 tasks of TaskBatchingWorker-target_localhost-2 due to permanent error
dsyer commented 8 years ago

It looks like it's just eureka failing to register with itself to me. You can switch off the registration if you have a standalone server (it's in the docs I believe).

KramKroc commented 8 years ago

Thank you @dsyer!

I believed my eureka server was running in Standalone mode, but I re-read the standalone server documentation again:

http://projects.spring.io/spring-cloud/docs/1.0.1/spring-cloud.html#_standalone_mode

I had thought you simply needed:

server:
  port: 8761

eureka:
  client:
    registerWithEureka: false
    fetchRegistry: false
  server:
    waitTimeInMsWhenSyncEmpty: 0
security:
  user:
    name: user
    password: password

To ensure standalone mode was enabled I thought all you needed was eureka.client.registerWithEureka and eureka.client.fetchRegistry set to false. When I added the other fields, eureka.instance.hostname and eureka.client.serviceUrl the 401 Auth error disappeared. The same applied when I commented out the eureka.client.serviceUrl so my working version how has the following:

server:
  port: 8761

eureka:
  instance:
    hostname: localhost
  client:
    registerWithEureka: false
    fetchRegistry: false
#    serviceUrl:
#      defaultZone: http://${eureka.instance.hostname}:${server.port}/eureka/
  server:
    waitTimeInMsWhenSyncEmpty: 0
security:
  user:
    name: user
    password: password

View difference here: https://github.com/KramKroc/eureka/commit/899a6932d0051cc36f31dc1a3f664e8d6dd5ab60#diff-256c57f5b357be82840a9b2207f0dcd5

In other words, to get this to work on OS-X (and I think linux too) the eureka.instance.hostname had to be set to localhost. I still think there is an issue here with how standalone mode with basic auth behaves on different OS's and I think the documentation on Standalone mode could be clearer on which specific fields need to be set to ensure standalone mode works.

dsyer commented 8 years ago

Hmm. I still don't quite see what you think happened. The only difference in your server where you say it works as expected is the eureka.instance.hostname (which shouldn't be used anyway in standalone mode). I also find it quite implausible that there really is a difference between OS types. Are you absolutely certain you don't have some other source of property values in your environments that don't work as you expect (the /env endpoint will tell you)?

We can't document anything unless we can get a crisp explanation of what works and what doesn't. There's no reason why it has to be you that figures it out, but if you do, please post back here with more details.

KramKroc commented 8 years ago

Hi @dsyer I'm not sure how to make it clearer:

  1. Repeated 401 errors in any application log would not be acceptable in any production system
  2. As more and more clients connect, errors are generated for each, meaning logs would quickly fill and roll over
  3. You indicated the error I was likely to be caused because the server wasn't set up as standalone, but one of the properties from the Standalone mode documentation, eureka.instance.hostname, was changed and got rid of the problem but you don't think I even need to set that? Again this indicates the standalone mode may not be clearly documented.
  4. I was able to confirm on a colleagues OS-X machine that the same issue (prior to latest tweak to eureka.instance.hostname) was reproduced for them, but I wasn't to reproduce the issue on a windows machine. @spencergibb indicated he couldn't reproduce it, so I wonder what OS he was using to test it? The issue was also manifest on a java8 docker image as well.
spencergibb commented 8 years ago

@KramKroc I'm using OSX

KramKroc commented 8 years ago

@spencergibb is there any reason why I shouldn't share a json dump of the /env endpoint from my eureka server to see if that would highlight anything?

spencergibb commented 8 years ago

@KramKroc my guess is that on https://github.com/KramKroc/eureka/commit/899a6932d0051cc36f31dc1a3f664e8d6dd5ab60, the eureka.instance.hostname wasn't the fix, but removing the location of eureka via defaultUrl, which doesn't have credentials in it.

spencergibb commented 8 years ago

It can't hurt. It's really hard to do much about it when we can't reproduce the problem. By default there is no security on the eureka server. As soon as you add security, the reason you would get a 401 is there is a client trying to connect without credentials or with incorrect credentials. eureka.instance.hostname has nothing to do with security and wouldn't fix a 401.

spencergibb commented 8 years ago

You're also using SNAPSHOTS on the server. Which is a moving target and use at your own risk.

KramKroc commented 8 years ago

Hi @spencergibb I've checked in a version using 1.3.1.RELASE for Spring Boot and Brixton.M3 for Spring Cloud.

If you look again at https://github.com/KramKroc/eureka/commit/899a6932d0051cc36f31dc1a3f664e8d6dd5ab60 you will see that eureka.client.serviceUrl.defautZone is commented out and was not added prior to eureka.instance.hostname being defined so I don't think that plays a part here (I presume there is a default value for eureka.client.serviceUrl.defautZone but either way, the problem disappeared with the addition of the eureka.instance.hostname definition for me).

Just looking at the log I pasted into https://github.com/spring-cloud/spring-cloud-netflix/issues/291#issuecomment-166242529 to my ill informed eye it looks like the server is not acting properly in standalone mode there at all and is trying to replicate the data locally and not using auth credentials which is why a 401 is appearing in that log rather than in the feign-eureka client logs. But that is just a hunch and as you can't reproduce the issue it may well just end up being closed unless someone else can reproduce this other than my local team.

MysteryAngle commented 8 years ago 
eureka:
  client:
    service-url: 
        defaultZone: http://${security.user.name}:${security.user.password}@${eureka.instance.hostname}:${server.port}/eureka/

It's work for me.

davidrdz93 commented 4 years ago

I've had a hard time with this problem. I was getting always 401 from my Discovery Server

I added instance.hostname on my service and i solved the issue

This is my configuration:

Eureka Server:

`server: port: 8761

spring: application: name: eureka

security: user: name: discUser password: discPassword

eureka: client: register-with-eureka: false fetch-registry: false

server: wait-time-in-ms-when-sync-empty: 0 `

Discovery Client:

`eureka: instance: prefer-ip-address: true hostname: localhost

client: register-with-eureka: true fetch-registry: true service-url: defaultZone: http://discUser:discPassword@${eureka.instance.hostname}:8761/eureka/`

apo1967 commented 2 years ago

I've had a hard time with this problem. I was getting always 401 from my Discovery Server

I added instance.hostname on my service and i solved the issue

This is my configuration:

Eureka Server:

`server: port: 8761

spring: application: name: eureka

security: user: name: discUser password: discPassword

eureka: client: register-with-eureka: false fetch-registry: false

server: wait-time-in-ms-when-sync-empty: 0 `

Discovery Client:

`eureka: instance: prefer-ip-address: true hostname: localhost

client: register-with-eureka: true fetch-registry: true service-url: defaultZone: http://discUser:discPassword@${eureka.instance.hostname}:8761/eureka/`

Thx, that helped me! I'd like to point out that the important part is to actually self-register with the credentials of the basic Spring Security config.

padilhas commented 8 months ago

I had the same problem and I fixed it removing the basic authentication of Eureka Server. My Eureka server was using autoconfigure for these 2 classes: SecurityAutoConfiguration and ManagemenWebSecurityAutoConfiguration. If for some reason you have to add a dependency to spring-cloud-starter-security, remove the 2 autoconfigure.