spring-cloud / spring-cloud-netflix

Integration with Netflix OSS components
http://cloud.spring.io/spring-cloud-netflix/
Apache License 2.0
4.87k stars 2.44k forks source link

status”:500,“error”:“Internal Server Error”,“exception”:“com.netflix.zuul.exception.ZuulException”,“message”:“REJECTED_THREAD_EXECUTION”} #3064

Closed rohitdec01 closed 5 years ago

rohitdec01 commented 6 years ago

{"timestamp":1531026130373,"status":500,"error":"Internal Server Error","exception":"com.netflix.zuul.exception.ZuulException","message":"REJECTED_THREAD_EXECUTION"}

I am getting above exception while running stress testing on my one of the API redirecting from zull gateway.

Here is some of the more points which I want to mentioned here.

I am using netflix zull and eureka for discovery my microservices (spring boot).

Everything is working fine from my angular application, postman and swagger.

Now I am trying to do stress testing using Jmeter tool which is working fine for <=10 no. of thread usersload. But as soon as I am mentioning more than 10 no. of thread users. It is giving me above mentioned exception.

I also tried to test my API directly (without gateway -zull) and that is also working fine in case of more than 10+ no. of thread users and that is working fine too.

I tried to replicate same scenario on my local laptop and used the same jmeter. And this time rather giving the server host name I try to test with my laptop hostname (all code and services are same). I mentioned 10+ no. of thread users in jmeter and that is working fine with no above exception.

I read lot of stack-overflow posts, github links and other web search for this issue but none of solution seems to working. I tried and some other

ribbon: ReadTimeout: 60000 ConnectTimeout: 60000

hystrix: command: default: execution: isolation: strategy: THREAD thread: timeoutInMilliseconds: 60000

holy12345 commented 6 years ago

HI @rohitdec01

Could you please change hystrix.threadpool.default.coreSize the number, because in my mind it default value is 10.

In addition i notice you have those parameter

ribbon. ReadTimeout: 60000
ribbon.ConnectTimeout: 60000
hystrix: command: default: execution: isolation: strategy: THREAD thread: timeoutInMilliseconds: 60000

Could you please change it like this

ribbon. ReadTimeout: 60000
ribbon.ConnectTimeout: 60000
hystrix: command: default: execution: isolation: strategy: THREAD thread: timeoutInMilliseconds: 240000

In my mind Hystrix timeout should include all retry time(include retry same server and next server), because ribbon retry same server default is 0 and retry next server default is 1.

Hope it can help you

thanks

rohitdec01 commented 6 years ago

thanks for your quick reply. I already tried below properties as well but getting some other Zull exception after this. (refer attached .png file)

{"timestamp":1531112495424,"status":500,"error":"Internal Server Error","exception":"com.netflix.zuul.exception.ZuulException","message":"SHORTCIRCUIT"}

rohitdec01 commented 6 years ago

application-properties

holy12345 commented 6 years ago

@rohitdec01

Remove hystrix.command.default.excution.isolation.timeout.enable properties, and try it.

rohitdec01 commented 6 years ago

No, still the same issue. Attached is the screenshot.

error

ryanjbaxter commented 6 years ago

Can you provide the entire exception?

Please dont use screenshots, put the code in the issue.

Please learn how to format code on GitHub.

rohitdec01 commented 6 years ago

My manager told me to remove zull and do the load testing untill this issue get fixed. So, I will come back in a day or 2 on this.

I will share the entire exception.

jprateekvmware commented 5 years ago

Any update here

ryanjbaxter commented 5 years ago

no

spring-projects-issues commented 5 years ago

If you would like us to look at this issue, please provide the requested information. If the information is not provided within the next 7 days this issue will be closed.

spring-projects-issues commented 5 years ago

Closing due to lack of requested feedback. If you would like us to look at this issue, please provide the requested information and we will re-open the issue.

sumanb21 commented 5 years ago

i also got same exception

Lakshman54 commented 2 years ago

I also got the same exception

ayushstwt commented 2 years ago

I also got the same Exception