search

logtail / logback-logtail

Better Stack Java Logback appender
https://betterstack.com/logs
MIT License
17 stars 4 forks source link

Performance issue when sending the log through JAX RS #1

Closed sgyyz closed 1 year ago

sgyyz commented 2 years ago

What

The performance is good when we did the local testing, but when we push to our test environment, there are 5~8 users are doing the requests at the same time, our target server became slow and slow. And revert the logback-logtail as our log dependency, everything backs to normal. 🤣

Why

After doing some research, it was caused by the JAX RS Client.

As we can see the client is a "singleton", it can not deal with massive requests at the same time. It likes our application has one database connection to process the massive user queries.

I suggest:

  1. if the JAX RS is the lib required, then we can use the connection pool to improve the performance.
  2. if the request client can decide by some customized code, like the Spring users may use the Feign or RestTemplate, it will be helpful for these users not bringing the JAX RS and Jersey related dependencies.
sgyyz commented 2 years ago

some findings: it's better to add the jersey-apache-connector to benefit the connect pool feature from it.

Refer to: http://www.theotherian.com/2013/08/jersey-client-2.0-httpclient-timeouts-max-connections.html

sgyyz commented 2 years ago

more reference LogglyBatchAppender from org.logback-extensions:logback-ext-loggly.

It's easy to rewrite the getHttpUrlConnection and processEntries methods to customize the request with Authorization and the request body content format.

cowwoc commented 1 year ago

For what it's worth, Jetty's HTTP Client is quite similar to JAX-RS's and it's supposed to scale very well.

PetrHeinz commented 1 year ago

Thanks @sgyyz for bringing this up 🙌

In version 0.3.0 JAX RS will no longer be used (see #13). I'll do some testing and should it be merged and released in no time.

Hope this helps! 🙂