airlift / drift

An annotation-based Java library for creating Thrift serializable types and services.
Apache License 2.0
242 stars 92 forks source link

i wanna know how to validate connection #96

Closed naah69 closed 5 years ago

naah69 commented 5 years ago

i wanna know how to validate connection. i can't find validate method in DriftClientFactory , DriftClient.

dain commented 5 years ago

Can you explain what you mean by "validate connection"? I ask because the client is independent of connections, and instead uses a connection pool under the covers.

naah69 commented 5 years ago

it means that how to validate this connection(or client) is available when i get it .

electrum commented 5 years ago

What higher level problem are you trying to solve? When you call a method, Drift will get a connection from the pool and send the request. If there is a network error, it will close the connection and open a new one automatically.

electrum commented 5 years ago

As a user of Drift, you should never have to care about validating connections, because Drift handles that for you.

naah69 commented 5 years ago

3q, i also wanna know the detail config of DriftNettyClientConfig. for example: max-idle-per-key、min-idle-per-key and so on

naah69 commented 5 years ago

i have a another question. when i had used the connections,how i can put back connections to pool

naah69 commented 5 years ago

when i use 500 threads to pressure test for 30 minutes . the factory always createing client.then it always throw exception. i guess it didn't manage connection

electrum commented 5 years ago

Drift connection pooling was disabled by default until the 1.15 release. It is now enabled by default.

We don’t yet have documentation for all the config properties. However, if you use Bootstrap to create the Guice injector, it will print all the properties at startup. See here for an example of how to use it: https://github.com/airlift/drift/blob/master/drift-integration-tests/src/test/java/io/airlift/drift/integration/guice/TestGuiceIntegration.java

naah69 commented 5 years ago

i write a plugin named spring-cloud-starter-drift. but i am not satisfied with it. when i use 500 threads to pressure test for 30 minutes,it always be timeout and some didn't timeout

electrum commented 5 years ago

Can you give us the client and server you are using to test? Or write a simpler version that reproduces the issue?

If we can run it ourselves and reproduce the issue, it should be easy to fix it.

dain commented 5 years ago

@naah69 a lot depends on the size of the RPCs you are doing and the average latency, but I guess you might be stressing stuff because the shear number of threads. Can you change your code to use less threads and async calls instead?

Anyway, typically, there are multiple servers so you are hammering a single server (this is the kind of setup used at Facebook). This will spread the load across multiple sockets, so if that is you target setup you should test with multiple servers. Otherwise, I would take a long look at changing the protocol to not need so many small RPCs, as that stresses servers.

naah69 commented 5 years ago

when i had finished this request ,how to return the connect to Factory.(idle connect)

electrum commented 5 years ago

The connection is automatically checked out from the pool as needed when you make a remote call. There is no way to borrow/return a connection manually, so it’s impossible to leak.

naah69 commented 5 years ago

ok.when i never use it.will it be GC?

electrum commented 5 years ago

Yes, there is background thread to cleanup idle connections. See https://github.com/airlift/drift/blob/master/drift-transport-netty/src/main/java/io/airlift/drift/transport/netty/client/ConnectionPool.java

naah69 commented 5 years ago

can i set some parameter that is about idle,or know detail of clean idle

electrum commented 5 years ago

You can set these:

thrift.client.connection-pool.max-size thrift.client.connection-pool.idle-timeout