CASSGO-3 How to set limit for pool connection

Please answer these questions before submitting your issue. Thanks!

What version of Cassandra are you using?

4.x

What version of Gocql are you using?

v1.6.0

What version of Go are you using?

1.19

What did you do?

I am creating a pool connection with cluster.NumConns = 20. so that it can create only 20 connection/pod. container := dbType + "." + dbName username := String(container+".username", "") password := String(container+".password", "") host := String(container+".host", "") db := String(container+".db", "") dc := String(container+".dc", "") port1 := String(container+".port", "")

// fmt.Println(container, " ", username, " ", password, " ", host, " ", db, " ", dc, " ", port1)
port, err := strconv.Atoi(port1)

HostList := strings.Split(host, ",")

cluster := gocql.NewCluster(HostList...)
pass := gocql.PasswordAuthenticator{username, password, nil}
cluster.Keyspace = db
cluster.Authenticator = pass
cluster.Consistency = gocql.LocalQuorum
//cluster.SerialConsistency = gocql.LocalSerial
if err == nil {
    cluster.Port = port // Default port
}

if dbName == "cass_comp_gcp" || dbName == "cass_comp_gcp_newIN" {
    cluster.DisableInitialHostLookup = true
    cluster.SocketKeepalive = 10 * time.Second
}
cluster.NumConns = 20
//
cluster.PoolConfig.HostSelectionPolicy = gocql.DCAwareRoundRobinPolicy(dc)
cluster.HostFilter = gocql.DataCentreHostFilter(dc)
cluster.Timeout = 5 * time.Second
if GetMode() != "prod" {
    cluster.ConnectTimeout = time.Second * 20
    //cluster.DisableInitialHostLookup = true
}
// cluster.SocketKeepalive = 10 * time.Second

return cluster.CreateSession()

What did you expect to see?

We are using the above function in our read API which is capable of serving around 5,000 requests per second. Our goal is to limit the number of connections created per pod to 20, even if there is an increase in request traffic. Although we have added a limit of 20 connections, we have observed that if the request count increases, it opens more than 20 connections, and some of them remain open even after the normal pace.

What did you see instead?

It is creating around 200-600 connections per pod, which is not getting closed even after the normal pace. This is causing a load issue for Cassandra.

If you are having connectivity related issues please share the following additional information

Describe your Cassandra cluster

please provide the following information

output of nodetool status
output of SELECT peer, rpc_address FROM system.peers
rebuild your application with the gocql_debug tag and post the output

NumConns sets an initial number of connections per node( by default 2) but not the maximum number of connections per node. Also by default, gocql maintains a minimum number of connections, but as demand increases, it will open more connections to handle the load. I think you are looking for similar limitation functionality as Python and Java drivers have. So we can add MaxNumberOfConnections or something to set the wanted limit, and we should add a detailed NumConns description to the documentation. EDIT: I`m sorry, I gave some misinformation. During MaxNumberOfConnections implementation and more detailed research, I have found that NumConns sets not only the initial value but the maximum too. Also, there's no mechanism to create additional connections when the load increases, I tried to reproduce your issue, and with the current driver implementation doesn't succeed. I believe you have an autoscaling mechanism, which with increasing load creates more replicas of your application instance.
Thank you for your appeal, in any case, documentation shouldn't be so unclear, I'll fix it.

apache / cassandra-gocql-driver