DataFrogman
commented
1 year ago The following code should near instantly duplicate the issue.
#[tokio::main]
async fn main() -> Result<()> {
let uri = "bolt://172.20.26.147:7687";
let user = "neo4j";
let password = "neo4j;
let config = ConfigBuilder::default()
.uri(uri)
.user(user)
.password(password)
.max_connections(10240)
.build()
.unwrap();
let graph = Arc::new(
Graph::connect(config)
.await
.unwrap()
);
let semaphore = Arc::new(Semaphore::new(1024));
let mut acc: usize = 0;
loop {
let permit = semaphore.clone();
let _permit = permit.acquire_owned().await.unwrap();
let cloned_graph = graph.clone();
let cloned_acc = acc.to_string();
tokio::spawn(async move {
let temp = cloned_graph.run(neo4rs::query(&format!("MERGE (n:Num {{num: '{cloned_acc}'}});"))).await;
match temp {
Ok(_) => (),
Err(err) => println!("{err:?}"),
}
std::mem::drop(_permit);
});
acc += 1;
}
}
knutwalker
I have an async program that runs
graph.executeandgraph.runcalls. I am using a semaphore to limit the number of concurrent tasks at once. My neo4j server is set to have 100x the number of concurrent tasks for its thread pool maximum and I am setting my max_connections for the ConfigBuilder at 10x the number of concurrent tasks. When I run agraph.executeorgraph.runI make sure toawaiton them, so each task should only have one live connection at a time.Despite this I very quickly hit
AuthenticationError("There are no available threads to serve this request at the moment. You can retry at a later time or consider increasing max thread pool size for bolt connector(s).")orUnexpectedMessage("unexpected response for RUN: Ok(Failure(Failure { metadata: BoltMap { value: {BoltString { value: \"code\" }: String(BoltString { value: \"Neo.TransientError.Request.NoThreadsAvailable\" }), BoltString { value: \"message\" }: String(BoltString { value: \"There are no available threads to serve this request at the moment. You can retry at a later time or consider increasing max thread pool size for bolt connector(s).\" })} } }))"). The higher my number of concurrent tasks the faster I will hit these errors, this leads me to believe that there is some lag time with closing the connections despite theawaitstatements. At 1024 concurrent tasks it takes about 15 seconds, at 512 concurrent tasks it occurs within a couple of minutes.