Texera / texera

Collaborative Machine-Learning-Centric Data Analytics Using Workflows
https://texera.github.io
Apache License 2.0
163 stars 75 forks source link

Address "Too Many Connections" Issue for Workflow Runtime Statistics #3078

Closed kunwp1 closed 1 week ago

kunwp1 commented 1 week ago

This PR addresses the "Too many connections" issue encountered when storing runtime statistics for workflows. Previously, a new JDBC connection was created for each workflow execution, which led to scalability issues when handling a large number of users or concurrent workflows.

Fix This update introduces a single, shared JDBC connection that is reused across all workflow executions, significantly improving scalability and reducing the risk of connection overload.