BBB plugin crashing under load

[JulianOliver]

We have a 5-figure Mattermost population using our instance day in and out. Big Blue Button has been used heavily, largely through MM, since launch a couple of weeks ago. The BBB plugin has crashed three times in this period, during peak usage times.

{"level":"error","ts":1587506698.767737,"caller":"mlog/log.go:175","msg":"Plugin failed to ServeHTTP, RPC call failed","plugin_id":"bigbluebutton","error":"unexpected EOF"}
{"level":"error","ts":1587506707.423524,"caller":"plugin/health_check.go:90","msg":"Health check failed for plugin","id":"bigbluebutton" ,"error":"Plugin RPC connection is not responding"}
{"level":"error","ts":1587506707.4306667,"caller":"mlog/log.go:175","msg":"RPC call OnDeactivate to plugin failed.","plugin_id":"bigbluebutton","error":"connection is shut down"}
{"level":"warn","ts":1587506707.4432015,"caller":"plugin/hclog_adapter.go:51","msg":"error closing client during Kill","plugin_id":"bigbluebutton","wrapped_extras":"errconnection is shut down"}
{"level":"warn","ts":1587506707.4432223,"caller":"plugin/hclog_adapter.go:53","msg":"plugin failed to exit gracefully","plugin_id":"bigbluebutton"}
{"level":"error","ts":1587506708.0133278,"caller":"app/plugin_key_value_store.go:85","msg":"Failed to query plugin key value","plugin_id":"bigbluebutton","key":"all_meetings","error":"SqlPluginStore.Get: Could not get plugin key value, plugin_id=bigbluebutton, key=all_meetings, err=Error 1040: Too many connections"}

Any ideas and/or patches would be greatly appreciated!

[melroy89]

Sounds like you need to increase the max number of database connections (MySQL?). Ps. I'm not a developer of Mattermost nor BBB.

[ghost]

@JulianOliver I think the actual error is way above in logs. Can you try attaching the complete log file? You can filter for plugin events by searching for "plugin_id":"bigbluebutton " and only including those lines.

[harshilsharma63]

@JulianOliver can you upgrade to the latest v3 release? It offers performance and stability improvements. Feel free to re-open this issue if you still face the problem.