RST packets when disconnecting SSL session and occasional errors when haproxy is loadbalancing mysql

[matejzero]

Description

I set up an HAProxy load balancer in front of a MySQL cluster to distribute connections from my PHP app to the database. However, I noticed that around 40% of the time, HAProxy logs an error with termination state "SD." According to the HAProxy documentation:

• S: The TCP session was unexpectedly aborted by the server or explicitly refused.
• D: The session was in the DATA phase.

I reproduced this issue using simple PHP code that connects and disconnects from the MySQL server:

<?php
$username = "foobar";
$servername = "lb.foo.bar";
$password = "foobar";
$dbname = "test";

$mysqli = new mysqli();
$mysqli->real_connect($servername, $username, $password, $dbname, NULL, NULL, MYSQLI_CLIENT_SSL);

if ($mysqli->connect_error) {
    die("Connection failed: " . $mysqli->connect_error);
}
echo "Connected successfully with SSL";

$mysqli->close();
?>

When analyzing the traffic with tcpdump, I observed a TLS encrypted alert packet, followed by the session being torn down and RST packets arriving from the server. Occasionally, two RST packets are sent, but when three RST packets are received, HAProxy logs the "SD" error.

Initially, I suspected a configuration issue, but my setup aligns with HAProxy's recommended configuration. I tested direct MySQL connections (bypassing HAProxy) and encountered the same connection teardown behavior. Interestingly, switching PHP to non-SSL resolved the issue, and I saw a normal TCP teardown sequence (FIN, ACK; FIN, ACK; ACK).

I also tested Python and Go applications with SSL, and both showed the expected TCP teardown sequence, similar to non-SSL connections in PHP. Only PHP with SSL seems to trigger the abnormal behavior, with a TLS encrypted alert packet followed by an unexpected termination.

Problematic tcpdump:

Same connection with Python code:

In summary:

• PHP with SSL causes unexpected RST packets and HAProxy "SD" errors.
• Python and Go, even with SSL, behave as expected.
• The issue seems to stem from how PHP handles SSL termination compared to other languages.

Is this a bug in PHP's SSL handling, or is this expected behavior when using SSL with MySQL in PHP?

PHP Version

PHP 8.3.11

Operating System

AlmaLinux 8

[matejzero]

I managed to decrypt the TLS traffic and TLS alert message is Close Notify, if that helps.