DBD::mysql connect memory leak [rt.cpan.org #80669]

[mbeijen]

Migrated from rt.cpan.org#80669 (status was 'open')

Requestors:

• sveta.smirnova@oracle.com

From sveta.smirnova@oracle.com on 2012-11-06 20:06:50:

Originally reported by Carlos Velasco at
http://bugs.mysql.com/bug.php?id=66859

Description:
DBD::mysql leaks memory when using connect

Seems related to bug #60531 but I am seeing this with good password.

How to repeat:
Perl is 5.16.1
DBD::mysql is 4.022
DBI is 1.622
mysql is mysql-5.5.27.tar.gz

Use this perl script:

#!/usr/bin/perl

use Devel::Leak;

use DBI;
use DBD::mysql;

my $DLcount;
my $DLhandle;
my $con;

my $con = DBI->connect('DBI:mysql:database=test;host=127.0.0.1', 'root',
'', {PrintError => 0,PrintWarn=>0});

$DLcount = Devel::Leak::NoteSV($DLhandle);
print "PRE 1 count: $DLcount\n";

undef $con;
$con = DBI->connect('DBI:mysql:database=test;host=127.0.0.1', 'root',
'', {PrintError => 0,PrintWarn=>0});

$DLcount = Devel::Leak::NoteSV($DLhandle);
print "POST 1 count: $DLcount\n";

$DLcount = Devel::Leak::NoteSV($DLhandle);
print "PRE 2 count: $DLcount\n";

undef $con;
$con = DBI->connect('DBI:mysql:database=test;host=127.0.0.1', 'root',
'', {PrintError => 0,PrintWarn=>0});

$DLcount = Devel::Leak::CheckSV($DLhandle);
print "POST 2 count: $DLcount\n";

Output shows:

# perl test.pl 
PRE 1 count: 12777
POST 1 count: 12778
PRE 2 count: 12778
new 0x7cce08 : SV = NULL(0x0) at 0x7cce08
  REFCNT = 1
  FLAGS = (PADSTALE,PADMY)
new 0x743250 : SV = PVAV(0x8f38e8) at 0x743250
  REFCNT = 1
  FLAGS = (PADSTALE,PADMY)
  ARRAY = 0x0
  FILL = -1
  MAX = -1
  ARYLEN = 0x0
  FLAGS = (REAL)
new 0x743b98 : SV = NULL(0x0) at 0x743b98
  REFCNT = 1
  FLAGS = (PADSTALE,PADMY)
old (1):
 0 SV = UNKNOWN(0xff) (0x743b80) at 0x7435c8
  REFCNT = 0
  FLAGS = ()
old (1):
 0 SV = UNKNOWN(0xff) (0x7435c8) at 0x7430a0
  REFCNT = 0
  FLAGS = ()
POST 2 count: 12779

[18 Sep 15:39] Carlos Velasco

But DBD::mysql leaks test shows ok

# SLOW_TESTS=1 make test
PERL_DL_NONLAZY=1 /usr/bin/perl-64 "-MExtUtils::Command::MM" "-e"
"test_harness(0, 'blib/lib', 'blib/arch')" t/*.t
t/00base.t .................. ok   
...
t/60leaks.t ................. ok

 [18 Sep 16:00] Sveta Smirnova

This is not duplicate of bug #60531

[18 Sep 18:12] Carlos Velasco

It seems this problem has been there for long long ago:

https://bugs.launchpad.net/ubuntu/+source/libdbd-mysql-perl/+bug/51746

From michielb@cpan.org on 2016-12-01 08:33:20:

I get different output, even on 4.022.

PRE 1 count: 13067
POST 1 count: 13071
PRE 2 count: 13071
new 0x7ff0a2a45668 :
new 0x7ff0a2a45758 :
new 0x7ff0a2956878 :
old (1):
 0
old (1):
 0
POST 2 count: 13072

@Pali, could you take a look?

From pali@cpan.org on 2017-01-13 22:03:21:

On Å tv Dec 01 03:33:20 2016, MICHIELB wrote:
> I get different output, even on 4.022.
> 
> PRE 1 count: 13067
> POST 1 count: 13071
> PRE 2 count: 13071
> new 0x7ff0a2a45668 :
> new 0x7ff0a2a45758 :
> new 0x7ff0a2956878 :
> old (1):
>  0
> old (1):
>  0
> POST 2 count: 13072

See: https://metacpan.org/pod/Devel::Leak
CAVEATS: Note that you need a perl built with -DDEBUGGING for sv_dump() to print anything, but counts are valid in any perl.

So this is probably reason why you do not see dumps.

> @Pali, could you take a look?

I'm getting result:

PRE 1 count: 13033
POST 1 count: 13034
PRE 2 count: 13034
POST 2 count: 13035

So it mens that there is some leak. All new values have flags PADSTALE,PADMY except one:

new 0x1d5cdf8 : SV = IV(0x1d5cde8) at 0x1d5cdf8
  REFCNT = 1
  FLAGS = ()
  IV = 0

Which looks like that scalar leak...

I spent some time to find it, but without luck. I have no idea where it is :-(

[vartiait]

Hi,

I located the leak to DBI driver hash ChildHandles array:

DB<16> x $drh
0  DBI::dr=HASH(0x7faa1f5152f8)
   'Attribution' => 'DBD::mysql by Patrick Galbraith'
   'ChildHandles' => ARRAY(0x7faa1ea3c278)
      0  undef
      1  undef
      2  undef
      3  DBI::db=HASH(0x7faa1ea6a730)
           empty hash
   'Err' => SCALAR(0x7faa20024a18)
      -> undef
   'Errstr' => SCALAR(0x7faa20024af0)
      -> undef
   'FetchHashKeyName' => 'NAME'
   'ImplementorClass' => 'DBD::mysql::dr'
   'Name' => 'mysql'
   'State' => SCALAR(0x7faa1f59c2c0)
      -> undef
   'TraceLevel' => 0
   'Version' => 4.035
  DB<17>

the array grows every time a new connection is made as old connection handles left there undef values.

A workaround by clearing the array after undeffing a connection handle fixes the leak:

$dbh = DBI->connect("DBI:mysql:database=radiator;host=localhost;port=3306", "root", "",
 {'RaiseError' => 1});
%{$dbi_drivers} = DBI->installed_drivers() unless defined $dbi_drivers;
...
$dbh->disconnect();
undef $dbh;
@{$dbi_drivers->{mysql}->{ChildHandles}} = grep defined, @{$dbi_drivers->{mysql}->{ChildHandles}};

[vartiait]

DBI's documentation states following:

https://metacpan.org/pod/DBI#ChildHandles

The ChildHandles attribute contains a reference to an array of all the handles created by this handle which are still accessible. The contents of the array are weak-refs and will become undef when the handle goes out of scope. (They're cleared out occasionally.)

ChildHandles returns undef if your perl version does not support weak references (check the Scalar::Util module). The referenced array returned should be treated as read-only.

and according to the code, cleaning is done every 120 handles

https://github.com/perl5-dbi/dbi/blob/master/DBI.xs#L1521

[graz68a]

any chance to use DBD::mysql with MariaDB 10.2 ?

Thank you

[pali]

@graz68a There is DBD::MariaDB https://metacpan.org/release/PALI/DBD-MariaDB-0.90_01 with support for MariaDB 10.2