nayakgi / perl-compiler

Automatically exported from code.google.com/p/perl-compiler
Other
0 stars 0 forks source link

DBI_MAGIC: Missing DBD::Pg{,PP} method "connect" via package "DBI::dr" #359

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago
Unfortunately I do not have in-depth analysis for this issue at this time, as I 
encountered it while investigating a different but related problem.

Summary: Simply using DBD::PgPP in isolation causes the following issue.

To reproduce: Run the attached test program.  To ensure the validity of the 
test program versus a known working Perl 5.14.4 installation, make sure 
Postgres is installed, and that the test program is modified with appropriate 
credentials.

The error:

-- snip --
erin@erin ~/Projects/perlcc$ perlcc -r pgpp.pl
Can't locate object method "connect" via package "DBI::dr" at 
/usr/local/perl-5.14.4d-nt/lib/site_perl/5.14.4/i686-linux/DBI.pm line 671.
Can't locate object method "disconnect_all" via package "DBI::dr".
END failed--call queue aborted at 
/usr/local/perl-5.14.4d-nt/lib/5.14.4/i686-linux/List/Util.pm line 671.
-- snip --

Currently using perl-compiler/master, commit c066c38.

Expected output: Nothing.  The program should run successfully.

I've also attached compilation logs with -v5 for further perusal.

Original issue reported on code.google.com by erin.schoenhals on 11 Jul 2014 at 4:29

Attachments:

GoogleCodeExporter commented 9 years ago
Fine, but on the other side:
Why on heaven would someone use DBD::PgPP over DBD::Pg?

Original comment by reini.urban on 11 Jul 2014 at 5:49

GoogleCodeExporter commented 9 years ago
Because DBD::Pg segfaults on perl 5.6

Original comment by todd.e.rinaldo on 11 Jul 2014 at 6:08

GoogleCodeExporter commented 9 years ago
In general: We are using a compiler, so we don't need to fallback to a pure 
perl version. At all. This is nonsense. So low prio.

This compiler does not segfault when loading a XS.

However the same error appears with the XS version. It's clearly a DBI problem, 
unrelated to IO::Socket

#! perl
use strict;
use DBI;

$ENV{DBI_TRACE} = 2;
my ($db, $user, $port) = ('rurban', 'rurban', 5433);
my $dbh = DBI->connect("dbi:Pg:dbname=$db;port=$port", $user, '', 
                      {
                        'RaiseError' => 1,
                        'PrintError' => 1
                      }
  );

vs

#! perl
use strict;
use DBI;

$ENV{DBI_TRACE} = 2;
my ($db, $user, $port) = ('rurban', 'rurban', 5433);
my $dbh = 
DBI->connect("dbi:PgPP:dbname=$db;port=$port;path=/var/run/postgresql/", $user, 
'', 
                       {
                         'RaiseError' => 1,
                         'PrintError' => 1,
                       }
  );

Original comment by reini.urban on 11 Jul 2014 at 6:45

GoogleCodeExporter commented 9 years ago
IO::Socket is outside the scope of the issue I raise here, so please ignore 
that as a course of further investigation.

Original comment by erin.schoenhals on 11 Jul 2014 at 7:04

GoogleCodeExporter commented 9 years ago
Erin: If it would have been a DBD:PgPP issue alone, it would have been related 
to IO::Socket missing IO::Handle.

Original comment by reini.urban on 12 Jul 2014 at 7:53

GoogleCodeExporter commented 9 years ago

Original comment by reini.urban on 14 Jul 2014 at 5:02

GoogleCodeExporter commented 9 years ago
It might be related either to wrong compile-time AMT method overload tables.

Add in DBI.pm:663

    my $connect_closure = sub {
    my ($old_dbh, $override_attr) = @_;

        if ($ENV{DBI_TRACE} > 1) {
          require Devel::Peek;
          #warn "connect_closure: ".Devel::Peek::Dump([$attr,\%attributes, $override_attr]);
          Devel::Peek::Dump($drh);
          warn "drh: ".ref $drh, ", ISA: ",join(" ",@DBI::ISA);
        }

and see the $drh AMT magic, and compare compiled vs uncompiled with -Do

pcc -O3 -v5 -A t/pg.pl -- -Do
vs 
p -Do t/pg.pl

Or to DBI using it's own private magic extension ~ (PERL_MAGIC_ext)

=> 
SV = IV(0x2294ef0) at 0x2294f00
  REFCNT = 2
  FLAGS = (PADMY,ROK)
  RV = 0x226e810
  SV = PVHV(0x23ff420) at 0x226e810
    REFCNT = 3
    FLAGS = (OBJECT,RMG,SHAREKEYS)
    MAGIC = 0x22770b0
      MG_VIRTUAL = &PL_vtbl_pack
      MG_TYPE = PERL_MAGIC_tied(P)
      MG_FLAGS = 0x02
        REFCOUNTED
      MG_OBJ = 0x22e1218
      SV = IV(0x22e1208) at 0x22e1218
        REFCNT = 1
        FLAGS = (PADMY,ROK)
        RV = 0x226eca8
        SV = PVHV(0x23ff400) at 0x226eca8
          REFCNT = 1
          FLAGS = (OBJECT,RMG,SHAREKEYS)
          MAGIC = 0x220af30
            MG_VIRTUAL = 0
            MG_TYPE = PERL_MAGIC_ext(~)
            MG_FLAGS = 0x02
              REFCOUNTED
            MG_OBJ = 0x226e8b8
            SV = PVMG(0x21f4640) at 0x226e8b8
              REFCNT = 1
              FLAGS = (OBJECT)
              IV = 0
              NV = 0
              PV = 0x217bfb0 "\25\2\20\0\0\0\0\0\1\0\0\0\0\0\0\0\20\350&\2\0\0\0\0\20\335\241\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0@_8\2\0\0\0\0\20\335\241\0\0\0\0\0\0\0\0\0\0\0\0\0\222\6\0\0\0\0\0\0p`$\2\0\0\0\0\10\355&\2\0\0\0\0\300\354&\2\0\0\0\0P\355&\2\0\0\0\0h\355&\2\0\0\0\0\0\0\0\0\0\0\0\0P\0\0\0\0\0\0\0 \355&\2\0\0\0\0\0\0\0\0\0\0\0\0"\0
              CUR = 152
              LEN = 160
              STASH = 0x2282d78 "DBD::Pg::dr_mem"
            MG_PTR = 0x217bfb0 ""
          STASH = 0x22830c0 "DBI::dr"
          ARRAY = 0x220b350  (0:9, 1:6, 3:1)
          hash quality = 90.0%
          KEYS = 9
          FILL = 7
          MAX = 15
          RITER = -1
          EITER = 0x0
    STASH = 0x22830c0   "DBI::dr"
    ARRAY = 0x0
    KEYS = 0
    FILL = 0
    MAX = 7
    RITER = -1
    EITER = 0x0

Original comment by reini.urban on 15 Jul 2014 at 10:25

GoogleCodeExporter commented 9 years ago
Work in branch dbi_ima:

- Save XSUBANY.any_long for all XSUB's, but some use any_ptr, 
pointing to the ~ mg_ptr dbi_ima.

- This DBI_MAGIC ~ holds the dbi_ima_t struct, which stores some ptrs: stash, 
gv, and for ithreads and BROKEN_DUP_ANY_PTR my_perl also.

The stash and gv is needed to install and find DBI methods. (A horrible 
implementation. In a normal world I would expect to set the ISA correctly or at 
least use overload or tie magic to find the run-time added methods for the used 
driver.)

Original comment by reini.urban on 17 Jul 2014 at 2:35

GoogleCodeExporter commented 9 years ago

Original comment by reini.urban on 17 Jul 2014 at 2:36

GoogleCodeExporter commented 9 years ago
The DBI_MAGIC ~ does not only store the dbi_ima_t struct, it also stores all 
internal dbi structs, which are imp_drh_t (driver), imp_dbh_t (database), 
imp_sth_t (statement), imp_fdh_t (field descriptor), imp_xxh_t (any).

you get the type from the stash name with "_mem" suffix, e.g. DBI::Pg::dr_mem 
is a imp_drh_t.

Certain parts of these structs need to be specialized at init2, e.g. the ptrs 
to the gv or stash, ...

Original comment by reini.urban on 17 Jul 2014 at 3:24

GoogleCodeExporter commented 9 years ago
The best fix is this patch for DBI.pm:

iff --git DBI.pm DBI.pm
index 2f84796..9efc7c5 100644
--- DBI.pm
+++ DBI.pm
@@ -506,7 +506,8 @@ my $keeperr = { O=>0x0004 };
     },
 );

-while ( my ($class, $meths) = each %DBI::DBI_methods ) {
+INIT {
+  while ( my ($class, $meths) = each %DBI::DBI_methods ) {
     my $ima_trace = 0+($ENV{DBI_IMA_TRACE}||0);
     while ( my ($method, $info) = each %$meths ) {
        my $fullmeth = "DBI::${class}::$method";
@@ -518,6 +519,7 @@ while ( my ($class, $meths) = each %DBI::DBI_methods ) {
        }
        DBI->_install_method($fullmeth, 'DBI.pm', $info);
     }
+  }
 }

Original comment by reini.urban on 18 Jul 2014 at 3:00

GoogleCodeExporter commented 9 years ago
This patch looks fine except it throws nasty runtime errors if DBI is loaded 
after INIT. Any way we could change the patch to work in both cases?

Original comment by todd.e.rinaldo on 18 Jul 2014 at 3:31

GoogleCodeExporter commented 9 years ago
Yes, this patch is just a dirty hack, and does not pass the DBI testsuite, and 
does not work in all scenarios.

I'm working in a perlcc specific solution to detect DBI and add the missing 
post-dl_init initializations by myself. The driver (e.g. DBD::Pg) can by 
compiled in or not. Usually not, but relying on that is too fragile.

Original comment by reini.urban on 18 Jul 2014 at 3:25

GoogleCodeExporter commented 9 years ago
Testing now the fix commit ef491cbcca76998124f6315c16a6930e9a1fbf84
Author: Reini Urban <rurban@cpanel.net>
Date:   Thu Jul 17 12:41:59 2014 -0500

    C 1.49_01: fix DBI #359

    special case DBI_MAGIC methods,
    add an extra call to DBI->_install_method to re-initialize compile-time
    internal ima pointers. ('~' DBI_MAGIC)

    also store XSUBANY.any_long and use a heuristic to skip storing
    compile-time pointers. storing ptrs broke Tie::Hash::NamedCapture (90)
    also add example t/pg.pl: DBI_TRACE=0 t/pg

Original comment by reini.urban on 18 Jul 2014 at 8:25

GoogleCodeExporter commented 9 years ago
here is a DBI patch that would avoid warnings

diff --git a/modules/DBI/DBI/DBI.pm b/modules/DBI/DBI/DBI.pm
index 2e9f6fe..92b8b9e 100644
--- a/modules/DBI/DBI/DBI.pm
+++ b/modules/DBI/DBI/DBI.pm
@@ -499,6 +499,8 @@ my $keeperr = { O=>0x0004 };
     },
 );

+# wrap the code in a sub
+sub init_install_method {
 while ( my ($class, $meths) = each %DBI::DBI_methods ) {
     my $ima_trace = 0+($ENV{DBI_IMA_TRACE}||0);
     while ( my ($method, $info) = each %$meths ) {
@@ -512,6 +514,16 @@ while ( my ($class, $meths) = each %DBI::DBI_methods ) {
        DBI->_install_method($fullmeth, 'DBI.pm', $info);
     }
 }
+} # end sub init_install_method
+# install methods at INIT time when compiled
+# view https://code.google.com/p/perl-compiler/issues/detail?id=359
+if ( !${^GLOBAL_PHASE} || ${^GLOBAL_PHASE} eq 'START' ) {
+  eval q/INIT { init_install_method() }/;
+} else {
+  init_install_method();
+}
+
+

 {
     package DBI::common;

Original comment by nicolas....@gmail.com on 22 Jul 2014 at 4:58

GoogleCodeExporter commented 9 years ago
note that such a patch cannot be submitted to DBI
as you will not be able to establish a db connection during a begin block...
which in most of the case will be a non sense... except if you want to read 
some values from a DB during the compilation in order to use them later

Original comment by nicolas....@gmail.com on 22 Jul 2014 at 5:28