search

bucardo / dbdpg

Perl Postgres driver DBD::Pg aka dbdpg
48 stars 36 forks source link

Compile error 'initdb' not recognized for ver 3.15.1 #96

Open Brian2099 opened 2 years ago

Brian2099 commented 2 years ago

Hi There,

I'm running Strawberry Perl on a Windows 10, 64 bit environment and am getting this error: Bad initdb output: 'initdb' is not recognized as an internal or external command,

This compile error occurs on multiple CPUs with slightly different versions of Strawberry Perl, 5.30.0 and 5.32.1 (latest version).

I did install the most up-to-date dependencies DBI and version, so that wasn't the issue.

I am running the command prompt as an administrator. I even tried using the force option, but the compile failed with the same error.

Many thanks!

C:\Strawberry\perl\bin\perl.exe" -MExtUtils::Command -e chmod -- 755 blib\arch\auto\DBD\Pg\Pg.xs.dll TURNSTEP/DBD-Pg-3.15.1.tar.gz C:\STRAWB~1\c\bin\gmake.exe -- OK Running make test for TURNSTEP/DBD-Pg-3.15.1.tar.gz "C:\Strawberry\perl\bin\perl.exe" -MExtUtils::Command::MM -e cp_nonempty -- Pg.bs blib\arch\auto\DBD\Pg\Pg.bs 644 "C:\Strawberry\perl\bin\perl.exe" "-MExtUtils::Command::MM" "-MTest::Harness" "-e" "undef Test::Harness::Switches; test_harness(0, 'blib\lib', 'blib\arch')" t/.t t/00_signature.t ....... skipped: Set the environment variable TEST_SIGNATURE to enable this test t/00basic.t ............ ok t/01connect.t .......... #

DBI Version 1.643

DBD::Pg Version 3.15.1

Perl Version 5.30.0

OS MSWin32

PostgreSQL (compiled) ?

PostgreSQL (target) ?

PostgreSQL (reported) ?

Default port ?

DBI_DSN ?

DBI_USER

Test schema dbd_pg_testschema

LANG C

Adjusted: initdb

Error was: Bad initdb output: 'initdb' is not recognized as an internal or external command, <-----------

operable program or batch file.

# Bailout called. Further testing stopped: Cannot continue: connection failed FAILED--Further testing stopped: Cannot continue: connection failed gmake: *** [Makefile:986: test_dynamic] Error 255 TURNSTEP/DBD-Pg-3.15.1.tar.gz C:\STRAWB~1\c\bin\gmake.exe test -- NOT OK //hint// to see the cpan-testers results for installing this module, try: reports TURNSTEP/DBD-Pg-3.15.1.tar.gz Stopping: 'install' fail

esabol commented 2 years ago

I know it's been 3 months, and you've probably already found a solution. I think you need to configure your PATH. You need to install the PostgreSQL tools and get them usable from the command line first. This StackOverflow seems to cover how to do this:

https://stackoverflow.com/questions/30401460/postgres-psql-not-recognized-as-an-internal-or-external-command

Brian2099 commented 2 years ago

esabol,

Thank you for responding. FYI, none of my development computers have ever had Postgres installed. All CPAN modules have installed fine since 2015 until the most recent update 90 days ago. I did confirm that the %PATH% does not contain the C:\Program Files\PostgreSQL\ folders. However, that was expected because I've never installed PG on my development computers. I installed Strawberry Perl on my development computers and then programmatically connect at run-time to the Posgres database server.

The temporary (hopefully) workaround was to install the module with the -T option (Do not test modules. Simply install them.). I only used this override when installing the DBD::Pg module.

I'm still wondering why the installs worked for 7 years and all of a sudden abend with an error. I really don't think installing Postgres on my development computers is the answer to the above issue.

Many thanks.

esabol commented 2 years ago

I'm still wondering why the installs worked for 7 years and all of a sudden abend with an error. I really don't think installing Postgres on my development computers is the answer to the above issue.

Well, it wasn't clear from your initial message that that's what your issue was about. I thought you just wanted a solution to the error, and I believe installing PostgreSQL would solve the problem you encountered with running these tests.

I'm surprised to learn it was ever possible to install DBD::Pg without installing PostgreSQL. I could have sworn you needed at least some PostgreSQL library (DLL on Windows) and some PostgreSQL C headers to compile the DBD::Pg source code, but perhaps I'm mistaken. I've only ever compiled, tested, and installed DBD::Pg on computers that also have PostgreSQL installed.

PhilterPaper commented 2 years ago

I may be experiencing a very similar problem to the OP's. It's Windows 10 with Strawberry Perl 5.24, 5.26, and 5.32. I don't believe that PostgreSQL is installed on my PC. All I'm interested in is avoiding this error message (and the upgrade failure) when attempting to upgrade my installed packages (DBD::Pg comes preinstalled with Strawberry). I do have a wrapper around the cpan upgrade process to avoid trying to update DBD::Pg, but for the benefit of folks who haven't rolled such a tool, it would be good to be able to run a general "upgrade [all]" and not have to worry about it dying in the middle of multiple upgrades from some package like DBD::Pg.

This would probably involve changes to CPAN so that its tool could query the installation state of a subsystem (such as PostgreSQL) and what version it is, before deciding that the DBD::Pg package needs to be upgraded. In my case, it would see that PostgreSQL isn't even installed, and so wouldn't even try to upgrade DBD::Pg. I reported something similar against DBD::Oracle, where the client is too old to bother updating the package, and B, which only matches a given Perl level.

shawnlaffan commented 5 months ago

Strawberry Perl provides a reduced set of postgres files, and this does not include initdb, pg_ctl, pg_config and a few others.
https://github.com/StrawberryPerl/build-extlibs/blob/d22291c1a0de960466d1983cb0af88bff8c974a7/build.sh#L819-L844

The tests started to fail some time after DBD::Pg 3.8.0, which is the version distributed with recent Strawberry Perls since the tests pass.

I've run some full postgres builds but am getting DBD::Pg failures related to dict_snowball - it seems the $libdir config value is not being populated or parsed correctly. I'll post more details here or in a new issue when I can provide reproduction steps and files but some hopefully useful info is below for now. FWIW, dict_snowball.dll lives in PKGLIBDIR.

FATAL:  could not access file "$libdir/dict_snowball": No such file or directory
> pg_config --libdir
C:/perls/STRAWB~1.1-R/c/lib
> pg_config --pkglibdir
C:/perls/STRAWB~1.1-R/c/lib/POSTGR~1
shawnlaffan commented 5 months ago

I've now run a world-bin build of postgresql using the Strawberry perl build system.

The set of packed files are at https://github.com/StrawberryPerl/build-extlibs/releases/download/gcc13.2_ucrt_posix/64bit_postgresql-16.2-bin_20240613_world_bin.zip . Not all are packed so some might still be needed.

This build uses the compiler setup used for SP 5.40-RC1. That SP release can be downloaded from https://github.com/StrawberryPerl/Perl-Dist-Strawberry/releases/tag/dev_5.40.0_RC1_UCRT

I can get further than before but now hit issues with database creation in t\01connect.t:

# DBD::Pg                     Version 3.18.0
# Perl                        Version 5.40.0
# OS                          MSWin32
# PostgreSQL (compiled)       ?
# PostgreSQL (target)         ?
# PostgreSQL (reported)       ?
# Default port                ?
# DBI_DSN                     ?

# DBI_USER                    <not set>
# Test schema                 dbd_pg_testschema
# Adjusted:                   initdb
# Error was: Previous failure (Could not startup new database (pg_ctl -o '-k C:/perls/dbdpg_exp/DBD-Pg-3.18.0/dbdpg_test_database/data/socket' -l C:/perls/dbdpg_exp/DBD-Pg-3.18.0/dbdpg_test_database/dbdpg_test.logfile -D C:/perls/dbdpg_exp/DBD-Pg-3.18.0/dbdpg_test_database/data start) () ())
Bailout called.  Further testing stopped:  Cannot continue: connection failed
Bail out!  Cannot continue: connection failed
skipped: Connection to database failed, cannot continue testing

The single quotes around the -k argument look suspicious to me. The Cmd shell prefers double quotes. Changing them and running from the Cmd prompt does not change the result, though.

To reproduce:

  1. Install SP 5.40-RC1 portable somewhere on your system. Use a short path as otherwise the test db names are too long under cpanm.
  2. Copy the c dir contents from the zip file into the Strawberry Perl c dir, overwriting as needed.
  3. Download and build the DBD::Pg distribution. I initially downloaded using cpanm --look but then copied the contents to a shorter path.
  4. Run the tests.

The log file contents are:

2024-06-13 15:02:33.664 AEST [32956] LOG:  starting PostgreSQL 16.2 on x86_64-w64-mingw32, compiled by x86_64-w64-mingw32-gcc.exe (MinGW-W64 x86_64-ucrt-posix-seh, built by Brecht Sanders, r8) 13.2.0, 64-bit
2024-06-13 15:02:33.667 AEST [32956] LOG:  listening on IPv4 address "127.0.0.1", port 5440
2024-06-13 15:02:33.670 AEST [32956] FATAL:  could not create lock file "C:/perls/dbdpg_exp/DBD-Pg-3.18.0/dbdpg_test_database/data/socket/.s.PGSQL.5440.lock": No such file or directory
2024-06-13 15:02:33.671 AEST [32956] LOG:  database system is shut down
2024-06-13 15:03:21.300 AEST [27136] LOG:  starting PostgreSQL 16.2 on x86_64-w64-mingw32, compiled by x86_64-w64-mingw32-gcc.exe (MinGW-W64 x86_64-ucrt-posix-seh, built by Brecht Sanders, r8) 13.2.0, 64-bit
2024-06-13 15:03:21.303 AEST [27136] LOG:  listening on IPv4 address "127.0.0.1", port 5440
2024-06-13 15:03:21.306 AEST [27136] FATAL:  could not create lock file "C:/perls/dbdpg_exp/DBD-Pg-3.18.0/dbdpg_test_database/data/socket/.s.PGSQL.5440.lock": No such file or directory
2024-06-13 15:03:21.307 AEST [27136] LOG:  database system is shut down
2024-06-13 15:03:45.471 AEST [22240] LOG:  starting PostgreSQL 16.2 on x86_64-w64-mingw32, compiled by x86_64-w64-mingw32-gcc.exe (MinGW-W64 x86_64-ucrt-posix-seh, built by Brecht Sanders, r8) 13.2.0, 64-bit
2024-06-13 15:03:45.473 AEST [22240] LOG:  listening on IPv4 address "127.0.0.1", port 5440
2024-06-13 15:03:45.476 AEST [22240] FATAL:  could not create lock file "C:/perls/dbdpg_exp/DBD-Pg-3.18.0/dbdpg_test_database/data/socket/.s.PGSQL.5440.lock": No such file or directory
2024-06-13 15:03:45.478 AEST [22240] LOG:  database system is shut down
esabol commented 5 months ago

@shawnlaffan wrote:

The single quotes around the -k argument look suspicious to me. The Cmd shell prefers double quotes. Changing them and running from the Cmd prompt does not change the result, though.

First off, thank you for supporting DBD::Pg in Strawberry Perl!

I suspect your problem is in t/dbdpg_test_setup.pl around here:

https://github.com/bucardo/dbdpg/blob/f3d3c928da874f48e30e2681db8d79361d40fe30/t/dbdpg_test_setup.pl#L565-L576

The if ($^O !~ /Win32/) { test implies that a socket directory should not be specified on Windows.

What happens if you drop the -o '-k ...' entirely and execute pg_ctl -l C:/perls/dbdpg_exp/DBD-Pg-3.18.0/dbdpg_test_database/dbdpg_test.logfile -D C:/perls/dbdpg_exp/DBD-Pg-3.18.0/dbdpg_test_database/data start?

In other words, change line 576 to my $COM = qq{$pg_ctl -l $testdir/dbdpg_test.logfile -D $testdir/data start};.

Alternatively, you might also try changing line 565 to if (1) { to force the setup of the socket directory on Win32. My googling shows that Windows support for sockets was added to PostgreSQL many years ago, so maybe it would work if the socket directory was setup properly (i.e. created)?

shawnlaffan commented 5 months ago

Dropping the socket component seems to work from the cmd prompt:

> pg_ctl -l C:/perls/dbdpg_exp/DBD-Pg-3.18.0/dbdpg_test_database/dbdpg_test.logfile -D C:/perls/dbdpg_exp/DBD-Pg-3.18.0/dbdpg_test_database/data start
waiting for server to start.... done
server started

I also changed the single quotes to double quotes in both places they are used (L191 and L576) but then discovered pg_resetwal.exe needs to be packed. I'm rebuilding and repacking everything at the moment.

shawnlaffan commented 5 months ago

We have progress now all the files are packed. https://github.com/StrawberryPerl/build-extlibs/releases/download/gcc13.2_ucrt_posix/64bit_postgresql-16.2-bin_20240613_dll_suffix_wf.zip

Removing the socket command seems to result in a hang so I left it in, but double quoted.

t/00_signature.t ....... skipped: Set the environment variable TEST_SIGNATURE to enable this test
t/00basic.t ............ ok
t/01connect.t .......... Please wait, creating new database (version 16.2) for testing
t/01connect.t .......... 29/30 #
# DBI                         Version 1.643
# DBD::Pg                     Version 3.18.0
# Perl                        Version 5.40.0
# OS                          MSWin32
# PostgreSQL (compiled)       160002
# PostgreSQL (target)         160002
# PostgreSQL (reported)       PostgreSQL 16.2 on x86_64-w64-mingw32, compiled by x86_64-w64-mingw32-gcc.exe (MinGW-W64 x86_64-ucrt-posix-seh, built by Brecht Sanders, r8) 13.2.0, 64-bit
# Default port                5432
# DBI_DSN                     dbi:Pg:db="postgres";port=5440;host=localhost
# DBI_USER                    z9803549
# Test schema                 dbd_pg_testschema
# LANG                        C
# Adjusted:                   initdb
t/01connect.t .......... ok
t/01constants.t ........ ok
t/02attribs.t .......... 245/293 DBD::Pg::db commit failed: no connection to the server at t/02attribs.t line 1449.
# Looks like your test exited with 255 just after 246.
t/02attribs.t .......... Dubious, test returned 255 (wstat 65280, 0xff00)
Failed 47/293 subtests
        (less 1 skipped subtest: 245 okay)
t/03dbmethod.t ......... 1/?
t/03dbmethod.t ......... 5/? #   Failed test 'DB handle method "last_insert_id" fails when called in a failed transaction'
#   at t/03dbmethod.t line 92.
#          got: '08000'
#     expected: '25P02'
DBD::Pg::db rollback failed: no connection to the server at t/03dbmethod.t line 95.
# Tests were run but no plan was declared and done_testing() was not seen.
# Looks like your test exited with 255 just after 5.
t/03dbmethod.t ......... Dubious, test returned 255 (wstat 65280, 0xff00)
Failed 1/5 subtests
t/03smethod.t .......... ok
t/04misc.t ............. 108/109 Error removing C:\perls\dbdpg_exp\DBD-Pg-3.18.0\dbdpg_test_DqQCXB.tst at C:/perls/strawberry-perl-5.40.0.1-RC1-64bit-portable/perl/lib/File/Temp.pm line 899.
t/04misc.t ............. ok
t/06bytea.t ............ ok
t/07copy.t ............. ok
t/08async.t ............ ok
t/09arrays.t ........... ok
t/10_pg_error_field.t .. ok
t/12placeholders.t ..... ok
t/20savepoints.t ....... ok
t/30unicode.t .......... ok
t/99cleanup.t .......... ok

Test Summary Report
-------------------
t/02attribs.t        (Wstat: 65280 (exited 255) Tests: 246 Failed: 0)
  Non-zero exit status: 255
  Parse errors: Bad plan.  You planned 293 tests but ran 246.
t/03dbmethod.t       (Wstat: 65280 (exited 255) Tests: 5 Failed: 1)
  Failed test:  5
  Non-zero exit status: 255
  Parse errors: No plan found in TAP output
Files=17, Tests=2345, 158 wallclock secs ( 0.11 usr +  0.01 sys =  0.12 CPU)
Result: FAIL
Failed 2/17 test programs. 1/2345 subtests failed.
gmake: *** [makefile:989: test_dynamic] Error 255
shawnlaffan commented 5 months ago

Although now it is failing again so I am not sure what caused the previous pass. 

> prove -vb
t/00_signature.t ....... skipped: Set the environment variable TEST_SIGNATURE to enable this test
t/00basic.t ............ ok
t/01connect.t .......... ----
#
# DBI                         Version 1.643
# DBD::Pg                     Version 3.18.0
# Perl                        Version 5.40.0
# OS                          MSWin32
# PostgreSQL (compiled)       ?
# PostgreSQL (target)         ?
# PostgreSQL (reported)       ?
# Default port                ?
# DBI_DSN                     ?
# DBI_USER                    <not set>
# Test schema                 dbd_pg_testschema
# Adjusted:                   initdb
# Error was: connection to server at "localhost" (::1), port 5440 failed: Connection refused (0x0000274D/10061)
Bailout called.  Further testing stopped:  Cannot continue: connection failed
t/01connect.t .......... skipped: Connection to database failed, cannot continue testing
Can't use an undefined value as a subroutine reference at C:/perls/strawberry-perl-5.40.0.1-RC1-64bit-portable/perl/lib/TAP/Harness.pm line 612.
gmake: *** [makefile:989: test_dynamic] Error 255

Edit: And some log details.

2024-06-13 19:39:04.611 AEST [8680] LOG:  starting PostgreSQL 16.2 on x86_64-w64-mingw32, compiled by x86_64-w64-mingw32-gcc.exe (MinGW-W64 x86_64-ucrt-posix-seh, built by Brecht Sanders, r8) 13.2.0, 64-bit
2024-06-13 19:39:04.614 AEST [8680] LOG:  listening on IPv4 address "127.0.0.1", port 5440
2024-06-13 19:39:04.617 AEST [8680] FATAL:  could not create lock file "C:/perls/dbdpg_exp/dbdpg/dbdpg_test_database/data/socket/.s.PGSQL.5440.lock": No such file or directory
2024-06-13 19:39:04.618 AEST [8680] LOG:  database system is shut down