raku-community-modules / DBIish

Database interface for Raku
https://raku.land/zef:raku-community-modules/DBIish
89 stars 31 forks source link

Alpine os failure - DBDish::SQLite: Can't connect: SQLITE_CANTOPEN (14) #190

Closed melezhik closed 4 years ago

melezhik commented 4 years ago

Hi! I could ses that DBIish tests fail on alpine:

https://rakudist.raku.org/sparky/report/alpine/265

15:45:41 06/10/2020 [bash: zef install .] ===> Testing: DBIish:ver<0.5.19>
15:46:30 06/10/2020 [bash: zef install .] [DBIish] Use of uninitialized value element of type Any in string context.
15:46:30 06/10/2020 [bash: zef install .] [DBIish] Methods .^name, .raku, .gist, or .say can be used to stringify it to something meaningful.
15:46:30 06/10/2020 [bash: zef install .] [DBIish]   in block <unit> at t/03-lib-util.t line 16
15:46:35 06/10/2020 [bash: zef install .] [DBIish] # DBIish: DBDish::mysql needs '(null)', not found
15:46:35 06/10/2020 [bash: zef install .] [DBIish] # Can't continue.
15:46:38 06/10/2020 [bash: zef install .] [DBIish] # Testing DBDish::mysql
15:46:39 06/10/2020 [bash: zef install .] [DBIish] # DBIish: DBDish::mysql needs '(null)', not found
15:46:39 06/10/2020 [bash: zef install .] [DBIish] # Can't continue.
15:46:39 06/10/2020 [bash: zef install .] [DBIish] # DBIish: DBDish::mysql needs '(null)', not found
15:46:39 06/10/2020 [bash: zef install .] [DBIish] # Can't continue.
15:46:40 06/10/2020 [bash: zef install .] [DBIish] # DBIish: DBDish::mysql needs '(null)', not found
15:46:40 06/10/2020 [bash: zef install .] [DBIish] # Can't continue.
15:46:41 06/10/2020 [bash: zef install .] [DBIish] # DBIish: DBDish::mysql needs '(null)', not found
15:46:41 06/10/2020 [bash: zef install .] [DBIish] # Can't continue.
15:46:42 06/10/2020 [bash: zef install .] [DBIish] # DBIish: DBDish::Pg needs '(null)', not found
15:46:42 06/10/2020 [bash: zef install .] [DBIish] # Can't continue.
15:46:42 06/10/2020 [bash: zef install .] [DBIish] # Testing DBDish::Pg
15:46:43 06/10/2020 [bash: zef install .] [DBIish] # DBIish: DBDish::Pg needs '(null)', not found
15:46:43 06/10/2020 [bash: zef install .] [DBIish] # Can't continue.
15:46:44 06/10/2020 [bash: zef install .] [DBIish] # DBIish: DBDish::Pg needs '(null)', not found
15:46:44 06/10/2020 [bash: zef install .] [DBIish] # Can't continue.
15:46:46 06/10/2020 [bash: zef install .] [DBIish] # DBIish: DBDish::Pg needs '(null)', not found
15:46:46 06/10/2020 [bash: zef install .] [DBIish] # Can't continue.
15:46:46 06/10/2020 [bash: zef install .] [DBIish] # DBIish: DBDish::Pg needs '(null)', not found
15:46:46 06/10/2020 [bash: zef install .] [DBIish] # Can't continue.
15:46:47 06/10/2020 [bash: zef install .] [DBIish] # DBIish: DBDish::Pg needs '(null)', not found
15:46:47 06/10/2020 [bash: zef install .] [DBIish] # Can't continue.
15:46:48 06/10/2020 [bash: zef install .] [DBIish] # DBIish: DBDish::Pg needs '(null)', not found
15:46:48 06/10/2020 [bash: zef install .] [DBIish] # Can't continue.
15:46:49 06/10/2020 [bash: zef install .] [DBIish] # DBIish: DBDish::Pg needs '(null)', not found
15:46:49 06/10/2020 [bash: zef install .] [DBIish] # Can't continue.
15:46:49 06/10/2020 [bash: zef install .] [DBIish] # DBIish: DBDish::Pg needs '(null)', not found
15:46:49 06/10/2020 [bash: zef install .] [DBIish] # Can't continue.
15:46:49 06/10/2020 [bash: zef install .] [DBIish] # DBIish: DBDish::Pg needs '(null)', not found
15:46:49 06/10/2020 [bash: zef install .] [DBIish] # Can't continue.
15:46:49 06/10/2020 [bash: zef install .] [DBIish] # DBIish: DBDish::Pg needs '(null)', not found
15:46:49 06/10/2020 [bash: zef install .] [DBIish] # Can't continue.
15:46:49 06/10/2020 [bash: zef install .] [DBIish] # DBIish: DBDish::Pg needs '(null)', not found
15:46:49 06/10/2020 [bash: zef install .] [DBIish] # Can't continue.
15:47:02 06/10/2020 [bash: zef install .] [DBIish] # Testing DBDish::SQLite
15:47:03 06/10/2020 [bash: zef install .] [DBIish] # Testing DBDish::SQLite
15:47:03 06/10/2020 [bash: zef install .] [DBIish] # DBDish::SQLite: Can't connect: SQLITE_CANTOPEN (14)
15:47:03 06/10/2020 [bash: zef install .] [DBIish] # Can't continue.
15:47:04 06/10/2020 [bash: zef install .] [DBIish] DBDish::SQLite: Can't connect: SQLITE_CANTOPEN (14)
15:47:04 06/10/2020 [bash: zef install .] [DBIish]   in method connect at /home/vlskirbufgeypqozwmxj/.zef/store/DBIish.git/d0c1d058f9a2d365cc0b751f2f7e9933bee4a0e0/lib/DBDish/SQLite.pm6 (DBDish::SQLite) line 36
15:47:04 06/10/2020 [bash: zef install .] [DBIish]   in method connect at /home/vlskirbufgeypqozwmxj/.zef/store/DBIish.git/d0c1d058f9a2d365cc0b751f2f7e9933bee4a0e0/lib/DBIish.pm6 (DBIish) line 52
15:47:04 06/10/2020 [bash: zef install .] [DBIish]   in block <unit> at t/48-sqlite-errors.t line 23
15:47:04 06/10/2020 [bash: zef install .] [DBIish] # You planned 17 tests, but ran 4
15:47:05 06/10/2020 [bash: zef install .] [DBIish] # Testing DBDish::Oracle
15:47:06 06/10/2020 [bash: zef install .] ===> Testing [FAIL]: DBIish:ver<0.5.19>
15:47:06 06/10/2020 [bash: zef install .] stderr: Aborting due to test failure: DBIish:ver<0.5.19> (use --force-test to override)
15:47:06 06/10/2020 [bash: zef install .] task exit status: 1
15:47:06 06/10/2020 [bash: zef install .] task bash: zef install . FAILED
JJ commented 4 years ago

This dockerizer creates automatically an Alpine container that includes DBIish and associated libraries. I think the gist of it here is to install a few libraries, mainly sqlite. This is the result here

melezhik commented 4 years ago

@JJ thanks for that. The issue is not with how to install DBIsh but with it failure (zef test) when sqlite is installed:

My system setup for external libraries could be found here:

https://github.com/melezhik/RakuDist/blob/master/modules/default-github/files/patches/FCO/Red/sparrowfile

rbt commented 4 years ago

I'm not able to confirm the issue. Do you have a link to the build script?

I tried with SQLite both uninstalled and installed.

The SQLITE_CANTOPEN lines are interesting. Is it trying to open the database file in a read-only filesystem or a missing path?

# docker run -it --entrypoint sh -l -c jjmerelo/alpine-raku

/home/raku # raku --version
This is Rakudo version 2020.05 built on MoarVM version 2020.05
implementing Raku 6.d.

## Needed for NativeHelpers::Blob
/home/raku # apk add build-base

/home/raku # zef install DBIish

<!-- SNIP -->
===> Installing: DBIish:ver<0.5.19>

/home/raku # apk install sqlite-libs

/home/raku # zef uninstall DBIish
===> Uninstalled from /root/raku-install/share/perl6/site
DBIish:ver<0.5.19>

/home/raku # zef install DBIish
===> Searching for: DBIish
===> Testing: DBIish:ver<0.5.19>
<!-- SNIP -->
[DBIish] # Testing DBDish::SQLite
[DBIish] # Testing DBDish::SQLite
[DBIish] # Testing DBDish::Oracle
===> Testing [OK] for DBIish:ver<0.5.19>
===> Installing: DBIish:ver<0.5.19>
melezhik commented 4 years ago

@rbt I am going to run DBIIsh only test with sqlite installed on alpine soon and give you a link.

melezhik commented 4 years ago

@rbt here is full log - https://rakudist.raku.org/sparky/report/alpine/271

I am trying to zef install DBIish with sqlite installed on alpine.

System package configuration is defined here - https://github.com/melezhik/RakuDist/blob/master/modules/default/files/patches/DBIish/sparrowfile

Testing scenario is defined here - https://github.com/melezhik/RakuDist/blob/master/modules/default/sparrowfile

It's nothing but simple zef install with user installation (note root), also this is executed on docker container which could be a hint for SQLITE_CANTOPEN bit, I don't know ...

We could enable more logging if need but it'll scenario modification (passing more options to zef to make it more verbose ).

melezhik commented 4 years ago

Interesting that for all 3 others OS (debian/centos/ununtu) DBIish tests succeed. With the same configuration (run on docker, the same Rakudo version and sqlitelib installed as a system package)

https://rakudist.raku.org/sparky/builds

We only have a problem with alpine here ...

JJ commented 4 years ago

Let me see if I can find what's wrong... But today I'm AFK.

El mié., 10 jun. 2020 a las 19:04, Alexey Melezhik (< notifications@github.com>) escribió:

Interesting that for all 3 others OS (debian/centos/ununtu) DBIish tests succeed. With the same configuration (run on docker, the same Rakudo version and sqlitelib installed as a system package)

https://rakudist.raku.org/sparky/builds

We only have a problem with alpine here ...

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/raku-community-modules/DBIish/issues/190#issuecomment-642140088, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAAD5GMD4OQBK523GSDV3TRV64KVANCNFSM4N2QN67A .

-- JJ

rbt commented 4 years ago

@melezhik Is it possible to run zef install --debug DBIish ?

If I'm counting right it seems to be failing on a basic connect test which does nothing more than create an empty "exec-empty.sqlite3" file in $CWD. Perhaps $CWD isn't writable but why only on alpine.

my $dbfile = 'exec-error';
ok (my $dbh =  DBIish.connect('SQLite', dbname => $dbfile)), 'Created';
melezhik commented 4 years ago

I've adjusted test script, now one can see CWD permissions (nothing suspicious) and zef install in debug mode:

https://rakudist.raku.org/sparky/report/alpine/279

melezhik commented 4 years ago
19:08:43 06/10/2020 [bash: zef cwd dir [/home/khfxjpnmrlzswbyovutg/.zef/store ...] total 40
19:08:43 06/10/2020 [bash: zef cwd dir [/home/khfxjpnmrlzswbyovutg/.zef/store ...] -rw-r--r--    1 khfxjpnm khfxjpnm       245 Jun 10 19:08 CREDITS
19:08:43 06/10/2020 [bash: zef cwd dir [/home/khfxjpnmrlzswbyovutg/.zef/store ...] -rw-r--r--    1 khfxjpnm khfxjpnm      2510 Jun 10 19:08 META6.json
19:08:43 06/10/2020 [bash: zef cwd dir [/home/khfxjpnmrlzswbyovutg/.zef/store ...] -rw-r--r--    1 khfxjpnm khfxjpnm     15536 Jun 10 19:08 README.pod
19:08:43 06/10/2020 [bash: zef cwd dir [/home/khfxjpnmrlzswbyovutg/.zef/store ...] drwxr-sr-x    2 khfxjpnm khfxjpnm      4096 Jun 10 19:08 examples
19:08:43 06/10/2020 [bash: zef cwd dir [/home/khfxjpnmrlzswbyovutg/.zef/store ...] drwxr-sr-x    4 khfxjpnm khfxjpnm      4096 Jun 10 19:08 lib
19:08:43 06/10/2020 [bash: zef cwd dir [/home/khfxjpnmrlzswbyovutg/.zef/store ...] drwxr-sr-x    2 khfxjpnm khfxjpnm      4096 Jun 10 19:08 t
19:08:43 06/10/2020 [bash: zef cwd dir [/home/khfxjpnmrlzswbyovutg/.zef/store ...] drwxr-sr-x    2 khfxjpnm khfxjpnm      4096 Jun 10 19:08 xt
rbt commented 4 years ago

Interesting.

t/43-sqlite-threads.t passes. It creates dbdish-sqlite-test-threaded.sqlite3, writes to it, pulls data out, then deletes it after execution.

t/46-sqlite-blob.t fails but skips the tests as the connection isn't valid. The DB file is different than the on in the thread test.

t/48-sqlite-errors.t fails and bails, again with DB file of a different name.

I'm stuck; can't think of any reason why 43 would pass while 46/48 fail to create the sqlite3 file.

abraxxa commented 4 years ago

Out of file handles? Disk full? Does SQLite pass on the file handle error?

melezhik commented 4 years ago

feel free to ask me guys, I can log into alpine docker and do any troubleshooting.

$ docker exec -i -t alpine-rakudist /bin/bash
bash-5.0# df -h
Filesystem                Size      Used Available Use% Mounted on
/dev/sda2               914.3G    126.0G    741.8G  15% /
tmpfs                    64.0M         0     64.0M   0% /dev
/dev/sda2               914.3G    126.0G    741.8G  15% /root/repo
tmpfs                     6.3G    252.0K      6.3G   0% /etc/hosts
shm                      62.5M         0     62.5M   0% /dev/shm
tmpfs                     6.3G    252.0K      6.3G   0% /etc/hostname
tmpfs                     6.3G    252.0K      6.3G   0% /run/.containerenv
tmpfs                     6.3G    252.0K      6.3G   0% /etc/resolv.conf
tmpfs                    31.4G         0     31.4G   0% /sys/fs/cgroup
udev                     31.4G         0     31.4G   0% /dev/null
udev                     31.4G         0     31.4G   0% /dev/random
udev                     31.4G         0     31.4G   0% /dev/full
udev                     31.4G         0     31.4G   0% /dev/tty
udev                     31.4G         0     31.4G   0% /dev/zero
udev                     31.4G         0     31.4G   0% /dev/urandom
tmpfs                    31.4G         0     31.4G   0% /proc/acpi
udev                     31.4G         0     31.4G   0% /proc/kcore
udev                     31.4G         0     31.4G   0% /proc/keys
udev                     31.4G         0     31.4G   0% /proc/timer_list
udev                     31.4G         0     31.4G   0% /proc/sched_debug
tmpfs                    31.4G         0     31.4G   0% /proc/scsi
tmpfs                    31.4G         0     31.4G   0% /sys/firmware
melezhik commented 4 years ago

@abraxxa I don't think these are the cases you've mentioned or all other modules tests would fail ...

melezhik commented 4 years ago

@rbt as an option you might want to run DBIIsh tests from github repo strait, using RakuDist. You may adjust tests make them more debuggable

melezhik commented 4 years ago

this is how one can run for GitHub repo, master branch - https://rakudist.raku.org/sparky/report/alpine/281 the same error is here ...

melezhik commented 4 years ago

Someone run for dbiish

20:26:20 06/10/2020 [zef fetch dbiish] ===> Searching for: dbiish
20:26:31 06/10/2020 [zef fetch dbiish] Failed to resolve any candidates. No reason to proceed

it should be DBIish

melezhik commented 4 years ago

so to run against GitHub need to set module name as https://github.com/raku-community-modules/DBIish in RakuDist web UI

rbt commented 4 years ago

@melezhik Thanks.

I added a piece to get english error messages for connection failures, but it just said "unable to open database file".

So I've tried adding the API for extended status codes. See if that gives back more information.

rbt commented 4 years ago

The extended status codes api (if I called it correctly) didn't provide more information, still spits out code 14.

A bit of debug stuff in the header of the file confirmed the working directory is what was expected, a file could be created in the working directory, and tmpdir is /tmp. I assume /tmp is world writable.

my $cwd = $*CWD;
say "CWD: $cwd";
say "IS DIR: ", ($cwd.IO ~~ :d);
$cwd.IO.add('somefile').spurt('FILES ARE WRITABLE');
say "TMPDIR: %s".sprintf($*TMPDIR);

Results are midway down https://rakudist.raku.org/sparky/report/alpine/287 .

melezhik commented 4 years ago

I assume /tmp is world writable.

Let me check it

melezhik commented 4 years ago

looks like that:

$ docker exec -i -t alpine-rakudist /bin/bash
bash-5.0# su - vgpweasmnktyozcxuqhf
d286df8c7970:~$ touch /tmp/foo2
d286df8c7970:~$ mkdir /tmp/ff/

but for sure I'd test it within test suite itself ...

rbt commented 4 years ago

@melezhik Do they work if you run the tests individually by hand?

Does that first test (43-sqlite-threads.t) work if you run it multiple times?

melezhik commented 4 years ago

@rbt here is what I get when run tests manually. multiple runs of 43-sqlite-threads.t works fine

d286df8c7970:/data/test/vgpweasmnktyozcxuqhf$ perl6 -I lib/ t/43-sqlite-threads.t
1..2
    ok 1 - Inserting thread 0 completed
    ok 2 - Inserting thread 1 completed
    ok 3 - Inserting thread 2 completed
    ok 4 - Inserting thread 3 completed
    ok 5 - Inserting thread 4 completed
    ok 6 - Correct number of rows were inserted
    1..6
ok 1 - Statements across threads on one connection
    ok 1 - Inserting thread 0 completed
    ok 2 - Inserting thread 1 completed
    ok 3 - Inserting thread 2 completed
    ok 4 - Inserting thread 3 completed
    ok 5 - Inserting thread 4 completed
    ok 6 - Correct number of rows were inserted
    1..6
ok 2 - Multiple connections, one per thread
d286df8c7970:/data/test/vgpweasmnktyozcxuqhf$ perl6 -I lib/ t/43-sqlite-threads.t
1..2
    ok 1 - Inserting thread 0 completed
    ok 2 - Inserting thread 1 completed
    ok 3 - Inserting thread 2 completed
    ok 4 - Inserting thread 3 completed
    ok 5 - Inserting thread 4 completed
    ok 6 - Correct number of rows were inserted
    1..6
ok 1 - Statements across threads on one connection
    ok 1 - Inserting thread 0 completed
    ok 2 - Inserting thread 1 completed
    ok 3 - Inserting thread 2 completed
    ok 4 - Inserting thread 3 completed
    ok 5 - Inserting thread 4 completed
    ok 6 - Correct number of rows were inserted
    1..6
ok 2 - Multiple connections, one per thread
d286df8c7970:/data/test/vgpweasmnktyozcxuqhf$ perl6 -I lib/ t/48-sqlite-errors.t
1..17
ok 1 - At start DBIish.err is 0
    1..2
    ok 1 - code dies
    ok 2 - right exception type (X::AdHoc)
ok 2 - Need database
CWD: /data/test/vgpweasmnktyozcxuqhf
IS DIR: True
TMPDIR: /tmp
    1..2
    ok 1 - code dies
    ok 2 - right exception type (X::DBDish::ConnectionFailed)
ok 3 - Die on invalid database
ok 4 - Error registered
DBDish::SQLite: Can't connect: unable to open database file (14)
  in method connect at /data/test/vgpweasmnktyozcxuqhf/lib/DBDish/SQLite.pm6 (DBDish::SQLite) line 43
  in method connect at /data/test/vgpweasmnktyozcxuqhf/lib/DBIish.pm6 (DBIish) line 52
  in block <unit> at t/48-sqlite-errors.t line 30

# You planned 17 tests, but ran 4
melezhik commented 4 years ago

Well, looks like I've found what causes the bug, though I don't userstand why:

Here is the patch that makes t/48-sqlite-errors.t passes:

my $dbfile = 'exec-error';
$dbfile.IO.e;

So one has to add just a single line that does all the treack - $dbfile.IO.e and the test will pass. (Or proably any other IO method, does not really matter, but I am not sure).

Another weirdness the test still fails if we refer the same way to any other IO object after we've refered to a database file, so for example this sequence of lines will make the test fail ( with the same SQLITE_CANTOPEN error ):

my $dbfile = 'exec-error';
$dbfile.IO.e;
"any-file".IO.e;

And even $*CWD.IO.e will cause the test failure:

my $dbfile = 'exec-error';
$dbfile.IO.e;
$*CWD.IO.e;

I wonder if it's an Alpine/Docker/SQLite bug or Rakudo or DBIIsh binding or all together. But that's really funny!

melezhik commented 4 years ago

Here is successful build for my patch for alpine

http://rakudist.raku.org/sparky/report/alpine/297

rbt commented 4 years ago

@melezhik I've tried a couple different Alpine images locally. Still can't reproduce it here.

What version of docker are you using? I have Docker version 19.03.8, build afacb8b7f0.

melezhik commented 4 years ago

This is an official alpine image

$ docker images
REPOSITORY                  TAG      IMAGE ID       CREATED        SIZE
localhost/alpine-rakudist   latest   2eefe624417e   2 weeks ago    266 MB
localhost/ubuntu-rakudist   latest   db2e908ef01e   2 weeks ago    2.22 GB
localhost/debian-rakudist   latest   25b4fc242478   2 weeks ago    1.95 GB
docker.io/library/debian    latest   5971ee6076a0   3 weeks ago    119 MB
docker.io/library/ubuntu    latest   1d622ef86b13   6 weeks ago    76.3 MB
docker.io/library/alpine    latest   f70734b6a266   6 weeks ago    5.88 MB
docker.io/library/centos    latest   470671670cac   4 months ago   245 MB

$ docker history f70734b6a266
ID             CREATED       CREATED BY                                      SIZE      COMMENT
f70734b6a266   6 weeks ago   /bin/sh -c #(nop) CMD ["/bin/sh"]               0B
melezhik commented 4 years ago

just in case I may drop current container, pull the latest alpine image and rerun test from the new image

melezhik commented 4 years ago

RakuDist uses podman:

podman version 1.6.2

$ dpkg --list|grep podman
ii  podman                                 1.6.2-1~ubuntu18.04~ppa1                        amd64        Manage pods, containers and container images.
melezhik commented 4 years ago

well, rerunning test on fresh docker container did not help, the same error http://rakudist.raku.org/sparky/report/alpine/303

The only deviation from vanilla alpine docker image I could think is:

https://github.com/melezhik/sparrow-plugins/blob/master/alpine-glibc-install/task.bash

I have to install glibc that way because it required to install whatevrable rakudo distributions.

You could see distinct log on the test log:

18:50:58 06/11/2020 [install glibc] stderr:  https://github.com/sgerrand/alpine-pkg-glibc/releases/download/2.31-r0/glibc-2.31-r0.apk
18:51:17 06/11/2020 [install glibc] stderr: ++ apk add --allow-untrusted glibc-2.31-r0.apk
18:51:17 06/11/2020 [install glibc] (1/1) Installing glibc (2.31-r0)
18:51:17 06/11/2020 [install glibc] OK: 229 MiB in 58 packages

Not sure how this could impact on sqlite/rakudo behavior if any

melezhik commented 4 years ago

after all I am not even sure if it worth to go forward, maybe it's just a strange mix of podman/alpine/sqlite behavior , and won't impact most of the users ...

melezhik commented 4 years ago

which rakudo version do you use btw? could you compare with the one used in the test. RakuDist uses whateverable rakudo binaries ...

On Thu, Jun 11, 2020, 1:25 PM rbt notifications@github.com wrote:

@melezhik https://github.com/melezhik I've tried a couple different Alpine images locally. Still can't reproduce it here.

What version of docker are you using? I have Docker version 19.03.8, build afacb8b7f0.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/raku-community-modules/DBIish/issues/190#issuecomment-642854773, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAHRHSPUWEST3Z4OPW7J53DRWEOTHANCNFSM4N2QN67A .

melezhik commented 4 years ago

I run the same test on docker on another server

Docker version 19.03.5, build 633a0ea

And it fails with the same error

rbt commented 4 years ago

@melezhik 2020.02.01 but I used the build interface earlier and the issue occurs on all Rakudo releases from 2019.11 up on that specific system.

melezhik commented 4 years ago

so you confirming the issue on your side?

rbt commented 4 years ago

No. I've not been able to reproduce it anywhere but the rakudist.raku.org interface.

melezhik commented 4 years ago

ok, let's put it on hold, I could spin up an alpine docker manually and repeate all the steps manualy so that avoid any rakudist related issues, and let's see what it gives.

it's nothing but install rakudo (I could use system package), sqlite and run zef install, should be simple.

melezhik commented 4 years ago

So I reproduced the issue on alpine docker, but I am afraid this is somehow relates to system dependencies (glibc?) or maybe how whateverable distro ties to them. Notice that I have to install glibc whihc is required by whateveravble distro of Rakudo, and that glibc installation is not natively supported by alpine, so I have to use workaround for that:

docker pull alpine
docker run -d -t --rm --name alpine-rakudist alpine
docker exec -i -t alpine-rakudist /bin/sh

# install system dependencies
apk add bash git wget tar zstd
apk add make gcc libc-dev tar which musl-dev

# install glibc 
wget --quiet --no-clobber --no-check-certificate https://github.com/sgerrand/alpine-pkg-glibc/releases/download/2.31-r0/glibc-2.31-r0.apk
apk add --allow-untrusted glibc-2.31-r0.apk

# install whateverable
bash
mkdir -p /data/whateverable
cd /data/whateverable/
wget -q https://whateverable.6lang.org/002acb1be2ba2a47ef8a48c30c340d43df91abed
zstd -dqc -- 002acb1be2ba2a47ef8a48c30c340d43df91abed | tar -x --absolute-names

# create test user
adduser -s /bin/bash test
su - test

# install zef
git clone https://github.com/ugexe/zef.git
cd zef

# export env vars
export LD_LIBRARY_PATH=/lib:/usr/lib/
export PATH=~/.perl6/bin:/tmp/whateverable/rakudo-moar/002acb1be2ba2a47ef8a48c30c340d43df91abed/bin/:$PATH

# install zef

perl6 -I . bin/zef install . --/test

# install DBIish
zef --debug install DBIish # it will produces the mentioned error, see full log in gist

See all details here - https://gist.github.com/melezhik/522ead6ebaa87bb1ab429377aaaec003

melezhik commented 4 years ago

So, my primary suspicious is a glibc, as it is the only system package that is not installed by using default Apline repositories.

cc @AlexDaniel

JJ commented 4 years ago

You can use musl-dev to install rakudo distributions, no need to glibc at all. Bash is not needed either.

El jue., 11 jun. 2020 a las 21:04, Alexey Melezhik (< notifications@github.com>) escribió:

well, rerunning test on fresh docker container did not help, the same error http://rakudist.raku.org/sparky/report/alpine/303

The only deviation from vanilla alpine docker image I could think is:

https://github.com/melezhik/sparrow-plugins/blob/master/alpine-glibc-install/task.bash

I have to install glibc that way because it required to install whatevrable rakudo distributions.

You could see distinct log on the test log:

18:50:58 06/11/2020 [install glibc] stderr: https://github.com/sgerrand/alpine-pkg-glibc/releases/download/2.31-r0/glibc-2.31-r0.apk 18:51:17 06/11/2020 [install glibc] stderr: ++ apk add --allow-untrusted glibc-2.31-r0.apk 18:51:17 06/11/2020 [install glibc] (1/1) Installing glibc (2.31-r0) 18:51:17 06/11/2020 [install glibc] OK: 229 MiB in 58 packages

Not sure how this could impact on sqlite/rakudo behavior if any

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/raku-community-modules/DBIish/issues/190#issuecomment-642872972, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAAD5H2SRJSKACVAK45GILRWETCLANCNFSM4N2QN67A .

-- JJ

melezhik commented 4 years ago

hi @JJ .

You're right at that point that musl is an official replacement for glibc in Alpine. This, for example explains the fact that the musl could be installed throughout an official alpine repositories, while glibc isn't.

However the way whateveable rakudo works it requires glibc in place and this probably screws things up on alpina as you could see I am using "unofficial" method of installation for glibc on Alpine ( and there is no official method to do it ).

But I am not positive here and we probably need to discuss all this in another issue in proper whateverable github repo, and this is why I cc @AlexDaniel here.

Yes bash is not required , I use it for simplicity when represent the test case just because I prefer to work in Bash session.

JJ commented 4 years ago

Why do you need it? What's the reverse dependency?

melezhik commented 4 years ago

The way I figure it out it that:

$  /tmp/whateverable/rakudo-moar/002acb1be2ba2a47ef8a48c30c340d43df91abed/bin/raku -e 1
-bash: /tmp/whateverable/rakudo-moar/002acb1be2ba2a47ef8a48c30c340d43df91abed/bin/raku: No such file or directory

$ apk add --allow-untrusted glibc-2.31-r0.apk
fetch http://dl-cdn.alpinelinux.org/alpine/v3.12/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.12/community/x86_64/APKINDEX.tar.gz
(1/1) Installing glibc (2.31-r0)
OK: 171 MiB in 47 packages

$  /tmp/whateverable/rakudo-moar/002acb1be2ba2a47ef8a48c30c340d43df91abed/bin/raku -e "say 111"
111

I belive @AlexDaniel as a maintainer of whateverable rakudo binaries would give a better explanation.

PS I tried to ldd on whateverable rakudo binary but it does not help much

JJ commented 4 years ago

The problem is that binaries in Alpine work in a slightly different way, because they don't use glibc. So you need to generate binaries for Alpine, or use another distro like Debian. Ubuntu might not work either, because every distro is locked to a glibc version... Been there, been burned, done that. Anyway, what I'd try to do is to create a specific Whateverable package... Using whatever distribution works with it.

melezhik commented 4 years ago

Well, RakuDist does not use distro/os specific packages for Rakudo , nor it generates any of them. Instead of that it just pulls whatever comes from whateverable site, see the whole process here:

https://github.com/melezhik/sparrow6-rakudo-install/blob/master/lib/Sparrow6/Rakudo/Install.pm6

On Fri, Jun 12, 2020, 10:14 AM Juan Julián Merelo Guervós < notifications@github.com> wrote:

The problem is that binaries in Alpine work in a slightly different way, because they don't use glibc. So you need to generate binaries for Alpine, or use another distro like Debian. Ubuntu might not work either, because every distro is locked to a glibc version... Been there, been burned, done that. Anyway, what I'd try to do is to create a specific Whateverable package... Using whatever distribution works with it.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/raku-community-modules/DBIish/issues/190#issuecomment-643327865, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAHRHSN4KU7C4U5RQCOXESLRWJA55ANCNFSM4N2QN67A .

JJ commented 4 years ago

Then I would definitely change the base distro to Debian. There's a slim version that's also lightweight; not as much as Alpine, I'm afraid, but that's the distro these applications have been created with and for, so you'll not run into any issues.

JJ commented 4 years ago

Alternatively, you could use the extraofficially official raku containers that I maintain here. There's one for every release, and they have already been tested. Including DBIish...

melezhik commented 4 years ago

And that is the case right now ☺️. You could see it when launch tests through RakuDist UI, Debian is selected by default. Like i said RakuDist supports 4 distros right now - Debian, Ubuntu, Centos and Alpine.

On my experience I only have such a weird bug with Alpine. Others OS are just fine.

On Fri, Jun 12, 2020, 10:23 AM Juan Julián Merelo Guervós < notifications@github.com> wrote:

Then I would definitely change the base distro to Debian. There's a slim version that's also lightweight; not as much as Alpine, I'm afraid, but that's the distro these applications have been created with and for, so you'll not run into any issues.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/raku-community-modules/DBIish/issues/190#issuecomment-643332681, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAHRHSIVOR2AWJCINY6AZT3RWJB7NANCNFSM4N2QN67A .

melezhik commented 4 years ago

Well it's not an issue with docker container itself but with whateverable Rakudo for Alpine.

Please also pay attention that just having one stable version of Rakudo does not work for RakuDist, because idea of the RakuDist is people want to test modules against different versions and probably commits of Rakudo and this is why whateverable is used as a method to install Rakudo versions.

On Fri, Jun 12, 2020, 10:26 AM Juan Julián Merelo Guervós < notifications@github.com> wrote:

Alternatively, you could use the extraofficially official raku containers that I maintain here https://hub.docker.com/r/jjmerelo/alpine-raku. There's one for every release, and they have already been tested. Including DBIish...

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/raku-community-modules/DBIish/issues/190#issuecomment-643334052, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAHRHSO4EWRTXSV6YPEDHZDRWJCJTANCNFSM4N2QN67A .