Prebuilt binary of version 5.0.4 still does not work on CentOS 7

hideishi-m commented 2 years ago

Hello, unfortunately another issue with prebuilt binary on CentOS 7. Additionally, older libstdc++ compatiblity is required.

$ npm i sqlite3

added 104 packages, and audited 105 packages in 7s

5 packages are looking for funding
  run `npm fund` for details

found 0 vulnerabilities
$ npm ls
sql@ /home/kusanagi/work/sql
└── sqlite3@5.0.4

$ node
Welcome to Node.js v16.14.2.
Type ".help" for more information.
> require('sqlite3')
Uncaught:
Error: /lib64/libstdc++.so.6: version `CXXABI_1.3.8' not found (required by /home/kusanagi/work/sql/node_modules/sqlite3/lib/binding/napi-v6-linux-glibc-x64/node_sqlite3.node)
    at Object.Module._extensions..node (node:internal/modules/cjs/loader:1187:18)
    at Module.load (node:internal/modules/cjs/loader:981:32)
    at Function.Module._load (node:internal/modules/cjs/loader:822:12)
    at Module.require (node:internal/modules/cjs/loader:1005:19)
    at require (node:internal/modules/cjs/helpers:102:18) {
  code: 'ERR_DLOPEN_FAILED'
}
>

$ rpm -qa | grep libstd
libstdc++-4.8.5-44.el7.x86_64
libstdc++-devel-4.8.5-44.el7.x86_64

$ env LANG=C objdump -p /lib64/libstdc++.so.6

/lib64/libstdc++.so.6:     file format elf64-x86-64

Program Header:
    LOAD off    0x0000000000000000 vaddr 0x0000000000000000 paddr 0x0000000000000000 align 2**21
         filesz 0x00000000000e8deb memsz 0x00000000000e8deb flags r-x
    LOAD off    0x00000000000e9d10 vaddr 0x00000000002e9d10 paddr 0x00000000002e9d10 align 2**21
         filesz 0x0000000000008aa8 memsz 0x000000000001d710 flags rw-
 DYNAMIC off    0x00000000000f04d8 vaddr 0x00000000002f04d8 paddr 0x00000000002f04d8 align 2**3
         filesz 0x0000000000000220 memsz 0x0000000000000220 flags rw-
    NOTE off    0x0000000000000200 vaddr 0x0000000000000200 paddr 0x0000000000000200 align 2**2
         filesz 0x0000000000000024 memsz 0x0000000000000024 flags r--
     TLS off    0x00000000000e9d10 vaddr 0x00000000002e9d10 paddr 0x00000000002e9d10 align 2**3
         filesz 0x0000000000000000 memsz 0x0000000000000020 flags r--
EH_FRAME off    0x00000000000c8b94 vaddr 0x00000000000c8b94 paddr 0x00000000000c8b94 align 2**2
         filesz 0x0000000000004a94 memsz 0x0000000000004a94 flags r--
   STACK off    0x0000000000000000 vaddr 0x0000000000000000 paddr 0x0000000000000000 align 2**4
         filesz 0x0000000000000000 memsz 0x0000000000000000 flags rw-
   RELRO off    0x00000000000e9d10 vaddr 0x00000000002e9d10 paddr 0x00000000002e9d10 align 2**0
         filesz 0x00000000000072f0 memsz 0x00000000000072f0 flags r--

Dynamic Section:
  NEEDED               libm.so.6
  NEEDED               libc.so.6
  NEEDED               ld-linux-x86-64.so.2
  NEEDED               libgcc_s.so.1
  SONAME               libstdc++.so.6
  INIT                 0x0000000000058b78
  FINI                 0x00000000000c259c
  INIT_ARRAY           0x00000000002e9d10
  INIT_ARRAYSZ         0x0000000000000040
  FINI_ARRAY           0x00000000002e9d50
  FINI_ARRAYSZ         0x0000000000000008
  GNU_HASH             0x0000000000000228
  STRTAB               0x000000000001c380
  SYMTAB               0x0000000000005bf8
  STRSZ                0x0000000000027411
  SYMENT               0x0000000000000018
  PLTGOT               0x00000000002f1000
  PLTRELSZ             0x0000000000003dc8
  PLTREL               0x0000000000000007
  JMPREL               0x0000000000054db0
  RELA                 0x0000000000045a80
  RELASZ               0x000000000000f330
  RELAENT              0x0000000000000018
  VERDEF               0x0000000000045588
  VERDEFNUM            0x000000000000001e
  VERNEED              0x00000000000459a0
  VERNEEDNUM           0x0000000000000004
  VERSYM               0x0000000000043792
  RELACOUNT            0x00000000000001dd

Version definitions:
1 0x01 0x025f4d66 libstdc++.so.6
2 0x00 0x08922974 GLIBCXX_3.4
3 0x00 0x02297f81 GLIBCXX_3.4.1
        GLIBCXX_3.4
4 0x00 0x02297f82 GLIBCXX_3.4.2
        GLIBCXX_3.4.1
5 0x00 0x02297f83 GLIBCXX_3.4.3
        GLIBCXX_3.4.2
6 0x00 0x02297f84 GLIBCXX_3.4.4
        GLIBCXX_3.4.3
7 0x00 0x02297f85 GLIBCXX_3.4.5
        GLIBCXX_3.4.4
8 0x00 0x02297f86 GLIBCXX_3.4.6
        GLIBCXX_3.4.5
9 0x00 0x02297f87 GLIBCXX_3.4.7
        GLIBCXX_3.4.6
10 0x00 0x02297f88 GLIBCXX_3.4.8
        GLIBCXX_3.4.7
11 0x00 0x02297f89 GLIBCXX_3.4.9
        GLIBCXX_3.4.8
12 0x00 0x0297f860 GLIBCXX_3.4.10
        GLIBCXX_3.4.9
13 0x00 0x0297f861 GLIBCXX_3.4.11
        GLIBCXX_3.4.10
14 0x00 0x0297f862 GLIBCXX_3.4.12
        GLIBCXX_3.4.11
15 0x00 0x0297f863 GLIBCXX_3.4.13
        GLIBCXX_3.4.12
16 0x00 0x0297f864 GLIBCXX_3.4.14
        GLIBCXX_3.4.13
17 0x00 0x0297f865 GLIBCXX_3.4.15
        GLIBCXX_3.4.14
18 0x00 0x0297f866 GLIBCXX_3.4.16
        GLIBCXX_3.4.15
19 0x00 0x0297f867 GLIBCXX_3.4.17
        GLIBCXX_3.4.16
20 0x00 0x0297f868 GLIBCXX_3.4.18
        GLIBCXX_3.4.17
21 0x00 0x0297f869 GLIBCXX_3.4.19
        GLIBCXX_3.4.18
22 0x00 0x056bafd3 CXXABI_1.3
23 0x00 0x0bafd171 CXXABI_1.3.1
        CXXABI_1.3
24 0x00 0x0bafd172 CXXABI_1.3.2
        CXXABI_1.3.1
25 0x00 0x0bafd173 CXXABI_1.3.3
        CXXABI_1.3.2
26 0x00 0x0bafd174 CXXABI_1.3.4
        CXXABI_1.3.3
27 0x00 0x0bafd175 CXXABI_1.3.5
        CXXABI_1.3.4
28 0x00 0x0bafd176 CXXABI_1.3.6
        CXXABI_1.3.5
29 0x00 0x0bafd177 CXXABI_1.3.7
        CXXABI_1.3.6
30 0x00 0x06bc6f71 CXXABI_TM_1

Version References:
  required from ld-linux-x86-64.so.2:
    0x0d696913 0x00 37 GLIBC_2.3
  required from libm.so.6:
    0x09691a75 0x00 34 GLIBC_2.2.5
  required from libgcc_s.so.1:
    0x09276060 0x00 38 GCC_4.2.0
    0x0b792653 0x00 36 GCC_3.3
    0x0b792650 0x00 33 GCC_3.0
  required from libc.so.6:
    0x06969194 0x00 40 GLIBC_2.14
    0x0d696914 0x00 39 GLIBC_2.4
    0x0d696913 0x00 35 GLIBC_2.3
    0x09691972 0x00 32 GLIBC_2.3.2
    0x09691a75 0x00 31 GLIBC_2.2.5

daniellockyer commented 2 years ago

Hey @hideishi-m, that's unfortunate 😕 It look like CentOS 7 ships with libstdc++ 6.0.19, which I believe comes from GCC 4.8.x and came out around 2014/2015.

I've been unable to find a way to compile the binaries with compatibility for this, so I'm stuck on how we could fix this. If you know of a way, I'd be glad to hear 🙂

In the mean time, you'll have to compile node-sqlite3 from source, or update the libraries your machine uses

hideishi-m commented 2 years ago

Hello @daniellockyer. I did a quick research and I think I found a way. Instead of using ubuntu-latest runner for CI, could you try using ubuntu-18.04 runner? It seems ubuntu-18.04 runner is shipped with libstdc++-6.0 which is compatible with CentOS 7's libstdc++ 6.0.19.

https://github.com/actions/virtual-environments/blob/main/images/linux/Ubuntu1804-Readme.md#installed-apt-packages

daniellockyer commented 2 years ago

Hey @hideishi-m 🙂 I can't seem to get that to work. My CI run here was done on ubuntu-18.04 and the napi-v6-linux-glibc-x64 binary inside the artifact zip still shows the same error on CentOS 7 😕

hideishi-m commented 2 years ago

Hello @daniellockyer, I see ... in that case I have also no idea how you can fix this. At least the existing prebuild binaries work on CentOS Stream 8 and AlmaLinux8. For CentOS 7, I will continue building my own binary via node-pre-gyp.

By the way, I also had to do some tweaks to build my own binary that node-pre-gyp sets -std=gnu++14 where as old gcc tool chain on CentOS 7 doesn't support this. I had to replace with -std=gnu++11 instead.

daniellockyer commented 2 years ago

I'm about to release 5.0.5 but it won't contain the fix for this 😕 I'll see if I can take another look soon 🙂

louislam commented 2 years ago

I've been unable to find a way to compile the binaries with compatibility for this, so I'm stuck on how we could fix this. If you know of a way, I'd be glad to hear 🙂

For your reference, as previously node-sqlite3 (<= 5.0.2) did not provide arm built, users were hard to install Uptime Kuma, so I forked this project to support it.

In order to cover more systems, in my fork, I built arm64 and amd64 directly on CentOS 7 docker image. https://github.com/louislam/node-sqlite3/blob/master/build.dockerfile

Although it was built on CentOS 7, the binary file is still working on the latest Debian (Bullseye).

I recently merged node-sqlite3 upstream source (5.0.3) with my build script, it is still working on CentOS 7.

BobbyWibowo commented 2 years ago

Hi there, back then sharp also had this issue with their pre-built binaries I believe their solution was to use CentOS 7 docker container within the Ubuntu runner, https://github.com/lovell/sharp/issues/2119#issuecomment-597301435 ^ However, the associated commit in that comment was back when they were still using Travis

A few months after that, they migrated to GitHub Actions (https://github.com/lovell/sharp/commit/e59e14688707215233bfb11e46cb9b58b2705d04), and they still stick with the same method Their latest CI actions config lives here https://github.com/lovell/sharp/blob/main/.github/workflows/ci.yml, which hopefully can be of reference

cjbland commented 2 years ago

I'm having a similar issue trying to build sqlite3 from source on a CentOS 7 using node v16. The specific error I'm getting is:

yarn global add sqlite3@5.0.8 ... omitted output ... g++: error: unrecognized command line option '-std=gnu++14'

Running the same command using node v14 build successfully. I inspected the generated files and it is using `-std=gnu++1y' instead. Does anyone have any idea why v14 works but v16 does not?

:edit:

Naturally 5 minutes after I posted this I found an answer. Node v16.2.0 changed the gnu++1y flag to gnu++14 because gnu++1y was deprecated. I ended up editing node-v16/include/node/common.gypi and changed the references. Everything built successfully after that. I'm not sure if this is the proper way to handle this, but it works for me for now.

csuich2 commented 2 years ago

Any thoughts on this? I'm having some issues where I need to bump from 5.0.2 to support ARM but doing so breaks CentOS 7.

davidrapin commented 2 years ago

I found this thread that seems to address a similar issue. This is my understanding so far:

node-sqlite3 requires c++14 features
centos7/rhel7 ship with gcc 4.2 by default
gcc 4.2 does not support c++14
to get c++14, you need to install gcc 10

Then, when running your Node.js app, you need to set LD_LIBRARY_PATH=$PATH::/YourPathTo/gcc-10.2.0/lib64/. For example, LD_LIBRARY_PATH=$PATH::/YourPathTo/gcc-10.2.0/lib64/ node my-script.js

I have not tested this workaround yet, since I don't have a centos7/rhel7 instance at hand. If anyone can try, I would be interested in the result.

heikkipora commented 2 years ago

I can confirm that the suggestion above by @davidrapin works nicely (CentOS 7.9.2009, node-sqlite3 v5.0.10)

installed gcc10-libstdc++ (available at least in Ghettoforge: https://mirror.ghettoforge.org/distributions/gf/el/7/gf/x86_64/)
set LD_LIBRARY_PATH=$PATH:/opt/gcc-10.2.1/usr/lib64 for my node app

Thanks for the tip!

hanseartic commented 2 years ago

I'm having a similar issue trying to build sqlite3 from source on a CentOS 7 using node v16. The specific error I'm getting is:

yarn global add sqlite3@5.0.8 ... omitted output ... g++: error: unrecognized command line option '-std=gnu++14'

Running the same command using node v14 build successfully. I inspected the generated files and it is using `-std=gnu++1y' instead. Does anyone have any idea why v14 works but v16 does not?

:edit:

Naturally 5 minutes after I posted this I found an answer. Node v16.2.0 changed the gnu++1y flag to gnu++14 because gnu++1y was deprecated. I ended up editing node-v16/include/node/common.gypi and changed the references. Everything built successfully after that. I'm not sure if this is the proper way to handle this, but it works for me for now.

@cjbland, it's possible to build from source in node v16.16.0 on centOS 7 without changing any references:

$ node --version
v16.16.0
$ node
Welcome to Node.js v16.16.0.
Type ".help" for more information.
> require("sqlite3")
Uncaught:
Error: /lib64/libstdc++.so.6: version `CXXABI_1.3.8' not found (required by /home/***/myproject/node_modules/sqlite3/lib/binding/napi-v6-linux-glibc-x64/node_sqlite3.node)
    at Object.Module._extensions..node (node:internal/modules/cjs/loader:1189:18)
    at Module.load (node:internal/modules/cjs/loader:981:32)
    at Function.Module._load (node:internal/modules/cjs/loader:822:12)
    at Module.require (node:internal/modules/cjs/loader:1005:19)
    at require (node:internal/modules/cjs/helpers:102:18) {
  code: 'ERR_DLOPEN_FAILED'
}
> .exit
$ rm -rf node-modules
$ npm i --build-from-source

added 107 packages, and audited 108 packages in 2m

5 packages are looking for funding
  run `npm fund` for details

found 0 vulnerabilities
$ npm ls sqlite3
myproject@0.0.1 /home/***/myproject
└── sqlite3@5.0.11

$ node
Welcome to Node.js v16.16.0.
Type ".help" for more information.
> require("sqlite3")
{
  Database: [Function: Database],
  Statement: [Function: Statement],
  Backup: [Function: Backup],
  OPEN_READONLY: 1,
  OPEN_READWRITE: 2,
  OPEN_CREATE: 4,
  OPEN_FULLMUTEX: 65536,
  OPEN_URI: 64,
  OPEN_SHAREDCACHE: 131072,
  OPEN_PRIVATECACHE: 262144,
  VERSION: '3.39.2',
  SOURCE_ID: '2022-07-21 15:24:47 698edb77537b67c41adc68f9b892db56bcf9a55e00371a61420f3ddd668e6603',
  VERSION_NUMBER: 3039002,
  OK: 0,
  ERROR: 1,
  INTERNAL: 2,
  PERM: 3,
  ABORT: 4,
  BUSY: 5,
  LOCKED: 6,
  NOMEM: 7,
  READONLY: 8,
  INTERRUPT: 9,
  IOERR: 10,
  CORRUPT: 11,
  NOTFOUND: 12,
  FULL: 13,
  CANTOPEN: 14,
  PROTOCOL: 15,
  EMPTY: 16,
  SCHEMA: 17,
  TOOBIG: 18,
  CONSTRAINT: 19,
  MISMATCH: 20,
  MISUSE: 21,
  NOLFS: 22,
  AUTH: 23,
  FORMAT: 24,
  RANGE: 25,
  NOTADB: 26,
  cached: { Database: [Function: Database], objects: {} },
  verbose: [Function (anonymous)]
}
.exit
$

PJButterfield commented 2 years ago

Just wanted to add thanks to @davidrapin and @heikkipora this was the best solution I could find for now.

putzwasser commented 1 year ago

In case anyone wants to try ghost on uberspace with sqlite, you'd do this:

cd GHOSTDIR
mkdir lib
cd lib
wget https://mirror.ghettoforge.org/distributions/gf/el/7/gf/x86_64/gcc10-libstdc++-10.2.1-7.gf.el7.x86_64.rpm
rpm2cpio gcc10-libstdc++-10.2.1-7.gf.el7.x86_64.rpm | cpio -idmv
mv ./opt/gcc-10.2.1/usr/lib64/* ./
rm -r ./opt/
LD_LIBRARY_PATH=$PATH:/path/to/your/GHOSTDIR/lib
# start ghost

This fetches the mentioned version of libstdc++ and unpacks it locally.

TryGhost / node-sqlite3

Prebuilt binary of version 5.0.4 still does not work on CentOS 7 #1582