search

xerial / sqlite-jdbc

SQLite JDBC Driver
Apache License 2.0
2.86k stars 619 forks source link

native lib doesn't work on `armv7l` #765

Closed gotson closed 1 year ago

gotson commented 2 years ago

@mastervol reported this in https://github.com/gotson/komga/issues/945

Failed to load native library:sqlite-3.39.2.0-4862e885-20d4-47f7-a1f8-ed7c54fd2855-libsqlitejdbc.so. osinfo: Linux/armv7
java.lang.UnsatisfiedLinkError: /tmp/sqlite-3.39.2.0-4862e885-20d4-47f7-a1f8-ed7c54fd2855-libsqlitejdbc.so: /tmp/sqlite-3.39.2.0-4862e885-20d4-47f7-a1f8-ed7c54fd2855-libsqlitejdbc.so: failed to map segment from shared object

The device is a Synology DS216 Play.

uname -m
armv7l

cat /proc/cpuinfo

processor       : 0
model name      : ARMv7 Processor rev 0 (v7l)
BogoMIPS        : 2996.63
Features        : swp half thumb fastmult vfp edsp neon vfpv3 tls
CPU implementer : 0x41
CPU architecture: 7
CPU variant     : 0x3
CPU part        : 0xc09
CPU revision    : 0

processor       : 1
model name      : ARMv7 Processor rev 0 (v7l)
BogoMIPS        : 2989.26
Features        : swp half thumb fastmult vfp edsp neon vfpv3 tls
CPU implementer : 0x41
CPU architecture: 7
CPU variant     : 0x3
CPU part        : 0xc09
CPU revision    : 0

Hardware        : STi SoC with Flattened Device Tree
Revision        : 0000
Serial          : 0000000000000000
gotson commented 2 years ago

@mastervol would you be able to try the same instructions (adjusting the classpath to use the correct filename) but with this jar instead ?

mastervol commented 2 years ago

Hm. Unfortunately another error.

root@stfuplay:/volume1/@appstore/komga/test# javac Sample.java
root@stfuplay:/volume1/@appstore/komga/test# java -classpath ".:sqlite-jdbc-3.39.2.1-SNAPSHOT.jar" Sample
Failed to load native library:sqlite-3.39.2.1-d8e0a70b-a93c-4cf3-a172-37a66fa2c994-libsqlitejdbc.so. osinfo: Linux/armv7
java.lang.UnsatisfiedLinkError: /tmp/sqlite-3.39.2.1-d8e0a70b-a93c-4cf3-a172-37a66fa2c994-libsqlitejdbc.so: /tmp/sqlite-3.39.2.1-d8e0a70b-a93c-4cf3-a172-37a66fa2c994-libsqlitejdbc.so: internal error
Error opening connection
root@stfuplay:/volume1/@appstore/komga/test#
gotson commented 2 years ago

I have been trying to find online some information about that processor, or about armv7l in general, but i can't find good information on how to cross-compile for that particular platform.

gotson commented 2 years ago

I'm wondering if the problem could be something else than compatibility of the binary.

Could you try to use a different directory for the native library, for example:

java -Djava.io.tmpdir="/volume1/@appstore/komga/test" -classpath ".:sqlite-jdbc-3.39.2.1-SNAPSHOT.jar" Sample
mastervol commented 2 years ago

I tried to use a different directory, still no luck. I can check if we can get support from the syno community, let me check the java projects there. 

root@stfuplay:/tmp/test# java -Djava.io.tmpdir="/tmp/test" -classpath ".:sqli
te-jdbc-3.39.2.1-SNAPSHOT.jar" Sample
Failed to load native library:sqlite-3.39.2.1-274cbefb-bc7d-4a49-bc4a-40ce36d
2d1d9-libsqlitejdbc.so. osinfo: Linux/armv7
java.lang.UnsatisfiedLinkError: /tmp/test/sqlite-3.39.2.1-274cbefb-bc7d-4a49-
bc4a-40ce36d2d1d9-libsqlitejdbc.so: /tmp/test/sqlite-3.39.2.1-274cbefb-bc7d-4
a49-bc4a-40ce36d2d1d9-libsqlitejdbc.so: internal error
Error opening connection
root@stfuplay:/tmp/test#
gotson commented 2 years ago

I tried to use a different directory, still no luck. I can check if we can get support from the syno community, let me check the java projects there.

root@stfuplay:/tmp/test# java -Djava.io.tmpdir="/tmp/test" -classpath ".:sqli
te-jdbc-3.39.2.1-SNAPSHOT.jar" Sample
Failed to load native library:sqlite-3.39.2.1-274cbefb-bc7d-4a49-bc4a-40ce36d
2d1d9-libsqlitejdbc.so. osinfo: Linux/armv7
java.lang.UnsatisfiedLinkError: /tmp/test/sqlite-3.39.2.1-274cbefb-bc7d-4a49-
bc4a-40ce36d2d1d9-libsqlitejdbc.so: /tmp/test/sqlite-3.39.2.1-274cbefb-bc7d-4
a49-bc4a-40ce36d2d1d9-libsqlitejdbc.so: internal error
Error opening connection
root@stfuplay:/tmp/test#

Can you try something other than /tmp? From what I could gather online it could have insufficient permissions hence my suggestions to use another dir that is not tmp

mastervol commented 2 years ago 
root@stfuplay:/volume1# cd test
root@stfuplay:/volume1/test# ls
Sample.class  sqlite-jdbc-3.39.2.0.jar  sqlite-jdbc-3.39.2.1-SNAPSHOT.jar
Sample.java   sqlite-jdbc-3.39.2.1.jar  wrapper
root@stfuplay:/volume1/test# chmod 777 * -r
chmod: cannot access '777': No such file or directory
root@stfuplay:/volume1/test# chmod 777 *.* -r
chmod: cannot access '777': No such file or directory
root@stfuplay:/volume1/test# chmod 777 *.*
root@stfuplay:/volume1/test# ls
Sample.class  sqlite-jdbc-3.39.2.0.jar  sqlite-jdbc-3.39.2.1-SNAPSHOT.jar
Sample.java   sqlite-jdbc-3.39.2.1.jar  wrapper
root@stfuplay:/volume1/test# java -Djava.io.tmpdir="/tmp/test" -classpath ".:
sqli
> ^C-jdbc-3.39.2.1-SNAPSHOT.jar" Sample
root@stfuplay:/volume1/test# java -Djava.io.tmpdir="/volume1/test" -classpath
 ".:sqli
> te-jdbc-3.39.2.1-SNAPSHOT.jar" Sample
No suitable driver found for jdbc:sqlite:sample.db
root@stfuplay:/volume1/test# java -Djava.io.tmpdir="/volume1/test" -classpath
 ".:sqlite-jdbc-3.39.2.1-SNAPSHOT.jar" Sample
Failed to load native library:sqlite-3.39.2.1-45c39a02-0c7b-4850-8418-18a227b
dc2ce-libsqlitejdbc.so. osinfo: Linux/armv7
java.lang.UnsatisfiedLinkError: /volume1/test/sqlite-3.39.2.1-45c39a02-0c7b-4
850-8418-18a227bdc2ce-libsqlitejdbc.so: /volume1/test/sqlite-3.39.2.1-45c39a0
2-0c7b-4850-8418-18a227bdc2ce-libsqlitejdbc.so: internal error
Error opening connection
root@stfuplay:/volume1/test#

Ok I tried a new directory and used 777 as well. Still same result.

mastervol commented 2 years ago

Maybe this helps: https://en.wikipedia.org/wiki/Comparison_of_ARMv7-A_processors I read that the STM Monaco STiH412 is supposed to have a Cortex A9 CPU, based on ARMv7 specification.

gotson commented 2 years ago

That's the current options used: https://github.com/xerial/sqlite-jdbc/blob/329fe0ae521af3353e29ee3888fac716c4281160/Makefile.common#L100

And we use this Dockcross image: dockcross/linux-armv7a-lts

The version i provided above was using dockcross/linux-armv7-lts, and -mfloat-abi=soft

mastervol commented 2 years ago

I am using gotify which works well on my synology ds216play. (https://github.com/gotify/server/releases/download/v2.1.5/gotify-linux-arm-7.zip) I think it also uses sqlite, maybe this helps with the building process?

https://gotify.net/docs/build

gotson commented 2 years ago

I am using gotify which works well on my synology ds216play. (https://github.com/gotify/server/releases/download/v2.1.5/gotify-linux-arm-7.zip) I think it also uses sqlite, maybe this helps with the building process?

https://gotify.net/docs/build

i checked what they have, but couldn't find anything useful for us

jmattheis commented 2 years ago

I've got a mail about this and found it interesting. It looks like the armv6 lib is actually for armv7l.

The armv7 docker images execution is emulated via https://github.com/multiarch/qemu-user-static. Gotify uses this cross-compiler for v7 https://github.com/gotify/build/blob/master/Dockerfile.linux-arm-7#L5

$ ll
.rwxr-xr-x  19M jm 27 Aug 13:31 gotify-linux-arm-7
.rw-r--r-- 1.7k jm 27 Aug 13:31 Sample.java
.rw-r--r--  12M jm 27 Aug 13:31 sqlite-jdbc-3.39.2.1-SNAPSHOT.jar
$ unzip sqlite-jdbc-3.39.2.1-SNAPSHOT.jar
$ rm sqlite-jdbc-3.39.2.1-SNAPSHOT.jar
$ docker run --rm -it -v $PWD:/a arm32v7/eclipse-temurin:11-jammy bash
root@c5c9338b68f8:/a# uname -m
armv7l
root@c5c9338b68f8:/a# ldd gotify-linux-arm-7
        libdl.so.2 => /lib/arm-linux-gnueabihf/libdl.so.2 (0x3f7e0000)
        libpthread.so.0 => /lib/arm-linux-gnueabihf/libpthread.so.0 (0x3f7c0000)
        libc.so.6 => /lib/arm-linux-gnueabihf/libc.so.6 (0x3f690000)
        /lib/ld-linux-armhf.so.3 (0x40000000)
root@c5c9338b68f8:/a# ./gotify-linux-arm-7
Starting Gotify version 2.1.5@2022-07-28-15:12:28
Started Listening for plain HTTP connection on :80
^C
root@c5c9338b68f8:/a# ldd org/sqlite/native/Linux/armv7/libsqlitejdbc.so
        not a dynamic executable
root@c5c9338b68f8:/a# ldd org/sqlite/native/Linux/armv6/libsqlitejdbc.so
        libc.so.6 => /lib/arm-linux-gnueabihf/libc.so.6 (0x3f5d0000)
        /lib/ld-linux-armhf.so.3 (0x40000000)
root@c5c9338b68f8:/a# export CLASSPATH="."
root@c5c9338b68f8:/a# javac Sample.java
root@c5c9338b68f8:/a# java Sample
Failed to load native library:sqlite-3.39.2.1-726884b8-66f4-442b-8c16-dba2d7369bb3-libsqlitejdbc.so. osinfo: Linux/armv7
java.lang.UnsatisfiedLinkError: /tmp/sqlite-3.39.2.1-726884b8-66f4-442b-8c16-dba2d7369bb3-libsqlitejdbc.so: /tmp/sqlite-3.39.2.1-726884b8-66f4-442b-8c16-dba2d7369bb3-libsqlitejdbc.so: cannot open shared object file: No such file or directory
Error opening connection
root@c5c9338b68f8:/a# cp org/sqlite/native/Linux/armv6/libsqlitejdbc.so org/sqlite/native/Linux/armv7/libsqlitejdbc.so
root@c5c9338b68f8:/a# java Sample
name = leo
id = 1
name = yui
id = 2

The error is a little bit different, but it's probably undefined behavior to load a library with a wrong arch.

gotson commented 2 years ago

the armv6 lib is actually for armv7l

Arm is backward compatible so the v6 library would work on v7

My feeling is that v7l doesn't support all the features of the current v7 compiled lib.

One way to fix it would be to detect v7l and downgrade it to v6 in the loader, since we can't figure out how to compile a v7 lib that works on that v7l.

gotson commented 2 years ago

@mastervol could you try the following:

mastervol commented 2 years ago

Ok here is the result: First with current path.

root@stfuplay:/volume1/@appstore/komga/test# ls
libsqlitejdbc.so  Sample.class  Sample.java  sqlite-jdbc-3.39.2.0.jar  sqlite-jdbc-3.39.2.1.jar  sqlite-jdbc-3.39.2.1-SNAPSHOT.jar  wrapper
root@stfuplay:/volume1/@appstore/komga/test# ls -ltr
total 35304
-rw-------  1 root  root  11993648 Aug  4 10:31 sqlite-jdbc-3.39.2.0.jar
drwx------  7 root  root      4096 Aug 15 09:28 wrapper
-rwxrwxrwx+ 1 share users 11994452 Aug 15 22:23 sqlite-jdbc-3.39.2.1-SNAPSHOT.jar
-rwxrwxrwx+ 1 share users     1652 Aug 15 22:31 Sample.java
-rw-------  1 root  root      2179 Aug 15 22:33 Sample.class
-rwx------  1 root  root  11994452 Aug 15 22:36 sqlite-jdbc-3.39.2.1.jar
-rw-------  1 root  root    136713 Aug 27 17:42 libsqlitejdbc.so
root@stfuplay:/volume1/@appstore/komga/test# mv libsqlitejdbc.so libsqlitejdbc-armv6.so
root@stfuplay:/volume1/@appstore/komga/test# java -Dorg.sqlite.lib.path=. -Dorg.sqlite.lib.name=libsqlitejdbc-armv6.so -classpath ".:sqlite-jdbc-3.39.2.1-SNAPSHOT.jar" Sample
OpenJDK Zero VM warning: You have loaded library /volume1/@appstore/komga/test/libsqlitejdbc-armv6.so which might have disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.
Failed to load native library:libsqlitejdbc-armv6.so. osinfo: Linux/armv7
java.lang.UnsatisfiedLinkError: /volume1/@appstore/komga/test/libsqlitejdbc-armv6.so: /volume1/@appstore/komga/test/libsqlitejdbc-armv6.so: invalid ELF header (Possible cause: endianness mismatch)
Error opening connection
root@stfuplay:/volume1/@appstore/komga/test#

Then with explicit path.

root@stfuplay:/volume1/@appstore/komga/test# java -Djava.io.tmpdir="/volume1/@appstore/komga/test" -Dorg.sqlite.lib.path=. -Dorg.sqlite.lib.name=libsqlitejdbc-armv6.so -
classpath ".:sqlite-jdbc-3.39.2.1-SNAPSHOT.jar" Sample
OpenJDK Zero VM warning: You have loaded library /volume1/@appstore/komga/test/libsqlitejdbc-armv6.so which might have disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.
Failed to load native library:libsqlitejdbc-armv6.so. osinfo: Linux/armv7
java.lang.UnsatisfiedLinkError: /volume1/@appstore/komga/test/libsqlitejdbc-armv6.so: /volume1/@appstore/komga/test/libsqlitejdbc-armv6.so: invalid ELF header (Possible cause: endianness mismatch)
Error opening connection
root@stfuplay:/volume1/@appstore/komga/test#
gotson commented 2 years ago

Thanks. For the sake of exhaustiveness and experiment, could you try with this one too? https://github.com/xerial/sqlite-jdbc/tree/master/src/main/resources/org/sqlite/native/Linux/arm It's for arm v5

mastervol commented 2 years ago

Hm it is the same result.

-rw-------  1 root  root    136603 Aug 28 09:14 libsqlitejdbc.so
root@stfuplay:/volume1/@appstore/komga/test# mv libsqlitejdbc.so libsqlitejdbc-armv5.so
root@stfuplay:/volume1/@appstore/komga/test# java -Dorg.sqlite.lib.path=. -Dorg.sqlite.lib.name=libsqlitejdbc-armv5.so -classpath ".:sqlite-jdbc-3.39.2.1-SNAPSHOT.jar" Sample
OpenJDK Zero VM warning: You have loaded library /volume1/@appstore/komga/test/libsqlitejdbc-armv5.so which might have disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.
Failed to load native library:libsqlitejdbc-armv5.so. osinfo: Linux/armv7
java.lang.UnsatisfiedLinkError: /volume1/@appstore/komga/test/libsqlitejdbc-armv5.so: /volume1/@appstore/komga/test/libsqlitejdbc-armv5.so: invalid ELF header (Possible cause: endianness mismatch)
Error opening connection
root@stfuplay:/volume1/@appstore/komga/test# chmod 777 *.so
root@stfuplay:/volume1/@appstore/komga/test# java -Dorg.sqlite.lib.path=. -Dorg.sqlite.lib.name=libsqlitejdbc-armv5.so -classpath ".:sqlite-jdbc-3.39.2.1-SNAPSHOT.jar" Sample
OpenJDK Zero VM warning: You have loaded library /volume1/@appstore/komga/test/libsqlitejdbc-armv5.so which might have disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.
Failed to load native library:libsqlitejdbc-armv5.so. osinfo: Linux/armv7
java.lang.UnsatisfiedLinkError: /volume1/@appstore/komga/test/libsqlitejdbc-armv5.so: /volume1/@appstore/komga/test/libsqlitejdbc-armv5.so: invalid ELF header (Possible cause: endianness mismatch)
Error opening connection
root@stfuplay:/volume1/@appstore/komga/test#
gotson commented 2 years ago

One solution would be to compile the library on your system, and use that using the aforementioned property to load it explicitly. I don't have any idea on how to cross compile a lib for your cpu unfortunately.

mastervol commented 2 years ago

Ok I have never cross compiled anything, but what I found is this. I assume these are the files of the tool chain I would need in order to cross compile on my hardware. https://sourceforge.net/projects/dsgpl/files/Tool%20Chain/DSM%207.0.0%20Tool%20Chains/STMicroelectronics%20Monaco%20Linux%203.10.108/monaco-gcc750_glibc226_hard-GPL.txz/download or rather and appropriate seems: https://global.download.synology.com/download/ToolChain/toolchain/7.1-42661/STMicroelectronics%20Monaco%20Linux%203.10.108/monaco-gcc850_glibc226_hard-GPL.txz

I have yet to find further documentation about the process or ask someone of the syno community ;-) Some basic information I found is here: https://help.synology.com/developer-guide/compile_applications/compile.html

gotson commented 2 years ago

You only need to cross-compile when you compile on a different hardware than what you target. Since you own the target hardware, if you compile directly on that, then you just compile.

You can check the guide here, you will need to install a few things, not sure how easy that is on a Synology. Then it's just a matter or running make native.

mastervol commented 2 years ago

Ok I think I have everything from this list you linked, not sure about the maven error though. It seems it is not relevant looking at the output of "make native".

root@stfuplay:/# /volume1/share/maven/check.sh
+ java --version
openjdk 11.0.12 2021-07-20 LTS
OpenJDK Runtime Environment (build 11.0.12+7-LTS)
OpenJDK Zero VM (build 11.0.12+7-LTS, interpreted mode)
+ perl --version

This is perl 5, version 28, subversion 3 (v5.28.3) built for armle-linux
(with 1 registered patch, see perl -V for more detail)

Copyright 1987-2020, Larry Wall

Perl may be copied only under the terms of either the Artistic License or the
GNU General Public License, which may be found in the Perl 5 source kit.

Complete documentation for Perl, including FAQ lists, should be found on
this system using "man perl" or "perldoc perl".  If you have access to the
Internet, point your browser at http://www.perl.org/, the Perl Home Page.

+ make --version
GNU Make 4.3
Built for arm-openwrt-linux-gnu
Copyright (C) 1988-2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
+ gcc --version
gcc (OpenWrt GCC 8.4.0) 8.4.0
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

+ curl --version
curl 7.79.1 (armle-unknown-linux-gnu) libcurl/7.79.1 OpenSSL/1.1.1n zlib/1.2.11 c-ares/1.14.0 nghttp2/1.41.0
Release-Date: 2021-09-22
Protocols: http https
Features: alt-svc AsynchDNS Debug HSTS HTTP2 HTTPS-proxy IPv6 Largefile libz NTLM NTLM_WB SSL TrackMemory UnixSockets
+ unzip -v
UnZip 6.00 of 20 April 2009, by Info-ZIP.  Maintained by C. Spieler.  Send
bug reports using http://www.info-zip.org/zip-bug.html; see README for details.

Latest sources and executables are at ftp://ftp.info-zip.org/pub/infozip/ ;
see ftp://ftp.info-zip.org/pub/infozip/UnZip.html for other sites.

Compiled with gcc 8.4.0 for Unix (Linux ELF).

UnZip special compilation options:
        COPYRIGHT_CLEAN (PKZIP 0.9x unreducing method not supported)
        SET_DIR_ATTRIB
        SYMLINKS (symbolic links supported, if RTL and file system permit)
        TIMESTAMP
        UNIXBACKUP
        USE_EF_UT_TIME
        USE_UNSHRINK (PKZIP/Zip 1.x unshrinking method supported)
        USE_DEFLATE64 (PKZIP 4.x Deflate64(tm) supported)
        UNICODE_SUPPORT [wide-chars, char coding: UTF-8] (handle UTF-8 paths)
        LARGE_FILE_SUPPORT (large files over 2 GiB supported)
        ZIP64_SUPPORT (archives using Zip64 for large files supported)
        VMS_TEXT_CONV
        [decryption, version 2.11 of 05 Jan 2007]

UnZip and ZipInfo environment options:
           UNZIP:  [none]
        UNZIPOPT:  [none]
         ZIPINFO:  [none]
      ZIPINFOOPT:  [none]
+ /volume1/share/maven/apache-maven-3.8.6/bin/mvn -v -Djava.io.tmpdir=/volume1/share/temp
Failed to load native library:jansi-2.4.0-79ef318959f261d8-libjansi.so. The native library file at /tmp/jansi-2.4.0-79ef318959f261d8-libjansi.so is not executable, make sure that the directory is mounted on a partition without the noexec flag, or set the jansi.tmpdir system property to point to a proper location.  osinfo: Linux/armv7
java.lang.UnsatisfiedLinkError: /tmp/jansi-2.4.0-79ef318959f261d8-libjansi.so: /tmp/jansi-2.4.0-79ef318959f261d8-libjansi.so: failed to map segment from shared object
Apache Maven 3.8.6 (84538c9988a25aec085021c365c560670ad80f63)
Maven home: /volume1/share/maven/apache-maven-3.8.6
Java version: 11.0.12, vendor: Oracle Corporation, runtime: /volume1/@appstore/java-11-openjdk/jvm/openjdk-11.0.12
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "3.10.108", arch: "arm", family: "unix"
root@stfuplay:/#

make native:

root@stfuplay:/volume1/share/maven/sqlite/sqlite-jdbc-3.39.2.1# export JAVA_HOME="/usr/local"
root@stfuplay:/volume1/share/maven/sqlite/sqlite-jdbc-3.39.2.1# echo $JAVA_HOME
/usr/local
root@stfuplay:/volume1/share/maven/sqlite/sqlite-jdbc-3.39.2.1# make native
Building OSInfo tool
Will build using target: Linux-armv7 (detected os: Linux arch: armv7)
perl -p -e "s/sqlite3_api;/sqlite3_api = 0;/g" \
    target/sqlite-amalgamation-3390200/sqlite3ext.h > target/sqlite-3.39.2-Linux-armv7/sqlite3ext.h
perl -p -e "s/^opendb_out:/  if(!db->mallocFailed && rc==SQLITE_OK){ rc = RegisterExtensionFunctions(db); }\nopendb_out:/;" \
    target/sqlite-amalgamation-3390200/sqlite3.c > target/sqlite-3.39.2-Linux-armv7/sqlite3.c.tmp
perl -p -e "s/^(static const char \* const sqlite3azCompileOpt.+)$/\1\n\n\/* This has been automatically added by sqlite-jdbc *\/\n  \"JDBC_EXTENSIONS\",/;" \
    target/sqlite-3.39.2-Linux-armv7/sqlite3.c.tmp > target/sqlite-3.39.2-Linux-armv7/sqlite3.c
cat src/main/ext/*.c >> target/sqlite-3.39.2-Linux-armv7/sqlite3.c
gcc -o target/sqlite-3.39.2-Linux-armv7/sqlite3.o -c -Itarget/sqlite-3.39.2-Linux-armv7 -Itarget/sqlite-amalgamation-3390200 -I/usr/local/include -Ilib/inc_linux -Os -mfloat-abi=hard -mfpu=vfp -fPIC -fvisibility=hidden   \
    -DSQLITE_ENABLE_LOAD_EXTENSION=1 \
    -DSQLITE_HAVE_ISNAN \
    -DHAVE_USLEEP=1 \
    -DSQLITE_ENABLE_COLUMN_METADATA \
    -DSQLITE_CORE \
    -DSQLITE_ENABLE_FTS3 \
    -DSQLITE_ENABLE_FTS3_PARENTHESIS \
    -DSQLITE_ENABLE_FTS5 \
    -DSQLITE_ENABLE_RTREE \
    -DSQLITE_ENABLE_STAT4 \
    -DSQLITE_ENABLE_DBSTAT_VTAB \
    -DSQLITE_THREADSAFE=1 \
    -DSQLITE_DEFAULT_MEMSTATUS=0 \
    -DSQLITE_DEFAULT_FILE_PERMISSIONS=0666 \
    -DSQLITE_MAX_VARIABLE_NUMBER=250000 \
    -DSQLITE_MAX_MMAP_SIZE=1099511627776 \
    -DSQLITE_MAX_LENGTH=2147483647 \
    -DSQLITE_MAX_COLUMN=32767 \
    -DSQLITE_MAX_SQL_LENGTH=1073741824 \
    -DSQLITE_MAX_FUNCTION_ARG=127 \
    -DSQLITE_MAX_ATTACHED=125 \
    -DSQLITE_MAX_PAGE_COUNT=4294967294 \
     \
    target/sqlite-3.39.2-Linux-armv7/sqlite3.c
In file included from /opt/include/features.h:452,
                 from /opt/include/bits/libc-header-start.h:33,
                 from /opt/include/stdio.h:27,
                 from target/sqlite-3.39.2-Linux-armv7/sqlite3.c:14044:
/opt/include/gnu/stubs.h:10:11: fatal error: gnu/stubs-hard.h: No such file or directory
 # include <gnu/stubs-hard.h>
           ^~~~~~~~~~~~~~~~~~
compilation terminated.
make: *** [Makefile:68: target/sqlite-3.39.2-Linux-armv7/sqlite3.o] Error 1
root@stfuplay:/volume1/share/maven/sqlite/sqlite-jdbc-3.39.2.1#
gotson commented 2 years ago

I reported the same issue when trying a different dockcross image.

Try to change on this line: -mfloat-abi=hard with -mfloat-abi=softfp or -mfloat-abi=soft.

mastervol commented 2 years ago

Ok I guess this is just a warning which is not important? This happens for both -mfloat-abi=softfp and -mfloat-abi=soft options.

gcc -o target/sqlite-3.39.2-Linux-armv7/sqlite3.o -c -Itarget/sqlite-3.39.2-Linux-armv7 -Itarget/sqlite-amalgamation-3390200 -I/usr/local/include -Ilib/inc_linux -Os -mfloat-abi=soft -mfpu=vfp -fPIC -fvisibility=hidden   \
    -DSQLITE_ENABLE_LOAD_EXTENSION=1 \
    -DSQLITE_HAVE_ISNAN \
    -DHAVE_USLEEP=1 \
    -DSQLITE_ENABLE_COLUMN_METADATA \
    -DSQLITE_CORE \
    -DSQLITE_ENABLE_FTS3 \
    -DSQLITE_ENABLE_FTS3_PARENTHESIS \
    -DSQLITE_ENABLE_FTS5 \
    -DSQLITE_ENABLE_RTREE \
    -DSQLITE_ENABLE_STAT4 \
    -DSQLITE_ENABLE_DBSTAT_VTAB \
    -DSQLITE_THREADSAFE=1 \
    -DSQLITE_DEFAULT_MEMSTATUS=0 \
    -DSQLITE_DEFAULT_FILE_PERMISSIONS=0666 \
    -DSQLITE_MAX_VARIABLE_NUMBER=250000 \
    -DSQLITE_MAX_MMAP_SIZE=1099511627776 \
    -DSQLITE_MAX_LENGTH=2147483647 \
    -DSQLITE_MAX_COLUMN=32767 \
    -DSQLITE_MAX_SQL_LENGTH=1073741824 \
    -DSQLITE_MAX_FUNCTION_ARG=127 \
    -DSQLITE_MAX_ATTACHED=125 \
    -DSQLITE_MAX_PAGE_COUNT=4294967294 \
     \
    target/sqlite-3.39.2-Linux-armv7/sqlite3.c
target/sqlite-3.39.2-Linux-armv7/sqlite3.c: In function 'openDatabase':
target/sqlite-3.39.2-Linux-armv7/sqlite3.c:174069:48: warning: implicit declaration of function 'RegisterExtensionFunctions'; did you mean 'sqlite3AlterFunctions'? [-Wimplicit-function-declaration]
   if(!db->mallocFailed && rc==SQLITE_OK){ rc = RegisterExtensionFunctions(db); }
"$JAVA_HOME/bin/javac"  -d target/common-lib -sourcepath src/main/java -h target/common-lib src/main/java/org/sqlite/core/NativeDB.java
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: src/main/java/org/sqlite/jdbc3/JDBC3Connection.java uses unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
mv target/common-lib/org_sqlite_core_NativeDB.h target/common-lib/NativeDB.h
gcc -Itarget/sqlite-3.39.2-Linux-armv7 -Itarget/sqlite-amalgamation-3390200 -I/usr/local/include -Ilib/inc_linux -Os -mfloat-abi=softfp -mfpu=vfp -fPIC -fvisibility=hidden   -I target/common-lib -c -o target/sqlite-3.39.2-Linux-armv7/NativeDB.o src/main/java/org/sqlite/core/NativeDB.c
gcc -Itarget/sqlite-3.39.2-Linux-armv7 -Itarget/sqlite-amalgamation-3390200 -I/usr/local/include -Ilib/inc_linux -Os -mfloat-abi=softfp -mfpu=vfp -fPIC -fvisibility=hidden   -o target/sqlite-3.39.2-Linux-armv7/libsqlitejdbc.so target/sqlite-3.39.2-Linux-armv7/NativeDB.o target/sqlite-3.39.2-Linux-armv7/sqlite3.o -shared -static-libgcc
cp target/sqlite-3.39.2-Linux-armv7/libsqlitejdbc.so /tmp/libsqlitejdbc.so
strip /tmp/libsqlitejdbc.so
cp /tmp/libsqlitejdbc.so target/sqlite-3.39.2-Linux-armv7/libsqlitejdbc.so
cp target/sqlite-3.39.2-Linux-armv7/libsqlitejdbc.so src/main/resources/org/sqlite/native/Linux/armv7/libsqlitejdbc.so
cp target/sqlite-3.39.2-Linux-armv7/libsqlitejdbc.so target/classes/org/sqlite/native/Linux/armv7/libsqlitejdbc.so

Here is the test result:

root@stfuplay:/volume1/@appstore/komga/test# root@stfuplay:/volume1/@appstore/komga/test# ls -ltr
total 37196
-rw-------  1 root  root  11993648 Aug  4 10:31 sqlite-jdbc-3.39.2.0.jar
drwx------  7 root  root      4096 Aug 15 09:28 wrapper
-rwxrwxrwx+ 1 share users     1652 Aug 15 22:31 Sample.java
-rw-------  1 root  root      2179 Aug 15 22:33 Sample.class
-rwx------  1 root  root  11994452 Aug 15 22:36 sqlite-jdbc-3.39.2.1.jar
-rwxrwxrwx  1 root  root    136713 Aug 27 17:42 libsqlitejdbc-armv6.so
-rwxrwxrwx+ 1 share users 11994452 Aug 27 18:03 sqlite-jdbc-3.39.2.1-SNAPSHOT.jar
-rwxrwxrwx  1 root  root    136603 Aug 28 09:14 libsqlitejdbc-armv5.so
-rwxrwxrwx+ 1 root  root    889280 Aug 31 08:05 libsqlitejdbc-arm7-softfp.so
-rwx------  1 root  root    893376 Aug 31 14:40 libsqlitejdbc-arm7-soft.so
-rwx------  1 root  root       292 Aug 31 14:44 test.sh
root@stfuplay:/volume1/@appstore/komga/test# ./test.sh
+ java -Dorg.sqlite.lib.path=. -Dorg.sqlite.lib.name=libsqlitejdbc-arm7-soft.so -classpath .:sqlite-jdbc-3.39.2.1-SNAPSHOT.jar Sample
Failed to load native library:libsqlitejdbc-arm7-soft.so. osinfo: Linux/armv7
java.lang.UnsatisfiedLinkError: /volume1/@appstore/komga/test/libsqlitejdbc-arm7-soft.so: /volume1/@appstore/komga/test/libsqlitejdbc-arm7-soft.so: internal error
Error opening connection
+ java -Dorg.sqlite.lib.path=. -Dorg.sqlite.lib.name=libsqlitejdbc-arm7-softfp.so -classpath .:sqlite-jdbc-3.39.2.1-SNAPSHOT.jar Sample
Failed to load native library:libsqlitejdbc-arm7-softfp.so. osinfo: Linux/armv7
java.lang.UnsatisfiedLinkError: /volume1/@appstore/komga/test/libsqlitejdbc-arm7-softfp.so: /volume1/@appstore/komga/test/libsqlitejdbc-arm7-softfp.so: internal error
Error opening connection
root@stfuplay:/volume1/@appstore/komga/test#

Any other hint you have for building or debugging this? Thanks.

gotson commented 2 years ago

Ok I guess this is just a warning which is not important?

The warning is indeed not important, happens for other platforms too, you can safely ignore it.

Let's try to remove the specifics of armv7 for native compilation and use the default for Linux/Unix.

In Makefile.common replace this line with this:

known_targets :=

Then try to run make native again.

mastervol commented 2 years ago

Still the same error, somehow some link is not satisfied.

I removed the known targets in the file Makefile.common, leaving it default as you described.

+ java -Dorg.sqlite.lib.path=. -Dorg.sqlite.lib.name=libsqlitejdbc-generic-native.so -classpath .:sqlite-jdbc-3.39.2.1-SNAPSHOT.jar Sample
Failed to load native library:libsqlitejdbc-generic-native.so. osinfo: Linux/armv7
java.lang.UnsatisfiedLinkError: /volume1/@appstore/komga/test/libsqlitejdbc-generic-native.so: /volume1/@appstore/komga/test/libsqlitejdbc-generic-native.so: internal error
Error opening connection
root@stfuplay:/volume1/@appstore/komga/test#
gotson commented 2 years ago

Could you ldd all those libs you built and post the results here?

mastervol commented 2 years ago 
root@stfuplay:/volume1/@appstore/komga/test# ldd
Sample.class                       libsqlitejdbc-armv5.so             sqlite-jdbc-3.39.2.1-SNAPSHOT.jar
Sample.java                        libsqlitejdbc-armv6.so             sqlite-jdbc-3.39.2.1.jar
libsqlitejdbc-arm7-soft.so         libsqlitejdbc-generic-native.so    test.sh
libsqlitejdbc-arm7-softfp.so       sqlite-jdbc-3.39.2.0.jar           wrapper/
root@stfuplay:/volume1/@appstore/komga/test# ldd libsqlitejdbc-generic-native.so
        libgcc_s.so.1 => /opt/lib/libgcc_s.so.1 (0x76e40000)
        libc.so.6 => /opt/lib/libc.so.6 (0x76cfe000)
        /opt/lib/ld-linux.so.3 (0x76f47000)
root@stfuplay:/volume1/@appstore/komga/test# ldd libsqlitejdbc-arm7-soft.so
        libc.so.6 => /opt/lib/libc.so.6 (0x76d4b000)
        /opt/lib/ld-linux.so.3 (0x76f78000)
root@stfuplay:/volume1/@appstore/komga/test# ldd libsqlitejdbc-arm7-softfp.so
        libc.so.6 => /opt/lib/libc.so.6 (0x76d40000)
        /opt/lib/ld-linux.so.3 (0x76f6c000)
root@stfuplay:/volume1/@appstore/komga/test# ldd libsqlitejdbc-armv6.so
        not a dynamic executable
root@stfuplay:/volume1/@appstore/komga/test# ldd libsqlitejdbc-armv5.so
        not a dynamic executable
root@stfuplay:/volume1/@appstore/komga/test#

Most of the binaries come from entware which I installed via: https://github.com/Entware/Entware/wiki/Install-on-Synology-NAS. [ARM & intel based devices](https://www.synology.com/en-global/knowledgebase/DSM/tutorial/General/What_kind_of_CPU_does_my_NAS_have) are supported. DSM version must be 3.2 or higher. Use armv5 installer for Marvell Kirkwood mv6282 cpu, you can use armv7 installer for newer ARM cpu's. Although armv7 repo was compiled with cortex-a9 optimization it was successfully tested on Marvell Armada XP based NAS. Use aarch64 installer for Realtek RTD129x CPU (64 bit arm).

Comparison of gcc I installed via ENTWARE and gcc from toolchain for my specific hardware: gcc ENTWARE (generic arm7):

root@stfuplay:/volume1/@appstore/komga/toolchain/arm-unknown-linux-gnueabi# gcc --version
gcc (OpenWrt GCC 8.4.0) 8.4.0
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

gcc from synology https://global.download.synology.com/download/ToolChain/toolchain/7.1-42661/STMicroelectronics%20Monaco%20Linux%203.10.108/monaco-gcc850_glibc226_hard-GPL.txz. Ok however I cannot execute those binaries somehow and opened a ticket at synology.com.


root@stfuplay:/volume1/@appstore/komga/toolchain/arm-unknown-linux-gnueabi/bin# ls
arm-unknown-linux-gnueabi-addr2line     arm-unknown-linux-gnueabi-gcc-8.5.0   arm-unknown-linux-gnueabi-ld.gold
arm-unknown-linux-gnueabi-ar            arm-unknown-linux-gnueabi-gcc-ar      arm-unknown-linux-gnueabi-nm
arm-unknown-linux-gnueabi-as            arm-unknown-linux-gnueabi-gcc-nm      arm-unknown-linux-gnueabi-objcopy
arm-unknown-linux-gnueabi-c++           arm-unknown-linux-gnueabi-gcc-ranlib  arm-unknown-linux-gnueabi-objdump
arm-unknown-linux-gnueabi-cc            arm-unknown-linux-gnueabi-gcov        arm-unknown-linux-gnueabi-populate
arm-unknown-linux-gnueabi-c++filt       arm-unknown-linux-gnueabi-gcov-dump   arm-unknown-linux-gnueabi-ranlib
arm-unknown-linux-gnueabi-cpp           arm-unknown-linux-gnueabi-gcov-tool   arm-unknown-linux-gnueabi-readelf
arm-unknown-linux-gnueabi-ct-ng.config  arm-unknown-linux-gnueabi-gfortran    arm-unknown-linux-gnueabi-size
arm-unknown-linux-gnueabi-dwp           arm-unknown-linux-gnueabi-gprof       arm-unknown-linux-gnueabi-strings
arm-unknown-linux-gnueabi-elfedit       arm-unknown-linux-gnueabi-ld          arm-unknown-linux-gnueabi-strip
arm-unknown-linux-gnueabi-g++           arm-unknown-linux-gnueabi-ld.bfd
arm-unknown-linux-gnueabi-gcc           arm-unknown-linux-gnueabi-ldd
root@stfuplay:/volume1/@appstore/komga/toolchain/arm-unknown-linux-gnueabi/bin# ./arm-unknown-linux-gnueabi-gcc
-ash: ./arm-unknown-linux-gnueabi-gcc: cannot execute binary file: Exec format error
mastervol commented 2 years ago

Synolog support said that the toolchains are not supposed to be installed on the synology nas.

Anything else I can try here? I found this also, maybe this helps: https://groups.google.com/g/sqlite_users/c/QvlhRBeNnx0?pli=1

noguespi commented 1 year ago

I've got the same issue armv7l

image

But isn't the library supposed to fallback on a pure-java implementation when the native library doesn't work ?

gotson commented 1 year ago

Pure Java was discontinued many years ago

noguespi commented 1 year ago

OK no big deal. I think there will be less and less of armv7l architecture anyway so it's not worth it to invest in supporting it.

lgtti commented 12 months ago

Please reconsider this issue because in IoT world armv7 (and armv7l) is used due to stability and power consumption. Many industrial gateways runs on this architecture (an example https://www.moxa.com/en/products/industrial-computing/arm-based-computers/uc-8200-series).