search

coleifer / pysqlite3

SQLite3 DB-API 2.0 driver from Python 3, packaged separately, with improvements
zlib License
189 stars 54 forks source link

Symbol not found: _sqlite3_enable_load_extension (on OS X) #1

Closed simonw closed 6 years ago

simonw commented 6 years ago

I tried to install this in a virtual environment on OS X and got the following error:

>>> import pysqlite3
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/private/tmp/pysqlite3/v/lib/python3.6/site-packages/pysqlite3-0.1.4-py3.6-macosx-10.13-x86_64.egg/pysqlite3/__init__.py", line 23, in <module>
    from pysqlite3.dbapi2 import *
  File "/private/tmp/pysqlite3/v/lib/python3.6/site-packages/pysqlite3-0.1.4-py3.6-macosx-10.13-x86_64.egg/pysqlite3/dbapi2.py", line 28, in <module>
    from pysqlite3._sqlite3 import *
ImportError: dlopen(/private/tmp/pysqlite3/v/lib/python3.6/site-packages/pysqlite3-0.1.4-py3.6-macosx-10.13-x86_64.egg/pysqlite3/_sqlite3.cpython-36m-darwin.so, 2): Symbol not found: _sqlite3_enable_load_extension
  Referenced from: /private/tmp/pysqlite3/v/lib/python3.6/site-packages/pysqlite3-0.1.4-py3.6-macosx-10.13-x86_64.egg/pysqlite3/_sqlite3.cpython-36m-darwin.so
  Expected in: flat namespace
 in /private/tmp/pysqlite3/v/lib/python3.6/site-packages/pysqlite3-0.1.4-py3.6-macosx-10.13-x86_64.egg/pysqlite3/_sqlite3.cpython-36m-darwin.so

Here's the full transcript:

tmp $ git clone https://github.com/coleifer/pysqlite3.git
Cloning into 'pysqlite3'...
remote: Counting objects: 86, done.
remote: Total 86 (delta 0), reused 0 (delta 0), pack-reused 86
Unpacking objects: 100% (86/86), done.
tmp $ cd pysqlite3/
pysqlite3 $ ls
LICENSE     README.rst  setup.cfg   src
MANIFEST.in lib     setup.py
pysqlite3 $ python3 -mvenv v
source v/bin/activatepysqlite3 $ source v/bin/activate
(v) pysqlite3 $ python setup.py install
running install
running bdist_egg
running egg_info
creating pysqlite3.egg-info
writing pysqlite3.egg-info/PKG-INFO
writing dependency_links to pysqlite3.egg-info/dependency_links.txt
writing top-level names to pysqlite3.egg-info/top_level.txt
writing manifest file 'pysqlite3.egg-info/SOURCES.txt'
reading manifest file 'pysqlite3.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no previously-included files matching '*~' found anywhere in distribution
warning: no previously-included files matching '*.pyc' found anywhere in distribution
writing manifest file 'pysqlite3.egg-info/SOURCES.txt'
installing library code to build/bdist.macosx-10.13-x86_64/egg
running install_lib
running build_py
creating build
creating build/lib.macosx-10.13-x86_64-3.6
creating build/lib.macosx-10.13-x86_64-3.6/pysqlite3
copying lib/__init__.py -> build/lib.macosx-10.13-x86_64-3.6/pysqlite3
copying lib/dump.py -> build/lib.macosx-10.13-x86_64-3.6/pysqlite3
copying lib/dbapi2.py -> build/lib.macosx-10.13-x86_64-3.6/pysqlite3
running build_ext
Builds a C extension linking against libsqlite3 library
building 'pysqlite3._sqlite3' extension
creating build/temp.macosx-10.13-x86_64-3.6
creating build/temp.macosx-10.13-x86_64-3.6/src
clang -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -Qunused-arguments -DMODULE_NAME="pysqlite3.dbapi2" -I/usr/include -I/private/tmp/pysqlite3/v/include -I/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/include/python3.6m -c src/module.c -o build/temp.macosx-10.13-x86_64-3.6/src/module.o
src/module.c:147:10: warning: 'sqlite3_enable_shared_cache' is deprecated: first
      deprecated in macOS 10.7 - Not supported [-Wdeprecated-declarations]
    rc = sqlite3_enable_shared_cache(do_enable);
         ^
/usr/include/sqlite3.h:5541:16: note: 'sqlite3_enable_shared_cache' has been
      explicitly marked deprecated here
SQLITE_API int sqlite3_enable_shared_cache(int);
               ^
1 warning generated.
clang -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -Qunused-arguments -DMODULE_NAME="pysqlite3.dbapi2" -I/usr/include -I/private/tmp/pysqlite3/v/include -I/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/include/python3.6m -c src/connection.c -o build/temp.macosx-10.13-x86_64-3.6/src/connection.o
src/connection.c:1044:9: warning: 'sqlite3_trace' is deprecated: first
      deprecated in macOS 10.12 [-Wdeprecated-declarations]
        sqlite3_trace(self->db, 0, (void*)0);
        ^~~~~~~~~~~~~
        sqlite3_trace_v2
/usr/include/sqlite3.h:2907:36: note: 'sqlite3_trace' has been explicitly marked
      deprecated here
SQLITE_API SQLITE_DEPRECATED void *sqlite3_trace(
                                   ^
src/connection.c:1048:9: warning: 'sqlite3_trace' is deprecated: first
      deprecated in macOS 10.12 [-Wdeprecated-declarations]
        sqlite3_trace(self->db, _trace_callback, trace_callback);
        ^~~~~~~~~~~~~
        sqlite3_trace_v2
/usr/include/sqlite3.h:2907:36: note: 'sqlite3_trace' has been explicitly marked
      deprecated here
SQLITE_API SQLITE_DEPRECATED void *sqlite3_trace(
                                   ^
2 warnings generated.
clang -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -Qunused-arguments -DMODULE_NAME="pysqlite3.dbapi2" -I/usr/include -I/private/tmp/pysqlite3/v/include -I/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/include/python3.6m -c src/cursor.c -o build/temp.macosx-10.13-x86_64-3.6/src/cursor.o
clang -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -Qunused-arguments -DMODULE_NAME="pysqlite3.dbapi2" -I/usr/include -I/private/tmp/pysqlite3/v/include -I/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/include/python3.6m -c src/cache.c -o build/temp.macosx-10.13-x86_64-3.6/src/cache.o
clang -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -Qunused-arguments -DMODULE_NAME="pysqlite3.dbapi2" -I/usr/include -I/private/tmp/pysqlite3/v/include -I/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/include/python3.6m -c src/microprotocols.c -o build/temp.macosx-10.13-x86_64-3.6/src/microprotocols.o
clang -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -Qunused-arguments -DMODULE_NAME="pysqlite3.dbapi2" -I/usr/include -I/private/tmp/pysqlite3/v/include -I/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/include/python3.6m -c src/prepare_protocol.c -o build/temp.macosx-10.13-x86_64-3.6/src/prepare_protocol.o
clang -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -Qunused-arguments -DMODULE_NAME="pysqlite3.dbapi2" -I/usr/include -I/private/tmp/pysqlite3/v/include -I/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/include/python3.6m -c src/statement.c -o build/temp.macosx-10.13-x86_64-3.6/src/statement.o
clang -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -Qunused-arguments -DMODULE_NAME="pysqlite3.dbapi2" -I/usr/include -I/private/tmp/pysqlite3/v/include -I/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/include/python3.6m -c src/util.c -o build/temp.macosx-10.13-x86_64-3.6/src/util.o
clang -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -Qunused-arguments -DMODULE_NAME="pysqlite3.dbapi2" -I/usr/include -I/private/tmp/pysqlite3/v/include -I/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/include/python3.6m -c src/row.c -o build/temp.macosx-10.13-x86_64-3.6/src/row.o
clang -bundle -undefined dynamic_lookup -Qunused-arguments build/temp.macosx-10.13-x86_64-3.6/src/module.o build/temp.macosx-10.13-x86_64-3.6/src/connection.o build/temp.macosx-10.13-x86_64-3.6/src/cursor.o build/temp.macosx-10.13-x86_64-3.6/src/cache.o build/temp.macosx-10.13-x86_64-3.6/src/microprotocols.o build/temp.macosx-10.13-x86_64-3.6/src/prepare_protocol.o build/temp.macosx-10.13-x86_64-3.6/src/statement.o build/temp.macosx-10.13-x86_64-3.6/src/util.o build/temp.macosx-10.13-x86_64-3.6/src/row.o -L/usr/lib -lsqlite3 -o build/lib.macosx-10.13-x86_64-3.6/pysqlite3/_sqlite3.cpython-36m-darwin.so
creating build/bdist.macosx-10.13-x86_64
creating build/bdist.macosx-10.13-x86_64/egg
creating build/bdist.macosx-10.13-x86_64/egg/pysqlite3
copying build/lib.macosx-10.13-x86_64-3.6/pysqlite3/__init__.py -> build/bdist.macosx-10.13-x86_64/egg/pysqlite3
copying build/lib.macosx-10.13-x86_64-3.6/pysqlite3/_sqlite3.cpython-36m-darwin.so -> build/bdist.macosx-10.13-x86_64/egg/pysqlite3
copying build/lib.macosx-10.13-x86_64-3.6/pysqlite3/dump.py -> build/bdist.macosx-10.13-x86_64/egg/pysqlite3
copying build/lib.macosx-10.13-x86_64-3.6/pysqlite3/dbapi2.py -> build/bdist.macosx-10.13-x86_64/egg/pysqlite3
byte-compiling build/bdist.macosx-10.13-x86_64/egg/pysqlite3/__init__.py to __init__.cpython-36.pyc
byte-compiling build/bdist.macosx-10.13-x86_64/egg/pysqlite3/dump.py to dump.cpython-36.pyc
byte-compiling build/bdist.macosx-10.13-x86_64/egg/pysqlite3/dbapi2.py to dbapi2.cpython-36.pyc
creating stub loader for pysqlite3/_sqlite3.cpython-36m-darwin.so
byte-compiling build/bdist.macosx-10.13-x86_64/egg/pysqlite3/_sqlite3.py to _sqlite3.cpython-36.pyc
creating build/bdist.macosx-10.13-x86_64/egg/EGG-INFO
copying pysqlite3.egg-info/PKG-INFO -> build/bdist.macosx-10.13-x86_64/egg/EGG-INFO
copying pysqlite3.egg-info/SOURCES.txt -> build/bdist.macosx-10.13-x86_64/egg/EGG-INFO
copying pysqlite3.egg-info/dependency_links.txt -> build/bdist.macosx-10.13-x86_64/egg/EGG-INFO
copying pysqlite3.egg-info/top_level.txt -> build/bdist.macosx-10.13-x86_64/egg/EGG-INFO
writing build/bdist.macosx-10.13-x86_64/egg/EGG-INFO/native_libs.txt
zip_safe flag not set; analyzing archive contents...
pysqlite3.__pycache__._sqlite3.cpython-36: module references __file__
creating dist
creating 'dist/pysqlite3-0.1.4-py3.6-macosx-10.13-x86_64.egg' and adding 'build/bdist.macosx-10.13-x86_64/egg' to it
removing 'build/bdist.macosx-10.13-x86_64/egg' (and everything under it)
Processing pysqlite3-0.1.4-py3.6-macosx-10.13-x86_64.egg
creating /private/tmp/pysqlite3/v/lib/python3.6/site-packages/pysqlite3-0.1.4-py3.6-macosx-10.13-x86_64.egg
Extracting pysqlite3-0.1.4-py3.6-macosx-10.13-x86_64.egg to /private/tmp/pysqlite3/v/lib/python3.6/site-packages
Adding pysqlite3 0.1.4 to easy-install.pth file

Installed /private/tmp/pysqlite3/v/lib/python3.6/site-packages/pysqlite3-0.1.4-py3.6-macosx-10.13-x86_64.egg
Processing dependencies for pysqlite3==0.1.4
Finished processing dependencies for pysqlite3==0.1.4
(v) pysqlite3 $ python
Python 3.6.5 (default, Mar 30 2018, 06:41:53) 
[GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.39.2)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import pysqlite3
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/private/tmp/pysqlite3/v/lib/python3.6/site-packages/pysqlite3-0.1.4-py3.6-macosx-10.13-x86_64.egg/pysqlite3/__init__.py", line 23, in <module>
    from pysqlite3.dbapi2 import *
  File "/private/tmp/pysqlite3/v/lib/python3.6/site-packages/pysqlite3-0.1.4-py3.6-macosx-10.13-x86_64.egg/pysqlite3/dbapi2.py", line 28, in <module>
    from pysqlite3._sqlite3 import *
ImportError: dlopen(/private/tmp/pysqlite3/v/lib/python3.6/site-packages/pysqlite3-0.1.4-py3.6-macosx-10.13-x86_64.egg/pysqlite3/_sqlite3.cpython-36m-darwin.so, 2): Symbol not found: _sqlite3_enable_load_extension
  Referenced from: /private/tmp/pysqlite3/v/lib/python3.6/site-packages/pysqlite3-0.1.4-py3.6-macosx-10.13-x86_64.egg/pysqlite3/_sqlite3.cpython-36m-darwin.so
  Expected in: flat namespace
 in /private/tmp/pysqlite3/v/lib/python3.6/site-packages/pysqlite3-0.1.4-py3.6-macosx-10.13-x86_64.egg/pysqlite3/_sqlite3.cpython-36m-darwin.so
>>>
simonw commented 6 years ago

I got the same error when I tried to install from PyPI using pip install pysqlite3

coleifer commented 6 years ago

How old is the system sqlite3? sqlite3 --version should show you.

coleifer commented 6 years ago

In connection.c we are checking the SQLite version for load extension availability, so as long as the version is reasonably recent it should be fine:

#if SQLITE_VERSION_NUMBER >= 3003008
#ifndef SQLITE_OMIT_LOAD_EXTENSION
#define HAVE_LOAD_EXTENSION
#endif
#endif

You might also make sure that your sqlite3 was compiled with ENABLE_LOAD_EXTENSION compile-time option set.

simonw commented 6 years ago 
~ $ sqlite3 --version
3.19.3 2017-06-27 16:48:08 2b0954060fe10d6de6d479287dd88890f1bef6cc1beca11bc6cdb79f72e2377b

~ $ sqlite3
SQLite version 3.19.3 2017-06-27 16:48:08
Enter ".help" for usage hints.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database.
sqlite> PRAGMA compile_options;
BUG_COMPATIBLE_20160819
COMPILER=clang-9.0.0
DEFAULT_CACHE_SIZE=2000
DEFAULT_CKPTFULLFSYNC
DEFAULT_JOURNAL_SIZE_LIMIT=32768
DEFAULT_PAGE_SIZE=4096
DEFAULT_SYNCHRONOUS=2
DEFAULT_WAL_SYNCHRONOUS=1
ENABLE_API_ARMOR
ENABLE_COLUMN_METADATA
ENABLE_DBSTAT_VTAB
ENABLE_FTS3
ENABLE_FTS3_PARENTHESIS
ENABLE_FTS3_TOKENIZER
ENABLE_FTS4
ENABLE_FTS5
ENABLE_JSON1
ENABLE_LOCKING_STYLE=1
ENABLE_PREUPDATE_HOOK
ENABLE_RTREE
ENABLE_SESSION
ENABLE_SNAPSHOT
ENABLE_SQLLOG
ENABLE_UNKNOWN_SQL_FUNCTION
ENABLE_UPDATE_DELETE_LIMIT
HAVE_ISNAN
MAX_LENGTH=2147483645
MAX_MMAP_SIZE=1073741824
MAX_VARIABLE_NUMBER=500000
OMIT_AUTORESET
OMIT_LOAD_EXTENSION
STMTJRNL_SPILL=131072
THREADSAFE=2
USE_URI
sqlite>
coleifer commented 6 years ago

You have: OMIT_LOAD_EXTENSION

So...no load extension.

coleifer commented 6 years ago

Can you try exporting CFLAGS="-DSQLITE_OMIT_LOAD_EXTENSION" before running setup.py build_ext?