Load sqlite_scanner with duckdb release 0.9.1

[antonylebechec]

What happens?

Load sqlite_scanner failed using "http://extensions.duckdb.org/v0.9.1/linux_amd64/sqlite_scanner.duckdb_extension.gz"

duckdb.duckdb.InvalidInputException: Invalid Input Error: Initialization function "sqlite_scanner_init" from file "/root/.duckdb/extensions/v0.9.1/linux_amd64_gcc4/sqlite_scanner.duckdb_extension" threw an exception: "INTERNAL Error: Missing DB manager"

To Reproduce

On a Docker container using AlmaLinux:8

$ wget http://extensions.duckdb.org/v0.9.1/linux_amd64/sqlite_scanner.duckdb_extension.gz
$ gzip -d sqlite_scanner.duckdb_extension.gz
$ python
Python 3.10.12 | packaged by conda-forge | (main, Jun 23 2023, 22:40:32) [GCC 12.3.0] on linux
>>> import duckdb
>>> conn=duckdb.connect()
>>> conn.query("INSTALL 'sqlite_scanner.duckdb_extension'")
>>> conn.query("LOAD sqlite_scanner")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
duckdb.duckdb.InvalidInputException: Invalid Input Error: Initialization function "sqlite_scanner_init" from file "/root/.duckdb/extensions/v0.9.1/linux_amd64_gcc4/sqlite_scanner.duckdb_extension" threw an exception: "INTERNAL Error: Missing DB manager"

OS:

AlmaLinux:8 Docker (MacOS x64)

SQLite Version:

3.26.0

DuckDB Version:

0.9.1

DuckDB Client:

Python

Full Name:

Antony Le Bechec

Affiliation:

Strasbourg Hospital

Have you tried this on the latest master branch?

• [X] I agree

Have you tried the steps to reproduce? Do they include all relevant data and configuration? Does the issue you report still appear there?

• [X] I agree

[Mytherin]

Thanks for the report!

It looks like you are using the wrong binaries. Your target platform is linux_amd64_gcc4 (as indicated in the path where the extension is installed). You have installed binaries for linux_amd64. Using binaries for other platforms than intended can lead to incorrect/strange behavior.

You are likely looking for http://extensions.duckdb.org/v0.9.1/linux_amd64_gcc4/sqlite_scanner.duckdb_extension.gz.

Is there any reason you cannot use the normal INSTALL 'sqlite_scanner' command which figures out the correct file to download automatically?

[antonylebechec]

Thanks @Mytherin for your reply!

This is probably a platform identification issue. Due to Docker and multi-plaform, this is not easy to determine which one is the good one (amd64, amd64_gcc4, x86_64...). I use this function to retrieve platform, but it seems to not work properly:

release_version_number = "v0.9.1"
extension_name = "sqlite_scanner"
system = platform.system()
machine = platform.machine()
platform_name = f"{system}_{machine_check}".lower()
url = f"http://extensions.duckdb.org/v{release_version_number}/{platform_name}/{extension_name}.duckdb_extension.gz"

Especially, for platform x86_64, no URL is available. Is that normal?

And yes, there is a reason why I do not use the normal INSTALL 'sqlite_scanner' command (same for json extension). Our proxy is not open for the URL extensions.duckdb.org. I have to include extensions within my code to prevent proxy limitations.

By the way, this is my code (devel branch for instance) : https://github.com/bioinfo-chru-strasbourg/howard/tree/devel It's a project mainly based on duckDB. So thank you for your work!

[Mytherin]

That function is close, but when used in Python extensions require the _gcc4 suffix in the platform, as the Python binaries are build with a different process with an old version of GCC. Your best bet is to ask DuckDB itself:

D PRAGMA platform;
┌───────────┐
│ platform  │
│  varchar  │
├───────────┤
│ osx_arm64 │
└───────────┘

[antonylebechec]

Thanks for your reply! I'll test this asap!

[Mytherin]

Especially, for platform x86_64, no URL is available. Is that normal?

DuckDB uses the names amd64, arm64 (64-bit) and i686 (32-bit)