isaacbrodsky / h3-duckdb

Bindings for H3 to DuckDB
Apache License 2.0
170 stars 8 forks source link

Using with DuckDB python api #73

Closed mtravis closed 7 months ago

mtravis commented 11 months ago

Firstly, thanks for the great extension. Just wondering if it is possible to use this extension along side the python version of DuckDB.

isaacbrodsky commented 11 months ago

I can confirm it is possible to load this extension from the Python bindings. I used Python bindings built from the same version of DuckDB as the extension.

In [8]: d = duckdb.connect(config={'allow_unsigned_extensions':'true'})

In [9]: d.query("LOAD 'build/release/extension/h3ext/h3ext.duckdb_extension'")

In [10]: d.query("SELECT h3_latlng_to_cell(0,0,0);")
Out[10]:
┌────────────────────────────┐
│ h3_latlng_to_cell(0, 0, 0) │
│           uint64           │
├────────────────────────────┤
│         578536630256664575 │
└────────────────────────────┘
mtravis commented 11 months ago

Thanks. I will give this a go. Is there any talk of this extension being adopted (if that's the right term) so installation will be easier?

alastairherd commented 11 months ago

Just tried to load the extension as you mentioned. I built it on WSL, so maybe my mistake is that I have to run it through WSL and not just Windows?:

%1 is not a valid Win32 application.

d = duckdb.connect(config={'allow_unsigned_extensions':'true'})
d.query("LOAD 'C:/Users/name/Downloads/duckdb_h3/build/release/extension/h3ext/h3ext.duckdb_extension'")
d.query("SELECT h3_latlng_to_cell(0,0,0);")
---------------------------------------------------------------------------
IOException: IO Error: Extension "C:\Users\name\Downloads\duckdb_h3\build\release\extension\h3ext\h3ext.duckdb_extension" could not be loaded: %1 is not a valid Win32 application.
isaacbrodsky commented 11 months ago

Just tried to load the extension as you mentioned. I built it on WSL, so maybe my mistake is that I have to run it through WSL and not just Windows?: ...

---------------------------------------------------------------------------
IOException: IO Error: Extension "C:\Users\name\Downloads\duckdb_h3\build\release\extension\h3ext\h3ext.duckdb_extension" could not be loaded: %1 is not a valid Win32 application.

Yes, this looks like you will need to load it through WSL.

You may also try using the prebuilt Windows extension (e.g. from the latest commit, here: https://github.com/isaacbrodsky/h3-duckdb/actions/runs/6956834214), although I have not tried that myself.

isaacbrodsky commented 11 months ago

Thanks. I will give this a go. Is there any talk of this extension being adopted (if that's the right term) so installation will be easier?

I haven't looked into that, but I would be happy to discuss what's involved & add supporting code for that.

alastairherd commented 11 months ago

Just tried to load the extension as you mentioned. I built it on WSL, so maybe my mistake is that I have to run it through WSL and not just Windows?: ...

---------------------------------------------------------------------------
IOException: IO Error: Extension "C:\Users\name\Downloads\duckdb_h3\build\release\extension\h3ext\h3ext.duckdb_extension" could not be loaded: %1 is not a valid Win32 application.

Yes, this looks like you will need to load it through WSL.

You may also try using the prebuilt Windows extension (e.g. from the latest commit, here: https://github.com/isaacbrodsky/h3-duckdb/actions/runs/6956834214), although I have not tried that myself.

That worked like a charm!

Thank you so much, this is a super helpful extension! Honestly don't know how I'd complete the project I'm working on without it.

Quick FYI, it might save you having to answer this question again if you have a link to that file in the README for other confused Windows users.

mgnolde commented 11 months ago

I can't get the extension to work inside Python through WSL (Ubuntu 22.04.2). Executing d.query("LOAD 'build/release/extension/h3ext/h3ext.duckdb_extension'") yields a Floating point exception. Here's what I did:

$ sudo apt-get install -y ninja-build cmake
$ git clone https://github.com/isaacbrodsky/h3-duckdb.git
$ cd h3-duckdb
$ git submodule update --init
$ cd h3
$ cmake .
$ cd ..
$ GEN=ninja BUILD_PYTHON=1 make release
$ ./build/release/duckdb -unsigned 
    >> v0.9.2 3c695d7ba9
    >> Enter ".help" for usage hints
$ python3 -m pip install duckdb==0.9.2
$ python3
    import duckdb
    duckdb.__version__
        >> '0.9.2'
    d = duckdb.connect(config={'allow_unsigned_extensions':'true'})
    d.query("LOAD 'build/release/extension/h3ext/h3ext.duckdb_extension'")
    >> Floating point exception

Any suggestion is appreciated.

PS: I can confirm that the prebuilt Windows version (which is mentioned above) works flawlessly out-of-the-box. However, I need to run it in Linux.

mtravis commented 11 months ago

Thanks. I will give this a go. Is there any talk of this extension being adopted (if that's the right term) so installation will be easier?

I haven't looked into that, but I would be happy to discuss what's involved & add supporting code for that.

@isaacbrodsky hope you don't mind but i've asked about this on the DuckDB discord server (as that's the most active place it seems) I'll report back once I've heard something.

isaacbrodsky commented 7 months ago

I believe this is solved as you can now download the release artifacts from the repo's README. Please open a new issue if the extension is not working under Python.