Missing SQLite3 dll on Windows

[cocoa-xu]

Not in a hurry to do this, but I'll leave some notes here.

~We can either require Windows users to do~

~or in the Makefile.win we can download official pre-compiled SQLite3 DLLs from https://www.sqlite.org/download.html and place them in priv/lib~

We'll download drivers from https://github.com/apache/arrow-adbc/releases/tag/apache-arrow-adbc-0.4.0 and place them in priv/lib (macOS/Linux) or priv/bin (Windows)

We need to put them in priv/bin on Windows because that's the default location ($PREFIX/bin) that the compiled DLL files will be installed to.

[Qqwy]

I went to look at what exqlite is doing as it is in a similar situation (being a NIF wrapper for SQLite3). Exqlite seems to include SQLite3's source files directly. see here.

One disadvantage of that approach is that users are given no choice of whether to use an included SQLite, the system's SQLite or maybe a particular separately-downloaded version. Not sure whether that is very important (one of SQLite's main tenets is extreme backwards-compatibility after all), but at least worth a thought.

[josevalim]

I guess a lot of this will depend on how we will adopt plugins, since sqlite3 support will come from a plugin. Perhaps the plugin package itself already includes sqlite3.dll? https://github.com/apache/arrow-adbc/releases/tag/apache-arrow-adbc-0.4.0

[Qqwy]

Perhaps the plugin package itself already includes sqlite3.dll?

Good thought! Yup, the SQLite3 driver also seems to include SQLite3 from source.

[josevalim]

Yeah, so maybe what we should do is, instead of compiling the sqlite3 one ourselves, we can download the Python one, unpack it, and extract the relevant file.

[cocoa-xu]

No problem! I'll do a PR that can download these drivers (sqlite, postgresql, flightsql, snowflake) from https://github.com/apache/arrow-adbc/releases/tag/apache-arrow-adbc-0.4.0

I remember that last time I talked with José that we could do something like

  def application do
    [
      env: [
        driver: [:sqlite, :postgresql]
      ]
    ]
  end

in the mix.exs file, then at compile-time, we can download these additional drivers.

We can also support downloading (custom) drivers at runtime. The API can be something like

Adbc.Driver.download_driver(driver_url, checksum)

[josevalim]

For now let’s only do sqlite3 as a starting point? You can have a compilers: […] entry that downloads and unpack it?

[cocoa-xu]

Closing this issue for now as it's been addressed in #8 :D