search

1yefuwang1 / vectorlite

Fast, SQL powered, in-process vector search for any language with an SQLite driver
https://1yefuwang1.github.io/vectorlite/
Apache License 2.0
269 stars 5 forks source link

Building locally and for Linux ARM #26

Open coogle opened 3 months ago

coogle commented 3 months ago

This looks like a very interesting project, thank you for working on this.

I checked out the code and tried to figure out how to compile from source locally on macOS without success:

bash ./build.sh
Preset CMake variables:

  CMAKE_BUILD_TYPE="Debug"
  CMAKE_EXPORT_COMPILE_COMMANDS="1"
  CMAKE_TOOLCHAIN_FILE:FILEPATH="vcpkg/scripts/buildsystems/vcpkg.cmake"

CMake Error at /opt/homebrew/Cellar/cmake/3.27.8/share/cmake/Modules/CMakeDetermineSystem.cmake:154 (message):
  Could not find toolchain file: vcpkg/scripts/buildsystems/vcpkg.cmake
Call Stack (most recent call first):
  CMakeLists.txt:8 (project)

CMake Error: CMake was unable to find a build program corresponding to "Ninja".  CMAKE_MAKE_PROGRAM is not set.  You probably need to select a different build tool.
CMake Error: CMAKE_CXX_COMPILER not set, after EnableLanguage
-- Configuring incomplete, errors occurred!

Could you provide some hints/docs on how to build this locally?

Also, would it be possible to add Linux ARM to the releases?

1yefuwang1 commented 2 months ago

Hi @coogle, thank you for the interest. The build instructions are here https://github.com/1yefuwang1/vectorlite?tab=readme-ov-file#prerequisites . Based on your error message, the reason of build failure is that Ninja is not on PATH. You can install it using brew.

About Linux ARM, I believe it should be easy to do given that vectorlite already works on macos ARM. Some changes to the ci.yml and pyproject.toml will do. I'll try to figure it out in the next release.

BTW, what language are you using? Currently, only python and nodejs have 1st class support.

andresattler commented 1 month ago

Oh yes Linux ARM support would be awesome

1yefuwang1 commented 1 month ago

@felipesantosk and I have put some effort into precompiling on ARM64 Linux using QEMU emulation. The wheel can be built successfully, and the artifact is available here https://github.com/1yefuwang1/vectorlite/actions/runs/11049779973. The problem is it takes about 1.5 hours to build on github actions due to emulation, which is unbearably slow.

We will explore cross-compiling later but probably not in the next release. Before we figure it out, you can try manual compilation on ARM64 Linux following build instructions. Hope it helps. @andresattler @coogle

1yefuwang1 commented 1 week ago

https://github.blog/news-insights/product-news/arm64-on-github-actions-powering-faster-more-efficient-build-systems/ ARM64 github actions runner for open source projects is coming. It would make it a lot easier.