Fix vectordb build, including macOS portability fixes

[laurynas-biveinis]

Fix some vectordb build issues, both general and macOS-specific:

• cmake/faiss.cmake references a non-existing BUILD_DIR CMake variable. Since it gets expanded to nothing, the FAISS build dir becomes /faiss, and then, if the root dir is unwritable: -- Adding faiss and its dependencies CMake Error at _build-debug/FAISS_EXTERNAL-prefix/tmp/FAISS_EXTERNAL-mkdirs.cmake:6 (file): file failed to create directory:

  /faiss

  because: Read-only file system Call Stack (most recent call first): /opt/homebrew/Cellar/cmake/3.29.2/share/cmake/Modules/ExternalProject.cmake:1816 (include) /opt/homebrew/Cellar/cmake/3.29.2/share/cmake/Modules/ExternalProject.cmake:4392 (_ep_set_directories) cmake/faiss.cmake:14 (ExternalProject_Add) CMakeLists.txt:2214 (INCLUDE)

• If Ninja is used as the CMake build generator, the build fails with: ninja: error: 'faiss/faiss/libfaiss.a', needed by 'runtime_output_directory/mysqld', missing and no known rule to make it This is fixed by setting BUILD_BYPRODUCTS option on ExternalProject_Add to the static library name.
• Replace INCLUDE_DIRECTORIES for sql_main with more idiomatic and non-polluting globally TARGET_INCLUDE_DIRECTORIES
• Remove "-lgfortran" from TARGET_LINK_LIBRARIES for sql_main as that is not Fortran code. If it is a dependency from one of ${BLAS_LIBRARIES} ${OPENMP_LIBRARY} ${FAISS_LIBRARY}, then it's their responsibility to include it.
• Fix trivial macOS specific build errors on storage/rocksdb/rdb_vector_db.cc: storage/rocksdb/rdb_vector_db.cc:90:26: error: cannot initialize a parameter of type 'uint64 ' (aka 'unsigned long long ') with an rvalue of type 'size_t ' (aka 'unsigned long ') if (reader.read_uint64(&actual_list_id)) { ^~~~~~~ storage/rocksdb/./rdb_buff.h:325:34: note: passing argument to parameter 'res' here bool read_uint64(uint64 *const res) { ^ storage/rocksdb/rdb_vector_db.cc:144:63: error: format specifies type 'unsigned long' but the argument has type 'value_type' (aka 'long long') [-Werror,-Wformat] "Failed to find matching pk for %lu", vector_id);

  
                                                      %lld
  include/mysql/components/services/log_builtins.h:838:66: note: expanded from macro 'LogPluginErrMsg'
    .message_quoted("Plugin " LOG_COMPONENT_TAG " reported", ##__VA_ARGS__)
                                                               ^~~~~~~~~~~
  storage/rocksdb/rdb_vector_db.cc:145:63: error: format specifies type 'unsigned long' but the argument has type 'value_type' (aka 'long long') [-Werror,-Wformat]
                      "Failed to find matching pk for %lu", vector_id);
                                                      ~~~   ^~~~~~~~~
                                                      %lld
  include/mysql/components/services/log_builtins.h:838:66: note: expanded from macro 'LogPluginErrMsg'
    .message_quoted("Plugin " LOG_COMPONENT_TAG " reported", ##__VA_ARGS__)
                                                               ^~~~~~~~~~~

[laurynas-biveinis]

This PR allows me to build on macOS but not Linux, where I haven't figured out which flavor of openmp to install and how to point to it yet, but this unblocks my current work alread. The "-lgfortran"-removing bit might be build-breaking, let me know, and I will revert it.

[facebook-github-bot]

@luqun has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

[luqun]

This PR allows me to build on macOS but not Linux, where I haven't figured out which flavor of openmp to install and how to point to it yet, but this unblocks my current work alread. The "-lgfortran"-removing bit might be build-breaking, let me know, and I will revert it.

yep. The "-lgfortran"-removing cause build-breaking, could you revert it?

[laurynas-biveinis]

This PR allows me to build on macOS but not Linux, where I haven't figured out which flavor of openmp to install and how to point to it yet, but this unblocks my current work alread. The "-lgfortran"-removing bit might be build-breaking, let me know, and I will revert it.

yep. The "-lgfortran"-removing cause build-breaking, could you revert it?

I will. Could you show the full linker error? Maybe I can replace it with something better at the same time

[luqun]

This PR allows me to build on macOS but not Linux, where I haven't figured out which flavor of openmp to install and how to point to it yet, but this unblocks my current work alread. The "-lgfortran"-removing bit might be build-breaking, let me know, and I will revert it.

yep. The "-lgfortran"-removing cause build-breaking, could you revert it?

I will. Could you show the full linker error? Maybe I can replace it with something better at the same time

ld.lld: error: undefined symbol: _gfortran_concat_string
>>> referenced by dgesvd.f
>>>               dgesvd.o:(dgesvd_) in archive /OpenBLAS/0.3.20/lib/libopenblas_pic.a
>>> referenced by dgesvd.f
>>>               dgesvd.o:(dgesvd_) in archive /OpenBLAS/0.3.20/lib/libopenblas_pic.a
>>> referenced by dormbr.f
>>>               dormbr.o:(dormbr_) in archive /OpenBLAS/0.3.20/lib/libopenblas_pic.a

[facebook-github-bot]

@laurynas-biveinis has updated the pull request. You must reimport the pull request before landing.

[laurynas-biveinis]

@luqun, replaced -lgfortran with a CMake variable initialized from gfortran --print-file-name libgfortran.a, add a missing standard C++ include as an LLVM 14+ fix, and rebased

[facebook-github-bot]

@luqun has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

[luqun]

Another linker error:

ld.lld: error: undefined symbol: quadmath_snprintf
>>> referenced by write_float.def:922 (../.././libgfortran/io/write_float.def:922)
>>>               write.o:(determine_en_precision) in archive /libgcc/11.x/lib/libgfortran.a
>>> referenced by write_float.def:1125 (../.././libgfortran/io/write_float.def:1125)
>>>               write.o:(get_float_string) in archive /libgcc/11.x/lib/libgfortran.a
>>> referenced by write_float.def:1125 (../.././libgfortran/io/write_float.def:1125)
>>>               write.o:(get_float_string) in archive /ibgcc/11.x/lib/libgfortran.a
>>> referenced 1 more times

[laurynas-biveinis]

@luqun I can push another fix that looks up libquadmath.a the same way, but this error is suspicious if your original plain -lgfortran was not failing. It suggests that the default -lgfortran and gfortran executable are using different toolchains, and I cannot confirm that remotely unless I push some CMake message calls and ask you to run them. So we have three options: 1) push gfortran looking up libquadmath.a; 2) push diagnostics to confirm whether -lgfortran and gfortran point to identical or different toolchains; 3) revert to your original -lgfortran, and me workarounding the issue. What do you think?

[luqun]

Maybe Let's try 2) first, if not, then 1).. if still doesn't work, use 3)

[facebook-github-bot]

@laurynas-biveinis has updated the pull request. You must reimport the pull request before landing.

[laurynas-biveinis]

My thinking was incorrect. I am using standard system toolchain, that has no notion of Fortran libraries, with the GCC installed as a third-party toolchain. I have fixed it here with a build workaround, and the existing -lgtfortran is correct, remove the patch, ready for review

[facebook-github-bot]

@luqun has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

[facebook-github-bot]

This pull request has been merged in facebook/mysql-5.6@f5ddb37ed297c1055a0f8398bd018d1f567a2772.