With Ubuntu 20.04 LTS, I install the dependency, but fail in compiling to arrow

[cyberspicecai]

Hi, Guys. My OS is NAME="Ubuntu 20.04.4 LTS", and I follow the guide to install mapd-deps-ubuntu.sh and install arrow(https://arrow.apache.org/install/), but I fail in compiling the arrow related source files.

[ 44%] Built target QueryEngineFunctionsTargets /home/check/omniscidb_2022_3_10/DataMgr/ForeignStorage/ArrowForeignStorage.cpp: In member function ‘virtual void ArrowCsvForeignStorage::registerTable(Catalog_Namespace::Catalog, std::pair<int, int>, const string&, const TableDescriptor&, const std::__cxx11::list&, Data_Namespace::AbstractBufferMgr)’: /home/check/omniscidb_2022_3_10/DataMgr/ForeignStorage/ArrowForeignStorage.cpp:1006:86: error: no matching function for call to ‘arrow::csv::TableReader::Make(arrow::MemoryPool&, std::shared_ptr&, arrow::csv::ReadOptions&, arrow::csv::ParseOptions&, arrow::csv::ConvertOptions&)’ io_context, inp, arrow_read_options, arrow_parse_options, arrow_convert_options); ^ In file included from /home/check/omniscidb_2022_3_10/DataMgr/ForeignStorage/ArrowForeignStorage.cpp:20: /usr/local/mapd-deps/lib/../include/arrow/csv/reader.h:50:47: note: candidate: ‘static arrow::Result<std::shared_ptr > arrow::csv::TableReader::Make(arrow::io::IOContext, std::shared_ptr, const arrow::csv::ReadOptions&, const arrow::csv::ParseOptions&, const arrow::csv::ConvertOptions&)’ static Result<std::shared_ptr> Make(io::IOContext io_context, ^~~~ /usr/local/mapd-deps/lib/../include/arrow/csv/reader.h:50:47: note: no known conversion for argument 1 from ‘arrow::MemoryPool’ to ‘arrow::io::IOContext’ /usr/local/mapd-deps/lib/../include/arrow/csv/reader.h:57:47: note: candidate: ‘static arrow::Result<std::shared_ptr > arrow::csv::TableReader::Make(arrow::MemoryPool*, arrow::io::IOContext, std::shared_ptr, const arrow::csv::ReadOptions&, const arrow::csv::ParseOptions&, const arrow::csv::ConvertOptions&)’ static Result<std::shared_ptr> Make( ^~~~ /usr/local/mapd-deps/lib/../include/arrow/csv/reader.h:57:47: note: candidate expects 6 arguments, 5 provided make[2]: [DataMgr/CMakeFiles/DataMgr.dir/build.make:232: DataMgr/CMakeFiles/DataMgr.dir/ForeignStorage/ArrowForeignStorage.cpp.o] Error 1 make[1]: [CMakeFiles/Makefile2:2139: DataMgr/CMakeFiles/DataMgr.dir/all] Error 2 make[1]: *** Waiting for unfinished jobs.... [ 44%] Built target mapd_java_components

Would pls help resolve it? Thanks a lot

---

OS

VERSION="20.04.4 LTS (Focal Fossa)" ID=ubuntu ID_LIKE=debian PRETTY_NAME="Ubuntu 20.04.4 LTS" VERSION_ID="20.04" HOME_URL="https://www.ubuntu.com/" SUPPORT_URL="https://help.ubuntu.com/" BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" VERSION_CODENAME=focal UBUNTU_CODENAME=focal

[andrewseidl]

There is no need to install Arrow separately, as the mapd-deps-ubuntu.sh (and mapd-deps-prebuilt.sh) scripts will build and install it automatically. Unless you're planning on changing versions of any of the dependencies, it will probably be quicker to use the mapd-deps-prebuilt.sh script instead as it includes prebuilt versions of all dependencies - no need to wait for your machine to build them.

But if you do need to use a separate version of Arrow, I believe the issue is that it must be built with the -DARROW_FILESYSTEM=ON -DARROW_S3=ON flags. See https://github.com/heavyai/heavydb/blob/01a87d1c6574f64a3cd965a01c73be270ad529f4/scripts/common-functions.sh#L85-L107

[cyberspicecai]

Hi, Andrew, Thanks for your quick reply. I do not modify the cmake file and use the simple cmdline of cmake -DCMAKE_BUILD_TYPE=release ..., the errors appear. From CMake output i find the arrow related item:-- Found Arrow: /usr/local/mapd-deps/lib/libarrow.so And i find that Arrow is installed in /usr/local/mapd-depsif : find ./ -name "arrow*" ./include/parquet/arrow ./include/arrow ./share/doc/arrow ./lib/pkgconfig/arrow-csv.pc ./lib/pkgconfig/arrow-json.pc ./lib/pkgconfig/arrow-cuda.pc ./lib/pkgconfig/arrow-compute.pc ./lib/pkgconfig/arrow-filesystem.pc ./lib/pkgconfig/arrow.pc ./lib/cmake/arrow ./lib/cmake/arrow/arrow-config.cmake

 I am not sure about the  script is ok, mapd-deps-ubuntu.sh or  mapd-deps-prebuilt.sh is good for Ubuntu 20.04.  I try mapd-deps-prebuilt.sh, ie, run this script on my server, and find the error pops up again? 
 would you pls check  Ubuntu 20.04  can be the compiling platform now?

  Thanks a lot.

[cdessanti]

I use the mapd deps prebuilt on ubuntu 20.04 without any issue

[cyberspicecai]

Hi, cdessanti I start one docker of ubuntu 20.04, run the script and restart it. source /usr/local/mapd-deps/mapd-deps.sh and cmake -DCMAKE_BUILD_TYPE=release -DENABLE_CUDA=off .. and find the same errors. I attach the cmake output and compile history. heavydb_compile_error_on_ubuntu20p04.txt Pls have a check. Thanks a lot. Cyberspice

[cyberspicecai]

Hi, Guys,

I think one certain version has a little bug. The branch of master 3e9065750b [origin/master: behind 438] will pop up compiling errors, which i checkouted at March 10, 2022. And the branch of master 01a87d1c65 [origin/master] is ok, which i checkout today.

Thanks for your help. cyberspice

[cyberspicecai]

I think some branch has bug. I finish compiling with the latest version.