search

VowpalWabbit / vowpal_wabbit

Vowpal Wabbit is a machine learning system which pushes the frontier of machine learning with techniques such as online, hashing, allreduce, reductions, learning2search, active, and interactive learning.
https://vowpalwabbit.org
Other
8.46k stars 1.93k forks source link

Building vowpalWabbit on macOS Catalina #2145

Closed Mystic98 closed 4 years ago

Mystic98 commented 4 years ago

I was trying to install vowpalWabbit on macOS Catalina via pip install vowpalwabbit, but it had always crashed with the same error:

pip install vowpalwabbit Collecting vowpalwabbit Using cached https://files.pythonhosted.org/packages/f4/09/50efc9630809fdd19056d2fff8e7b8e374063002b5a70b016854fe5e8492/vowpalwabbit-8.7.0.post1.tar.gz Installing collected packages: vowpalwabbit Running setup.py install for vowpalwabbit ... error Complete output from command /Users/evgenich/vowpal1_env/bin/python3 -u -c "import setuptools, tokenize;file='/private/var/folders/yr/rlzd11t108bfbdd5cf1m8qw40000gn/T/pip-install-xpy6i2lk/vowpalwabbit/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" install --record /private/var/folders/yr/rlzd11t108bfbdd5cf1m8qw40000gn/T/pip-record-arh2kz3z/install-record.txt --single-version-externally-managed --compile --install-headers /Users/evgenich/vowpal1_env/include/site/python3.7/vowpalwabbit: running install running build running build_py creating build creating build/lib.macosx-10.9-x86_64-3.7 creating build/lib.macosx-10.9-x86_64-3.7/vowpalwabbit copying python/vowpalwabbit/pyvw.py -> build/lib.macosx-10.9-x86_64-3.7/vowpalwabbit copying python/vowpalwabbit/init.py -> build/lib.macosx-10.9-x86_64-3.7/vowpalwabbit copying python/vowpalwabbit/sklearn_vw.py -> build/lib.macosx-10.9-x86_64-3.7/vowpalwabbit running egg_info writing python/vowpalwabbit.egg-info/PKG-INFO writing dependency_links to python/vowpalwabbit.egg-info/dependency_links.txt writing top-level names to python/vowpalwabbit.egg-info/top_level.txt reading manifest file 'python/vowpalwabbit.egg-info/SOURCES.txt' reading manifest template 'MANIFEST.in' warning: no files found matching 'python/.dll' under directory 'python/bin' warning: no files found matching 'python/packages.config' under directory 'python/windows' warning: no files found matching 'python/.vcxproj' under directory 'python/windows' warning: no previously-included files matching '.o' found anywhere in distribution warning: no previously-included files matching '.exe' found anywhere in distribution warning: no previously-included files matching '.pyc' found anywhere in distribution warning: no previously-included files matching '*' found under directory 'vowpalwabbit/.nuget' writing manifest file 'python/vowpalwabbit.egg-info/SOURCES.txt' copying python/vowpalwabbit/conda_install.sh -> build/lib.macosx-10.9-x86_64-3.7/vowpalwabbit running build_ext creating build/temp.macosx-10.9-x86_64-3.7 cmake -DCMAKE_BUILD_TYPE=Release -DPY_VERSION=3.7 -DBUILD_PYTHON=On -DBUILD_TESTS=Off -DWARNINGS=Off -DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/private/var/folders/yr/rlzd11t108bfbdd5cf1m8qw40000gn/T/pip-install-xpy6i2lk/vowpalwabbit/build/lib.macosx-10.9-x86_64-3.7 /private/var/folders/yr/rlzd11t108bfbdd5cf1m8qw40000gn/T/pip-install-xpy6i2lk/vowpalwabbit CMake Warning (dev) at CMakeLists.txt:4 (set): implicitly converting 'TYPE' to 'STRING' type. This warning is for project developers. Use -Wno-dev to suppress it.

-- The C compiler identification is AppleClang 11.0.0.11000033
-- The CXX compiler identification is AppleClang 11.0.0.11000033
-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc
-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++
-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- VowpalWabbit Version: 8.7.0.post1
-- Number of processors: 4
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
CMake Warning at /usr/local/lib/cmake/boost_program_options-1.71.0/libboost_program_options-variant-shared.cmake:59 (message):
  Target Boost::program_options already has an imported location
  '/usr/local/lib/libboost_program_options-mt.dylib', which will be
  overwritten with '/usr/local/lib/libboost_program_options.dylib'
Call Stack (most recent call first):
  /usr/local/lib/cmake/boost_program_options-1.71.0/boost_program_options-config.cmake:43 (include)
  /usr/local/lib/cmake/Boost-1.71.0/BoostConfig.cmake:117 (find_package)
  /usr/local/lib/cmake/Boost-1.71.0/BoostConfig.cmake:182 (boost_find_component)
  /usr/local/Cellar/cmake/3.15.5/share/cmake/Modules/FindBoost.cmake:443 (find_package)
  CMakeLists.txt:125 (find_package)

CMake Warning at /usr/local/lib/cmake/boost_system-1.71.0/libboost_system-variant-shared.cmake:59 (message):
  Target Boost::system already has an imported location
  '/usr/local/lib/libboost_system-mt.dylib', which will be overwritten with
  '/usr/local/lib/libboost_system.dylib'
Call Stack (most recent call first):
  /usr/local/lib/cmake/boost_system-1.71.0/boost_system-config.cmake:43 (include)
  /usr/local/lib/cmake/Boost-1.71.0/BoostConfig.cmake:117 (find_package)
  /usr/local/lib/cmake/Boost-1.71.0/BoostConfig.cmake:182 (boost_find_component)
  /usr/local/Cellar/cmake/3.15.5/share/cmake/Modules/FindBoost.cmake:443 (find_package)
  CMakeLists.txt:125 (find_package)

CMake Warning at /usr/local/lib/cmake/boost_unit_test_framework-1.71.0/libboost_unit_test_framework-variant-shared.cmake:59 (message):
  Target Boost::unit_test_framework already has an imported location
  '/usr/local/lib/libboost_unit_test_framework-mt.dylib', which will be
  overwritten with '/usr/local/lib/libboost_unit_test_framework.dylib'
Call Stack (most recent call first):
  /usr/local/lib/cmake/boost_unit_test_framework-1.71.0/boost_unit_test_framework-config.cmake:43 (include)
  /usr/local/lib/cmake/Boost-1.71.0/BoostConfig.cmake:117 (find_package)
  /usr/local/lib/cmake/Boost-1.71.0/BoostConfig.cmake:182 (boost_find_component)
  /usr/local/Cellar/cmake/3.15.5/share/cmake/Modules/FindBoost.cmake:443 (find_package)
  CMakeLists.txt:125 (find_package)

-- Found Boost: /usr/local/lib/cmake/Boost-1.71.0/BoostConfig.cmake (found version "1.71.0") found components:  program_options system thread unit_test_framework
-- Found ZLIB: /usr/lib/libz.dylib (found version "1.2.11")
-- Python 3.7 will be used for VowpalWabbit Python bindings
-- Found PythonInterp: /Library/Frameworks/Python.framework/Versions/3.7/bin/python3.7 (found suitable version "3.7", minimum required is "3.7")
-- Found PythonLibs: /Library/Frameworks/Python.framework/Versions/3.7/lib/libpython3.7m.dylib (found suitable version "3.7.0", minimum required is "3.7")
CMake Warning at /usr/local/lib/cmake/boost_python-1.71.0/libboost_python-variant-shared-py3.7.cmake:69 (message):
  Target Boost::python already has an imported location
  '/usr/local/lib/libboost_python37-mt.dylib', which will be overwritten with
  '/usr/local/lib/libboost_python37.dylib'
Call Stack (most recent call first):
  /usr/local/lib/cmake/boost_python-1.71.0/boost_python-config.cmake:43 (include)
  /usr/local/lib/cmake/Boost-1.71.0/BoostConfig.cmake:117 (find_package)
  /usr/local/lib/cmake/Boost-1.71.0/BoostConfig.cmake:182 (boost_find_component)
  /usr/local/Cellar/cmake/3.15.5/share/cmake/Modules/FindBoost.cmake:443 (find_package)
  python/CMakeLists.txt:23 (find_package)

-- Found Boost: /usr/local/lib/cmake/Boost-1.71.0/BoostConfig.cmake (found version "1.71.0") found components:  system python37
-- Configuring done
-- Generating done
-- Build files have been written to: /private/var/folders/yr/rlzd11t108bfbdd5cf1m8qw40000gn/T/pip-install-xpy6i2lk/vowpalwabbit/build/temp.macosx-10.9-x86_64-3.7
cmake --build . --config Release -- -j8 pylibvw
Scanning dependencies of target allreduce
[  2%] Building CXX object vowpalwabbit/CMakeFiles/allreduce.dir/vw_exception.cc.o
[  2%] Building CXX object vowpalwabbit/CMakeFiles/allreduce.dir/allreduce_threads.cc.o
[  3%] Building CXX object vowpalwabbit/CMakeFiles/allreduce.dir/allreduce_sockets.cc.o
[  5%] Linking CXX static library liballreduce.a
[  5%] Built target allreduce
Scanning dependencies of target vw
[  6%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/options_boost_po.cc.o
[  7%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/options_serializer_boost_po.cc.o
[  8%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/global_data.cc.o
[  8%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/io_buf.cc.o
[ 10%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/parse_regressor.cc.o
[ 11%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/parse_primitives.cc.o
[ 12%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/unique_sort.cc.o
[ 13%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/cache.cc.o
[ 15%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/rand48.cc.o
[ 15%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/simple_label.cc.o
[ 16%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/multiclass.cc.o
[ 17%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/oaa.cc.o
[ 18%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/multilabel_oaa.cc.o
[ 20%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/boosting.cc.o
[ 21%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/ect.cc.o
[ 21%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/marginal.cc.o
[ 22%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/autolink.cc.o
[ 24%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/binary.cc.o
[ 25%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/lrq.cc.o
[ 26%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/cost_sensitive.cc.o
[ 27%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/multilabel.cc.o
[ 29%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/label_dictionary.cc.o
[ 29%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/csoaa.cc.o
[ 30%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/cb.cc.o
[ 31%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/cb_adf.cc.o
[ 32%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/cb_algs.cc.o
[ 34%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/search.cc.o
[ 35%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/search_meta.cc.o
[ 35%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/search_sequencetask.cc.o
[ 36%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/search_dep_parser.cc.o
[ 37%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/search_hooktask.cc.o
[ 39%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/search_multiclasstask.cc.o
[ 40%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/search_entityrelationtask.cc.o
[ 41%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/search_graph.cc.o
[ 41%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/parse_example.cc.o
[ 43%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/scorer.cc.o
[ 44%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/network.cc.o
[ 45%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/parse_args.cc.o
[ 46%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/accumulate.cc.o
[ 48%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/gd.cc.o
[ 48%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/learner.cc.o
[ 49%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/mwt.cc.o
[ 50%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/lda_core.cc.o
[ 51%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/gd_mf.cc.o
[ 53%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/mf.cc.o
[ 54%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/bfgs.cc.o
[ 55%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/noop.cc.o
[ 55%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/print.cc.o
[ 56%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/example.cc.o
[ 58%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/parser.cc.o
[ 59%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/loss_functions.cc.o
/private/var/folders/yr/rlzd11t108bfbdd5cf1m8qw40000gn/T/pip-install-xpy6i2lk/vowpalwabbit/vowpalwabbit/parser.cc:389:26: warning: 'daemon' is deprecated: first deprecated in macOS 10.5 - Use posix_spawn APIs instead. [-Wdeprecated-declarations]
      if (!all.active && daemon(1, 1))
                         ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/stdlib.h:292:6: note: 'daemon' has been explicitly marked deprecated here
int      daemon(int, int) __DARWIN_1050(daemon) __OSX_AVAILABLE_BUT_DEPRECATED_MSG(__MAC_10_0, __MAC_10_5, __IPHONE_2_0, __IPHONE_2_0, "Use posix_spawn APIs instead.") __WATCHOS_PROHIBITED __TVOS_PROHIBITED;
         ^
[ 60%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/sender.cc.o
[ 62%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/nn.cc.o
[ 62%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/confidence.cc.o
[ 63%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/bs.cc.o
[ 64%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/cbify.cc.o
[ 65%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/warm_cb.cc.o
[ 67%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/explore_eval.cc.o
[ 68%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/topk.cc.o
1 warning generated.
[ 68%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/stagewise_poly.cc.o
[ 69%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/log_multi.cc.o
[ 70%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/recall_tree.cc.o
[ 72%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/memory_tree.cc.o
[ 73%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/active.cc.o
[ 74%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/active_cover.cc.o
[ 74%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/cs_active.cc.o
[ 75%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/kernel_svm.cc.o
[ 77%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/best_constant.cc.o
[ 78%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/ftrl.cc.o
[ 79%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/svrg.cc.o
[ 81%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/lrqfa.cc.o
[ 82%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/interact.cc.o
[ 82%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/comp_io.cc.o
[ 83%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/interactions.cc.o
[ 84%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/vw_validate.cc.o
[ 86%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/audit_regressor.cc.o
[ 87%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/gen_cs_example.cc.o
[ 88%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/cb_explore.cc.o
[ 88%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/action_score.cc.o
[ 89%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/cb_explore_adf.cc.o
[ 91%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/OjaNewton.cc.o
[ 92%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/baseline.cc.o
[ 93%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/classweight.cc.o
[ 94%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/vw_exception.cc.o
[ 94%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/no_label.cc.o
[ 96%] Building CXX object vowpalwabbit/CMakeFiles/vw.dir/shared_feature_merger.cc.o
[ 97%] Linking CXX static library libvw.a
[ 97%] Built target vw
Scanning dependencies of target pylibvw
[ 98%] Building CXX object python/CMakeFiles/pylibvw.dir/pylibvw.cc.o
[100%] Linking CXX shared library ../../lib.macosx-10.9-x86_64-3.7/pylibvw.dylib
[100%] Built target pylibvw
building 'pylibvw' extension
gcc -bundle -undefined dynamic_lookup -arch x86_64 -g -o build/lib.macosx-10.9-x86_64-3.7/pylibvw.so
clang: error: no input files
error: command 'gcc' failed with exit status

Can somebody help with this problem? I've tried different python versions like 3.6.8, 3.7.0.,3.7.5.,38.0, but the problem was the same all the time.

peterychang commented 4 years ago

This is a known issue on the 8.7.0.post1 release of vowpalwabbit and will be resolved on the next pypi release.

In the meantime, you can install from source https://github.com/VowpalWabbit/vowpal_wabbit/wiki/Python#osx