search

giotto-ai / giotto-tda

A high-performance topological machine learning toolbox in Python
https://giotto-ai.github.io/gtda-docs
Other
858 stars 175 forks source link

[BUG] Dev install fails on macOS Catalina with Python 3.8 #446

Closed lewtun closed 4 years ago

lewtun commented 4 years ago

Describe the bug

I am running into C++ errors during the developer install (stack trace below) šŸ˜­

To reproduce

  1. Create a conda env with Python 3.8: 
    conda create python=3.8 --name gtda && conda activate gtda
  2. Install the external dependencies: 
    brew install gcc cmake boost
  3. Install the remaining dependencies: 
    python -m pip install -e ".[dev]"

Expected behavior

Install step passes without error.

Actual behaviour

  Running setup.py develop for giotto-tda
ERROR: Command errored out with exit status 1:
     command: /Users/lewtun/miniconda3/envs/gtda/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/Users/lewtun/git/giotto-tda/setup.py'"'"'; __file__='"'"'/Users/lewtun/git/giotto-tda/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps
         cwd: /Users/lewtun/git/giotto-tda/
    Complete output (145 lines):
    running develop
    running egg_info
    writing giotto_tda.egg-info/PKG-INFO
    writing dependency_links to giotto_tda.egg-info/dependency_links.txt
    writing requirements to giotto_tda.egg-info/requires.txt
    writing top-level names to giotto_tda.egg-info/top_level.txt
    reading manifest file 'giotto_tda.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    writing manifest file 'giotto_tda.egg-info/SOURCES.txt'
    running build_ext
    -- pybind11 v2.6.0 dev
    -- Configuring done
    -- Generating done
    -- Build files have been written to: /Users/lewtun/git/giotto-tda/build/temp.macosx-10.9-x86_64-3.8
    [  9%] Built target gtda_cech_complex
    [ 13%] Building CXX object CMakeFiles/gtda_wasserstein.dir/gtda/externals/bindings/wasserstein_bindings.cpp.o
    Scanning dependencies of target gtda_periodic_cubical_complex
    [ 18%] Building CXX object CMakeFiles/gtda_periodic_cubical_complex.dir/gtda/externals/bindings/periodic_cubical_complex_bindings.cpp.o
    clang: warning: optimization flag '-frounding-math' is not supported [-Wignored-optimization-argument]
    clang: warning: argument unused during compilation: '-shared' [-Wunused-command-line-argument]
    In file included from /Users/lewtun/git/giotto-tda/gtda/externals/bindings/wasserstein_bindings.cpp:6:
    In file included from /Users/lewtun/git/giotto-tda/gtda/externals/bindings/../hera/wasserstein/wasserstein.h:39:
    In file included from /Users/lewtun/git/giotto-tda/gtda/externals/bindings/../hera/wasserstein/auction_runner_gs.h:36:
    In file included from /Users/lewtun/git/giotto-tda/gtda/externals/bindings/../hera/wasserstein/auction_oracle.h:36:
    In file included from /Users/lewtun/git/giotto-tda/gtda/externals/bindings/../hera/wasserstein/auction_oracle_kdtree_restricted.h:44:
    In file included from /Users/lewtun/git/giotto-tda/gtda/externals/bindings/../hera/wasserstein/dnn/geometry/euclidean-fixed.h:16:
    In file included from /Users/lewtun/git/giotto-tda/gtda/externals/bindings/../hera/wasserstein/dnn/geometry/../parallel/tbb.h:138:
    /usr/local/include/boost/progress.hpp:23:1: warning: This header is deprecated. Use the facilities in <boost/timer/timer.hpp> or <boost/timer/progress_display.hpp> instead. [-W#pragma-messages]
    BOOST_HEADER_DEPRECATED( "the facilities in <boost/timer/timer.hpp> or <boost/timer/progress_display.hpp>" )
    ^
    /usr/local/include/boost/config/header_deprecated.hpp:23:37: note: expanded from macro 'BOOST_HEADER_DEPRECATED'
    # define BOOST_HEADER_DEPRECATED(a) BOOST_PRAGMA_MESSAGE("This header is deprecated. Use " a " instead.")
                                        ^
    /usr/local/include/boost/config/pragma_message.hpp:24:34: note: expanded from macro 'BOOST_PRAGMA_MESSAGE'
    # define BOOST_PRAGMA_MESSAGE(x) _Pragma(BOOST_STRINGIZE(message(x)))
                                     ^
    <scratch space>:141:2: note: expanded from here
     message("This header is deprecated. Use " "the facilities in <boost/timer/timer.hpp> or <boost/timer/progress_display.hpp>" " instead.")
     ^
    In file included from /Users/lewtun/git/giotto-tda/gtda/externals/bindings/wasserstein_bindings.cpp:6:
    In file included from /Users/lewtun/git/giotto-tda/gtda/externals/bindings/../hera/wasserstein/wasserstein.h:39:
    In file included from /Users/lewtun/git/giotto-tda/gtda/externals/bindings/../hera/wasserstein/auction_runner_gs.h:36:
    In file included from /Users/lewtun/git/giotto-tda/gtda/externals/bindings/../hera/wasserstein/auction_oracle.h:36:
    In file included from /Users/lewtun/git/giotto-tda/gtda/externals/bindings/../hera/wasserstein/auction_oracle_kdtree_restricted.h:44:
    In file included from /Users/lewtun/git/giotto-tda/gtda/externals/bindings/../hera/wasserstein/dnn/geometry/euclidean-fixed.h:16:
    In file included from /Users/lewtun/git/giotto-tda/gtda/externals/bindings/../hera/wasserstein/dnn/geometry/../parallel/tbb.h:138:
    In file included from /usr/local/include/boost/progress.hpp:25:
    /usr/local/include/boost/timer.hpp:21:1: warning: This header is deprecated. Use the facilities in <boost/timer/timer.hpp> instead. [-W#pragma-messages]
    BOOST_HEADER_DEPRECATED( "the facilities in <boost/timer/timer.hpp>" )
    ^
    /usr/local/include/boost/config/header_deprecated.hpp:23:37: note: expanded from macro 'BOOST_HEADER_DEPRECATED'
    # define BOOST_HEADER_DEPRECATED(a) BOOST_PRAGMA_MESSAGE("This header is deprecated. Use " a " instead.")
                                        ^
    /usr/local/include/boost/config/pragma_message.hpp:24:34: note: expanded from macro 'BOOST_PRAGMA_MESSAGE'
    # define BOOST_PRAGMA_MESSAGE(x) _Pragma(BOOST_STRINGIZE(message(x)))
                                     ^
    <scratch space>:144:2: note: expanded from here
     message("This header is deprecated. Use " "the facilities in <boost/timer/timer.hpp>" " instead.")
     ^
    In file included from /Users/lewtun/git/giotto-tda/gtda/externals/bindings/wasserstein_bindings.cpp:6:
    In file included from /Users/lewtun/git/giotto-tda/gtda/externals/bindings/../hera/wasserstein/wasserstein.h:39:
    In file included from /Users/lewtun/git/giotto-tda/gtda/externals/bindings/../hera/wasserstein/auction_runner_gs.h:36:
    In file included from /Users/lewtun/git/giotto-tda/gtda/externals/bindings/../hera/wasserstein/auction_oracle.h:37:
    /Users/lewtun/git/giotto-tda/gtda/externals/bindings/../hera/wasserstein/auction_oracle_kdtree_single_diag.h:89:14: warning: 'decltype(auto)' type specifier is a C++14 extension [-Wc++14-extensions]
        decltype(auto) emplace(Args&&... args)
                 ^
    /Users/lewtun/git/giotto-tda/gtda/externals/bindings/../hera/wasserstein/auction_oracle_kdtree_single_diag.h:89:5: error: deduced return types are a C++14 extension
        decltype(auto) emplace(Args&&... args)
        ^
    /Users/lewtun/git/giotto-tda/gtda/externals/bindings/../hera/wasserstein/auction_oracle_kdtree_single_diag.h:98:14: warning: 'decltype(auto)' type specifier is a C++14 extension [-Wc++14-extensions]
        decltype(auto) insert(const ItemSliceR& item) { return keeper.insert(item); }
                 ^
    /Users/lewtun/git/giotto-tda/gtda/externals/bindings/../hera/wasserstein/auction_oracle_kdtree_single_diag.h:98:5: error: deduced return types are a C++14 extension
        decltype(auto) insert(const ItemSliceR& item) { return keeper.insert(item); }
        ^
    In file included from /Users/lewtun/git/giotto-tda/gtda/externals/bindings/periodic_cubical_complex_bindings.cpp:6:
    In file included from /Users/lewtun/git/giotto-tda/gtda/externals/gudhi-devel/src/python/include/Cubical_complex_interface.h:14:
    In file included from /Users/lewtun/git/giotto-tda/gtda/externals/gudhi-devel/src/Bitmap_cubical_complex/include/gudhi/Bitmap_cubical_complex.h:15:
    /Users/lewtun/git/giotto-tda/gtda/externals/gudhi-devel/src/Bitmap_cubical_complex/include/gudhi/Bitmap_cubical_complex_periodic_boundary_conditions_base.h:117:15: warning: 'Gudhi::cubical_complex::Bitmap_cubical_complex_periodic_boundary_conditions_base<double>::compute_incidence_between_cells' hides overloaded virtual function [-Woverloaded-virtual]
      virtual int compute_incidence_between_cells(std::size_t coface, std::size_t face) {
                  ^
    /Users/lewtun/git/giotto-tda/gtda/externals/gudhi-devel/src/Bitmap_cubical_complex/include/gudhi/Bitmap_cubical_complex.h:48:39: note: in instantiation of template class 'Gudhi::cubical_complex::Bitmap_cubical_complex_periodic_boundary_conditions_base<double>' requested here
    class Bitmap_cubical_complex : public T {
                                          ^
    /Users/lewtun/git/giotto-tda/gtda/externals/gudhi-devel/src/python/include/Cubical_complex_interface.h:27:42: note: in instantiation of template class 'Gudhi::cubical_complex::Bitmap_cubical_complex<Gudhi::cubical_complex::Bitmap_cubical_complex_periodic_boundary_conditions_base<double> >' requested here
    class Cubical_complex_interface : public Bitmap_cubical_complex<CubicalComplexOptions> {
                                             ^
    /Library/Developer/CommandLineTools/usr/bin/../include/c++/v1/type_traits:1661:38: note: in instantiation of template class 'Gudhi::cubical_complex::Cubical_complex_interface<Gudhi::cubical_complex::Bitmap_cubical_complex_periodic_boundary_conditions_base<double> >' requested here
        : public integral_constant<bool, __is_polymorphic(_Tp)> {};
                                         ^
    /Users/lewtun/git/giotto-tda/gtda/externals/pybind11/include/pybind11/pybind11.h:1096:38: note: in instantiation of template class 'std::__1::is_polymorphic<Gudhi::cubical_complex::Cubical_complex_interface<Gudhi::cubical_complex::Bitmap_cubical_complex_periodic_boundary_conditions_base<double> > >' requested here
        static_assert(!has_alias || std::is_polymorphic<type>::value,
                                         ^
    /Users/lewtun/git/giotto-tda/gtda/externals/bindings/periodic_cubical_complex_bindings.cpp:26:3: note: in instantiation of template class 'pybind11::class_<Gudhi::cubical_complex::Cubical_complex_interface<Gudhi::cubical_complex::Bitmap_cubical_complex_periodic_boundary_conditions_base<double> >>' requested here
      py::class_<Periodic_cubical_complex_inst>(
      ^
    /Users/lewtun/git/giotto-tda/gtda/externals/gudhi-devel/src/Bitmap_cubical_complex/include/gudhi/Bitmap_cubical_complex_base.h:131:15: note: hidden overloaded virtual function 'Gudhi::cubical_complex::Bitmap_cubical_complex_base<double>::compute_incidence_between_cells' declared here: different qualifiers ('const' vs unqualified)
      virtual int compute_incidence_between_cells(std::size_t coface, std::size_t face) const {
                  ^
    In file included from /Users/lewtun/git/giotto-tda/gtda/externals/bindings/periodic_cubical_complex_bindings.cpp:16:
    In file included from /Users/lewtun/git/giotto-tda/gtda/externals/pybind11/include/pybind11/pybind11.h:44:
    In file included from /Users/lewtun/git/giotto-tda/gtda/externals/pybind11/include/pybind11/detail/../attr.h:13:
    In file included from /Users/lewtun/git/giotto-tda/gtda/externals/pybind11/include/pybind11/cast.h:13:
    In file included from /Users/lewtun/git/giotto-tda/gtda/externals/pybind11/include/pybind11/detail/../pytypes.h:12:
    /Users/lewtun/git/giotto-tda/gtda/externals/pybind11/include/pybind11/detail/common.h:789:5: error: static_assert failed due to requirement 'detail::integral_constant<bool, false>::value' "pybind11::overload_cast<...> requires compiling in C++14 mode"
        static_assert(detail::deferred_t<std::false_type, Args...>::value,
        ^             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    /Users/lewtun/git/giotto-tda/gtda/externals/bindings/periodic_cubical_complex_bindings.cpp:34:12: note: in instantiation of template class 'pybind11::overload_cast<>' requested here
               py::overload_cast<>(&Periodic_cubical_complex_inst::dimension,
               ^
    1 warning and 1 error generated.
    make[2]: *** [CMakeFiles/gtda_periodic_cubical_complex.dir/gtda/externals/bindings/periodic_cubical_complex_bindings.cpp.o] Error 1
    make[1]: *** [CMakeFiles/gtda_periodic_cubical_complex.dir/all] Error 2
    make[1]: *** Waiting for unfinished jobs....
    4 warnings and 2 errors generated.
    make[2]: *** [CMakeFiles/gtda_wasserstein.dir/gtda/externals/bindings/wasserstein_bindings.cpp.o] Error 1
    make[1]: *** [CMakeFiles/gtda_wasserstein.dir/all] Error 2
    make: *** [all] Error 2
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/Users/lewtun/git/giotto-tda/setup.py", line 152, in <module>
        setup(name=DISTNAME,
      File "/Users/lewtun/miniconda3/envs/gtda/lib/python3.8/site-packages/setuptools/__init__.py", line 165, in setup
        return distutils.core.setup(**attrs)
      File "/Users/lewtun/miniconda3/envs/gtda/lib/python3.8/distutils/core.py", line 148, in setup
        dist.run_commands()
      File "/Users/lewtun/miniconda3/envs/gtda/lib/python3.8/distutils/dist.py", line 966, in run_commands
        self.run_command(cmd)
      File "/Users/lewtun/miniconda3/envs/gtda/lib/python3.8/distutils/dist.py", line 985, in run_command
        cmd_obj.run()
      File "/Users/lewtun/miniconda3/envs/gtda/lib/python3.8/site-packages/setuptools/command/develop.py", line 38, in run
        self.install_for_development()
      File "/Users/lewtun/miniconda3/envs/gtda/lib/python3.8/site-packages/setuptools/command/develop.py", line 140, in install_for_development
        self.run_command('build_ext')
      File "/Users/lewtun/miniconda3/envs/gtda/lib/python3.8/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/Users/lewtun/miniconda3/envs/gtda/lib/python3.8/distutils/dist.py", line 985, in run_command
        cmd_obj.run()
      File "/Users/lewtun/git/giotto-tda/setup.py", line 106, in run
        self.build_extension(ext)
      File "/Users/lewtun/git/giotto-tda/setup.py", line 148, in build_extension
        subprocess.check_call(['cmake', '--build', '.'] + build_args,
      File "/Users/lewtun/miniconda3/envs/gtda/lib/python3.8/subprocess.py", line 364, in check_call
        raise CalledProcessError(retcode, cmd)
    subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--config', 'Release', '--', '-j2']' returned non-zero exit status 2.
    ----------------------------------------
ERROR: Command errored out with exit status 1: /Users/lewtun/miniconda3/envs/gtda/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/Users/lewtun/git/giotto-tda/setup.py'"'"'; __file__='"'"'/Users/lewtun/git/giotto-tda/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps Check the logs for full command output.

Versions

Additional context

@MonkeyBreaker any ideas / tips on what's going wrong?

ulupo commented 4 years ago

That's interesting, things are OK with me on macOS-10.15.6 and Python 3.8.3. And I think I even had 10.16.5 + 3.8.3 in a previous machine...

MonkeyBreaker commented 4 years ago

I'll try to reproduce it (but I do not have any Mac machine...), but I'm a bit surprised that the error is:

Quite strange, I cannot see your gcc version, could you please put the gcc version: gcc --version

ulupo commented 4 years ago

I had a working installation with gcc versioned at 9.3. I've now upgraded gcc to 10.2.0 via brew (leaving boost at 1.720_2) and still cannot reproduce @lewtun's issue.

MonkeyBreaker commented 4 years ago

That's bothersome, @lewtun did you run python -m pip install -e . in a directory where you had already build in the past ? What I mean is, did you already have a build directory ?

If yes, try to delete the build directory and also delete the gtda/externals/pybind11 directory so he download the latest version.

I'm looking if I find a reason of this happening.

ulupo commented 4 years ago

I also just upgraded boost from 1.72.0_2 to 1.73.0, and still can install without issues.

MonkeyBreaker commented 4 years ago

I don't think it's related to boost (at least at the moment) I think more about a compilation conflict if the folder already contained a previous build with a different version. But I'll wait for @lewtun input first.

ulupo commented 4 years ago

Sure, I just wanted to help you try reproduce as I have access to a mac :). I too had previous issues which ended up resolved after cleaning the build folders.

MonkeyBreaker commented 4 years ago

Sure thank you :)

It's always a bit frustrating when compilation fails and it's hard to reproduce :sweat_smile:

At least at the moment, on pybind11 side, the requirement is to be at >=c++11 (as in the past) and from what I see, the error arise with the hera compilation, and the hera version we use did not change at the moment.

lewtun commented 4 years ago

Hi @MonkeyBreaker thanks for the tips! Here's the result from gcc --version:

Configured with: --prefix=/Library/Developer/CommandLineTools/usr --with-gxx-include-dir=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/4.2.1
Apple clang version 11.0.3 (clang-1103.0.32.62)
Target: x86_64-apple-darwin19.5.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

Is it possible that the problem is that I'm using clang instead of a native gcc compiler?

This is a clean install, but in any case I tried as you suggested and deleted the build and pybind directories - the error remains šŸ˜æ.

For reference I'm running cmake version 3.18.1

ulupo commented 4 years ago

Incidentally, running gcc --version gives me similar results as @lewtun, namely

Configured with: --prefix=/Library/Developer/CommandLineTools/usr --with-gxx-include-dir=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/4.2.1
Apple clang version 11.0.3 (clang-1103.0.32.62)
Target: x86_64-apple-darwin19.6.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

However, in my case things work smoothly.

lewtun commented 4 years ago

I'll try a completely clean install later tonight and report back if there's still a problem.

MonkeyBreaker commented 4 years ago

Just to force everything, could you just try to add this to CMakeLists.txt file at root of the project:

Delete the build directory and run everything again.

MonkeyBreaker commented 4 years ago

@ulupo ony your side when you repeated the installation procedure, you were in a conda environment ?

ulupo commented 4 years ago

@MonkeyBreaker yep, a fresh one created just for this test. And I had a fresh clone of the repo.

lewtun commented 4 years ago

@MonkeyBreaker your tweak to CMakeLists.txtdid the trick šŸŽ‰ šŸš€ šŸ„³ ! Should we include this line in future?

MonkeyBreaker commented 4 years ago

We shouldn't :sweat_smile: Normally the CMakeLists.txt file has for each module the flag enable, it's really really strange

BTW, which cmake version both of you are you using ?

ulupo commented 4 years ago

Running cmake --version, I get 3.17.2.

lewtun commented 4 years ago

I'm running cmake version 3.18.1

ulupo commented 4 years ago

@MonkeyBreaker, what do you think about this:

MonkeyBreaker commented 4 years ago

To have more details of what does cmake do in your case, could you please share the output of a file with and without the option enable (set_property(...))

The file: build/<something_related_to_your_architecture>/CMakeFiles/gtda_bottleneck.dir/flags.make

Before running each version, please delete the build folder.

On my system I have something like:

# CMAKE generated file: DO NOT EDIT!
# Generated by "Unix Makefiles" Generator, CMake Version 3.10

# compile CXX with /usr/bin/c++
CXX_FLAGS = -DVERSION_INFO=\"0.2.2\" -O3 -DNDEBUG -flto -fno-fat-lto-objects -fPIC -fvisibility=hidden   -Wall -O3 -std=gnu++14

CXX_DEFINES = -DBOOST_ALL_NO_LIB=1 -DBOOST_RESULT_OF_USE_DECLTYPE=1 -DBOOST_SYSTEM_NO_DEPRECATED=1 -Dgtda_bottleneck_EXPORTS

CXX_INCLUDES = -I/home/julian/workspace/TWS/giotto_learn_github/gtda/externals/hera -I/home/julian/workspace/TWS/giotto_learn_github/gtda/externals/hera/bottleneck/include -isystem /home/julian/workspace/TWS/giotto_learn_github/gtda/externals/pybind11/include -isystem /usr/include/python3.6m
MonkeyBreaker commented 4 years ago

@ulupo I would like to not integrate my "hack" into the library, if there's a problem, I would prefer to find where the configuration is wrong.

Updating the cmake could be a good idea, on my system I have a prehistoric version (cmake version 3.10.2)

ulupo commented 4 years ago

@MonkeyBreaker just checking in to say that I tried upgrading my cmake to be the same version as @lewtun's, i.e. 3.18.1, and I still cannot reproduce the issue on a fresh install (new conda env, removed build files).

MonkeyBreaker commented 4 years ago

@ulupo Thank you for the insights, at least it doesn't see to be the cmake version the culprit.

@lewtun If you can please, provide the content of build/<something_related_to_your_architecture>/CMakeFiles/gtda_bottleneck.dir/flags.make, when the hack I give you is enable and also when disable. (don't forget to delete the build directory ;)

But right now I'm struggling to understand why this has arisen now ...

lewtun commented 4 years ago

@MonkeyBreaker see below for the contents of flags.make:

With hack

# CMAKE generated file: DO NOT EDIT!
# Generated by "Unix Makefiles" Generator, CMake Version 3.18

# compile CXX with /Library/Developer/CommandLineTools/usr/bin/c++
CXX_DEFINES = -DBOOST_ALL_NO_LIB=1 -DBOOST_RESULT_OF_USE_DECLTYPE=1 -DBOOST_SYSTEM_NO_DEPRECATED=1 -Dgtda_bottleneck_EXPORTS

CXX_INCLUDES = -I/usr/local/include -isystem /Users/lewtun/git/giotto-tda/gtda/externals/pybind11/include -isystem /Users/lewtun/miniconda3/envs/gtda/include/python3.8

CXX_FLAGS = -DVERSION_INFO=\"0.1.4\" -O3 -DNDEBUG -flto=thin -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -fPIC -fvisibility=hidden -Wall -O3 -Wno-register -Wno-deprecated-register -std=gnu++14

Without hack

# CMAKE generated file: DO NOT EDIT!
# Generated by "Unix Makefiles" Generator, CMake Version 3.18

# compile CXX with /Library/Developer/CommandLineTools/usr/bin/c++
CXX_DEFINES = -DBOOST_ALL_NO_LIB=1 -DBOOST_RESULT_OF_USE_DECLTYPE=1 -DBOOST_SYSTEM_NO_DEPRECATED=1 -Dgtda_bottleneck_EXPORTS

CXX_INCLUDES = -I/usr/local/include -isystem /Users/lewtun/git/giotto-tda/gtda/externals/pybind11/include -isystem /Users/lewtun/miniconda3/envs/gtda/include/python3.8

CXX_FLAGS = -DVERSION_INFO=\"0.1.4\" -O3 -DNDEBUG -flto=thin -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -fPIC -fvisibility=hidden -Wall -O3 -Wno-register -Wno-deprecated-register -std=gnu++11

As far as I can tell, the contents are the same šŸ¤”. I'm not sure whether it is worth spending much time trying to debug a non-reproducible bug ... I'm happy to use you hack and leave this as a mystery for now (my macbook is > 5 years old, so it could be something weird there).

MonkeyBreaker commented 4 years ago

@lewtun Don't worry but thank you, we can clearly see the issue in your files:

With hack

-O3 -Wno-register -Wno-deprecated-register -std=gnu++14

Without hack

-O3 -Wno-register -Wno-deprecated-register -std=gnu++11

Now why the standard isn't set properly to c++14, is quite strange, I'll look on cmake forums if I can find something related to this issue.

MonkeyBreaker commented 4 years ago

Could you try on the meantime 2 different versions of CMakeLists.txt

Each time of course delete the build directory and attach the output of the same file used before.

I don't expect the build to success, I only updated for bottleneck module, I want to see if any of the two methods set correctly the flag.

I'll write an issue on the issue tracker from cmake because I suspect that the issue is somewhere there.

Sorry for the inconvenience ...

MonkeyBreaker commented 4 years ago

Maybe I found out the reason, see link.

If the fix resolves the issue, good, but I don't understand why everyone isn't affected by the issue.

After trying the CMake files in my previous answer, take the orginal CMakeList.txt file, without any modification, and add before the project(gtda_bindings LANGUAGES CXX) line (line 2), the following:

# Fix behavior of CMAKE_CXX_STANDARD when targeting macOS.
if (POLICY CMP0025)
  cmake_policy(SET CMP0025 NEW)
endif ()
lewtun commented 4 years ago

Hi @MonkeyBreaker, I tried doing the install with the 2 versions of CMakeLists.txt that you provided. In each case I do not find a file is produced in build/<something_related_to_your_architecture>/CMakeFiles/gtda_bottleneck.dir/flags.make; indeed there is no gtda_bottleneck.dir folder at all and the error says that it can't find cmake/HelperBoost.cmake šŸ¤” . In case it helps, below are the stack traces:

Using cfeatures_CMakeLists.txt

  Running setup.py develop for giotto-tda
    ERROR: Command errored out with exit status 1:
     command: /Users/lewtun/miniconda3/envs/gtda/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/Users/lewtun/git/giotto-tda/setup.py'"'"'; __file__='"'"'/Users/lewtun/git/giotto-tda/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps
         cwd: /Users/lewtun/git/giotto-tda/
    Complete output (56 lines):
    running develop
    running egg_info
    writing giotto_tda.egg-info/PKG-INFO
    writing dependency_links to giotto_tda.egg-info/dependency_links.txt
    writing requirements to giotto_tda.egg-info/requires.txt
    writing top-level names to giotto_tda.egg-info/top_level.txt
    reading manifest file 'giotto_tda.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    writing manifest file 'giotto_tda.egg-info/SOURCES.txt'
    running build_ext
    -- The CXX compiler identification is AppleClang 11.0.3.11030032
    -- Detecting CXX compiler ABI info
    -- Detecting CXX compiler ABI info - done
    -- Check for working CXX compiler: /Library/Developer/CommandLineTools/usr/bin/c++ - skipped
    -- Detecting CXX compile features
    -- Detecting CXX compile features - done
    -- pybind11 v2.6.0 dev
    -- Found PythonInterp: /Users/lewtun/miniconda3/envs/gtda/bin/python (found version "3.8.5")
    -- Found PythonLibs: /Users/lewtun/miniconda3/envs/gtda/lib/libpython3.8.dylib
    CMake Error at CMakeLists.txt:7 (include):
      include could not find load file:

        cmake/HelperBoost.cmake

    -- Found OpenMP_CXX: -Xclang -fopenmp (found version "3.1")
    -- Found OpenMP: TRUE (found version "3.1")
    -- Configuring incomplete, errors occurred!
    See also "/Users/lewtun/git/giotto-tda/build/temp.macosx-10.9-x86_64-3.8/CMakeFiles/CMakeOutput.log".
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/Users/lewtun/git/giotto-tda/setup.py", line 152, in <module>
        setup(name=DISTNAME,
      File "/Users/lewtun/miniconda3/envs/gtda/lib/python3.8/site-packages/setuptools/__init__.py", line 165, in setup
        return distutils.core.setup(**attrs)
      File "/Users/lewtun/miniconda3/envs/gtda/lib/python3.8/distutils/core.py", line 148, in setup
        dist.run_commands()
      File "/Users/lewtun/miniconda3/envs/gtda/lib/python3.8/distutils/dist.py", line 966, in run_commands
        self.run_command(cmd)
      File "/Users/lewtun/miniconda3/envs/gtda/lib/python3.8/distutils/dist.py", line 985, in run_command
        cmd_obj.run()
      File "/Users/lewtun/miniconda3/envs/gtda/lib/python3.8/site-packages/setuptools/command/develop.py", line 38, in run
        self.install_for_development()
      File "/Users/lewtun/miniconda3/envs/gtda/lib/python3.8/site-packages/setuptools/command/develop.py", line 140, in install_for_development
        self.run_command('build_ext')
      File "/Users/lewtun/miniconda3/envs/gtda/lib/python3.8/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/Users/lewtun/miniconda3/envs/gtda/lib/python3.8/distutils/dist.py", line 985, in run_command
        cmd_obj.run()
      File "/Users/lewtun/git/giotto-tda/setup.py", line 106, in run
        self.build_extension(ext)
      File "/Users/lewtun/git/giotto-tda/setup.py", line 146, in build_extension
        subprocess.check_call(['cmake', ext.sourcedir] + cmake_args,
      File "/Users/lewtun/miniconda3/envs/gtda/lib/python3.8/subprocess.py", line 364, in check_call
        raise CalledProcessError(retcode, cmd)
    subprocess.CalledProcessError: Command '['cmake', '/Users/lewtun/git/giotto-tda', '-DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/Users/lewtun/git/giotto-tda/gtda/externals/modules', '-DPYTHON_EXECUTABLE=/Users/lewtun/miniconda3/envs/gtda/bin/python', '-DCMAKE_BUILD_TYPE=Release']' returned non-zero exit status 1.
    ----------------------------------------
ERROR: Command errored out with exit status 1: /Users/lewtun/miniconda3/envs/gtda/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/Users/lewtun/git/giotto-tda/setup.py'"'"'; __file__='"'"'/Users/lewtun/git/giotto-tda/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps Check the logs for full command output.

Using checkEnable_CMakeLists.txt

  Running setup.py develop for giotto-tda
    ERROR: Command errored out with exit status 1:
     command: /Users/lewtun/miniconda3/envs/gtda/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/Users/lewtun/git/giotto-tda/setup.py'"'"'; __file__='"'"'/Users/lewtun/git/giotto-tda/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps
         cwd: /Users/lewtun/git/giotto-tda/
    Complete output (57 lines):
    running develop
    running egg_info
    writing giotto_tda.egg-info/PKG-INFO
    writing dependency_links to giotto_tda.egg-info/dependency_links.txt
    writing requirements to giotto_tda.egg-info/requires.txt
    writing top-level names to giotto_tda.egg-info/top_level.txt
    reading manifest file 'giotto_tda.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    writing manifest file 'giotto_tda.egg-info/SOURCES.txt'
    running build_ext
    Cloning into '/Users/lewtun/git/giotto-tda/gtda/externals/pybind11'...
    -- The CXX compiler identification is AppleClang 11.0.3.11030032
    -- Detecting CXX compiler ABI info
    -- Detecting CXX compiler ABI info - done
    -- Check for working CXX compiler: /Library/Developer/CommandLineTools/usr/bin/c++ - skipped
    -- Detecting CXX compile features
    -- Detecting CXX compile features - done
    -- pybind11 v2.6.0 dev
    -- Found PythonInterp: /Users/lewtun/miniconda3/envs/gtda/bin/python (found version "3.8.5")
    -- Found PythonLibs: /Users/lewtun/miniconda3/envs/gtda/lib/libpython3.8.dylib
    CMake Error at CMakeLists.txt:7 (include):
      include could not find load file:

        cmake/HelperBoost.cmake

    -- Found OpenMP_CXX: -Xclang -fopenmp (found version "3.1")
    -- Found OpenMP: TRUE (found version "3.1")
    -- Configuring incomplete, errors occurred!
    See also "/Users/lewtun/git/giotto-tda/build/temp.macosx-10.9-x86_64-3.8/CMakeFiles/CMakeOutput.log".
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/Users/lewtun/git/giotto-tda/setup.py", line 152, in <module>
        setup(name=DISTNAME,
      File "/Users/lewtun/miniconda3/envs/gtda/lib/python3.8/site-packages/setuptools/__init__.py", line 165, in setup
        return distutils.core.setup(**attrs)
      File "/Users/lewtun/miniconda3/envs/gtda/lib/python3.8/distutils/core.py", line 148, in setup
        dist.run_commands()
      File "/Users/lewtun/miniconda3/envs/gtda/lib/python3.8/distutils/dist.py", line 966, in run_commands
        self.run_command(cmd)
      File "/Users/lewtun/miniconda3/envs/gtda/lib/python3.8/distutils/dist.py", line 985, in run_command
        cmd_obj.run()
      File "/Users/lewtun/miniconda3/envs/gtda/lib/python3.8/site-packages/setuptools/command/develop.py", line 38, in run
        self.install_for_development()
      File "/Users/lewtun/miniconda3/envs/gtda/lib/python3.8/site-packages/setuptools/command/develop.py", line 140, in install_for_development
        self.run_command('build_ext')
      File "/Users/lewtun/miniconda3/envs/gtda/lib/python3.8/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/Users/lewtun/miniconda3/envs/gtda/lib/python3.8/distutils/dist.py", line 985, in run_command
        cmd_obj.run()
      File "/Users/lewtun/git/giotto-tda/setup.py", line 106, in run
        self.build_extension(ext)
      File "/Users/lewtun/git/giotto-tda/setup.py", line 146, in build_extension
        subprocess.check_call(['cmake', ext.sourcedir] + cmake_args,
      File "/Users/lewtun/miniconda3/envs/gtda/lib/python3.8/subprocess.py", line 364, in check_call
        raise CalledProcessError(retcode, cmd)
    subprocess.CalledProcessError: Command '['cmake', '/Users/lewtun/git/giotto-tda', '-DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/Users/lewtun/git/giotto-tda/gtda/externals/modules', '-DPYTHON_EXECUTABLE=/Users/lewtun/miniconda3/envs/gtda/bin/python', '-DCMAKE_BUILD_TYPE=Release']' returned non-zero exit status 1.
    ----------------------------------------
ERROR: Command errored out with exit status 1: /Users/lewtun/miniconda3/envs/gtda/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/Users/lewtun/git/giotto-tda/setup.py'"'"'; __file__='"'"'/Users/lewtun/git/giotto-tda/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps Check the logs for full command output.

I also tried your final suggestion to fix the behaviour of CMAKE_CXX_STANDARD but this does not appear to work:

Contents of flags.make

# CMAKE generated file: DO NOT EDIT!
# Generated by "Unix Makefiles" Generator, CMake Version 3.18

# compile CXX with /Library/Developer/CommandLineTools/usr/bin/c++
CXX_DEFINES = -DBOOST_ALL_NO_LIB=1 -DBOOST_RESULT_OF_USE_DECLTYPE=1 -DBOOST_SYSTEM_NO_DEPRECATED=1 -Dgtda_bottleneck_EXPORTS

CXX_INCLUDES = -I/usr/local/include -isystem /Users/lewtun/git/giotto-tda/gtda/externals/pybind11/include -isystem /Users/lewtun/miniconda3/envs/gtda/include/python3.8

CXX_FLAGS = -DVERSION_INFO=\"0.1.4\" -O3 -DNDEBUG -flto=thin -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -fPIC -fvisibility=hidden -Wall -O3 -Wno-register -Wno-deprecated-register -std=gnu++11

Stack trace

  Running setup.py develop for giotto-tda
    ERROR: Command errored out with exit status 1:
     command: /Users/lewtun/miniconda3/envs/gtda/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/Users/lewtun/git/giotto-tda/setup.py'"'"'; __file__='"'"'/Users/lewtun/git/giotto-tda/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps
         cwd: /Users/lewtun/git/giotto-tda/
    Complete output (120 lines):
    running develop
    running egg_info
    writing giotto_tda.egg-info/PKG-INFO
    writing dependency_links to giotto_tda.egg-info/dependency_links.txt
    writing requirements to giotto_tda.egg-info/requires.txt
    writing top-level names to giotto_tda.egg-info/top_level.txt
    reading manifest file 'giotto_tda.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    writing manifest file 'giotto_tda.egg-info/SOURCES.txt'
    running build_ext
    -- The CXX compiler identification is AppleClang 11.0.3.11030032
    -- Detecting CXX compiler ABI info
    -- Detecting CXX compiler ABI info - done
    -- Check for working CXX compiler: /Library/Developer/CommandLineTools/usr/bin/c++ - skipped
    -- Detecting CXX compile features
    -- Detecting CXX compile features - done
    -- pybind11 v2.6.0 dev
    -- Found PythonInterp: /Users/lewtun/miniconda3/envs/gtda/bin/python (found version "3.8.5")
    -- Found PythonLibs: /Users/lewtun/miniconda3/envs/gtda/lib/libpython3.8.dylib
    -- Found Boost: /usr/local/lib/cmake/Boost-1.72.0/BoostConfig.cmake (found suitable version "1.72.0", minimum required is "1.56")
    -- Found OpenMP_CXX: -Xclang -fopenmp (found version "3.1")
    -- Found OpenMP: TRUE (found version "3.1")
    -- Configuring done
    -- Generating done
    -- Build files have been written to: /Users/lewtun/git/giotto-tda/build/temp.macosx-10.9-x86_64-3.8
    Scanning dependencies of target gtda_cech_complex
    [  4%] Building CXX object CMakeFiles/gtda_cech_complex.dir/gtda/externals/bindings/cech_complex_bindings.cpp.o
    clang: warning: optimization flag '-frounding-math' is not supported [-Wignored-optimization-argument]
    clang: warning: argument unused during compilation: '-shared' [-Wunused-command-line-argument]
    Scanning dependencies of target gtda_wasserstein
    [  9%] Building CXX object CMakeFiles/gtda_wasserstein.dir/gtda/externals/bindings/wasserstein_bindings.cpp.o
    In file included from /Users/lewtun/git/giotto-tda/gtda/externals/bindings/wasserstein_bindings.cpp:6:
    In file included from /Users/lewtun/git/giotto-tda/gtda/externals/bindings/../hera/wasserstein/wasserstein.h:39:
    In file included from /Users/lewtun/git/giotto-tda/gtda/externals/bindings/../hera/wasserstein/auction_runner_gs.h:36:
    In file included from /Users/lewtun/git/giotto-tda/gtda/externals/bindings/../hera/wasserstein/auction_oracle.h:36:
    In file included from /Users/lewtun/git/giotto-tda/gtda/externals/bindings/../hera/wasserstein/auction_oracle_kdtree_restricted.h:44:
    In file included from /Users/lewtun/git/giotto-tda/gtda/externals/bindings/../hera/wasserstein/dnn/geometry/euclidean-fixed.h:16:
    In file included from /Users/lewtun/git/giotto-tda/gtda/externals/bindings/../hera/wasserstein/dnn/geometry/../parallel/tbb.h:138:
    /usr/local/include/boost/progress.hpp:23:1: warning: This header is deprecated. Use the facilities in <boost/timer/timer.hpp> or <boost/timer/progress_display.hpp> instead. [-W#pragma-messages]
    BOOST_HEADER_DEPRECATED( "the facilities in <boost/timer/timer.hpp> or <boost/timer/progress_display.hpp>" )
    ^
    /usr/local/include/boost/config/header_deprecated.hpp:23:37: note: expanded from macro 'BOOST_HEADER_DEPRECATED'
    # define BOOST_HEADER_DEPRECATED(a) BOOST_PRAGMA_MESSAGE("This header is deprecated. Use " a " instead.")
                                        ^
    /usr/local/include/boost/config/pragma_message.hpp:24:34: note: expanded from macro 'BOOST_PRAGMA_MESSAGE'
    # define BOOST_PRAGMA_MESSAGE(x) _Pragma(BOOST_STRINGIZE(message(x)))
                                     ^
    <scratch space>:141:2: note: expanded from here
     message("This header is deprecated. Use " "the facilities in <boost/timer/timer.hpp> or <boost/timer/progress_display.hpp>" " instead.")
     ^
    In file included from /Users/lewtun/git/giotto-tda/gtda/externals/bindings/wasserstein_bindings.cpp:6:
    In file included from /Users/lewtun/git/giotto-tda/gtda/externals/bindings/../hera/wasserstein/wasserstein.h:39:
    In file included from /Users/lewtun/git/giotto-tda/gtda/externals/bindings/../hera/wasserstein/auction_runner_gs.h:36:
    In file included from /Users/lewtun/git/giotto-tda/gtda/externals/bindings/../hera/wasserstein/auction_oracle.h:36:
    In file included from /Users/lewtun/git/giotto-tda/gtda/externals/bindings/../hera/wasserstein/auction_oracle_kdtree_restricted.h:44:
    In file included from /Users/lewtun/git/giotto-tda/gtda/externals/bindings/../hera/wasserstein/dnn/geometry/euclidean-fixed.h:16:
    In file included from /Users/lewtun/git/giotto-tda/gtda/externals/bindings/../hera/wasserstein/dnn/geometry/../parallel/tbb.h:138:
    In file included from /usr/local/include/boost/progress.hpp:25:
    /usr/local/include/boost/timer.hpp:21:1: warning: This header is deprecated. Use the facilities in <boost/timer/timer.hpp> instead. [-W#pragma-messages]
    BOOST_HEADER_DEPRECATED( "the facilities in <boost/timer/timer.hpp>" )
    ^
    /usr/local/include/boost/config/header_deprecated.hpp:23:37: note: expanded from macro 'BOOST_HEADER_DEPRECATED'
    # define BOOST_HEADER_DEPRECATED(a) BOOST_PRAGMA_MESSAGE("This header is deprecated. Use " a " instead.")
                                        ^
    /usr/local/include/boost/config/pragma_message.hpp:24:34: note: expanded from macro 'BOOST_PRAGMA_MESSAGE'
    # define BOOST_PRAGMA_MESSAGE(x) _Pragma(BOOST_STRINGIZE(message(x)))
                                     ^
    <scratch space>:144:2: note: expanded from here
     message("This header is deprecated. Use " "the facilities in <boost/timer/timer.hpp>" " instead.")
     ^
    In file included from /Users/lewtun/git/giotto-tda/gtda/externals/bindings/wasserstein_bindings.cpp:6:
    In file included from /Users/lewtun/git/giotto-tda/gtda/externals/bindings/../hera/wasserstein/wasserstein.h:39:
    In file included from /Users/lewtun/git/giotto-tda/gtda/externals/bindings/../hera/wasserstein/auction_runner_gs.h:36:
    In file included from /Users/lewtun/git/giotto-tda/gtda/externals/bindings/../hera/wasserstein/auction_oracle.h:37:
    /Users/lewtun/git/giotto-tda/gtda/externals/bindings/../hera/wasserstein/auction_oracle_kdtree_single_diag.h:89:14: warning: 'decltype(auto)' type specifier is a C++14 extension [-Wc++14-extensions]
        decltype(auto) emplace(Args&&... args)
                 ^
    /Users/lewtun/git/giotto-tda/gtda/externals/bindings/../hera/wasserstein/auction_oracle_kdtree_single_diag.h:89:5: error: deduced return types are a C++14 extension
        decltype(auto) emplace(Args&&... args)
        ^
    /Users/lewtun/git/giotto-tda/gtda/externals/bindings/../hera/wasserstein/auction_oracle_kdtree_single_diag.h:98:14: warning: 'decltype(auto)' type specifier is a C++14 extension [-Wc++14-extensions]
        decltype(auto) insert(const ItemSliceR& item) { return keeper.insert(item); }
                 ^
    /Users/lewtun/git/giotto-tda/gtda/externals/bindings/../hera/wasserstein/auction_oracle_kdtree_single_diag.h:98:5: error: deduced return types are a C++14 extension
        decltype(auto) insert(const ItemSliceR& item) { return keeper.insert(item); }
        ^
    4 warnings and 2 errors generated.
    make[2]: *** [CMakeFiles/gtda_wasserstein.dir/gtda/externals/bindings/wasserstein_bindings.cpp.o] Error 1
    make[1]: *** [CMakeFiles/gtda_wasserstein.dir/all] Error 2
    make[1]: *** Waiting for unfinished jobs....
    [ 13%] Linking CXX shared module ../../gtda/externals/modules/gtda_cech_complex.cpython-38-darwin.so
    [ 13%] Built target gtda_cech_complex
    make: *** [all] Error 2
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/Users/lewtun/git/giotto-tda/setup.py", line 152, in <module>
        setup(name=DISTNAME,
      File "/Users/lewtun/miniconda3/envs/gtda/lib/python3.8/site-packages/setuptools/__init__.py", line 165, in setup
        return distutils.core.setup(**attrs)
      File "/Users/lewtun/miniconda3/envs/gtda/lib/python3.8/distutils/core.py", line 148, in setup
        dist.run_commands()
      File "/Users/lewtun/miniconda3/envs/gtda/lib/python3.8/distutils/dist.py", line 966, in run_commands
        self.run_command(cmd)
      File "/Users/lewtun/miniconda3/envs/gtda/lib/python3.8/distutils/dist.py", line 985, in run_command
        cmd_obj.run()
      File "/Users/lewtun/miniconda3/envs/gtda/lib/python3.8/site-packages/setuptools/command/develop.py", line 38, in run
        self.install_for_development()
      File "/Users/lewtun/miniconda3/envs/gtda/lib/python3.8/site-packages/setuptools/command/develop.py", line 140, in install_for_development
        self.run_command('build_ext')
      File "/Users/lewtun/miniconda3/envs/gtda/lib/python3.8/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/Users/lewtun/miniconda3/envs/gtda/lib/python3.8/distutils/dist.py", line 985, in run_command
        cmd_obj.run()
      File "/Users/lewtun/git/giotto-tda/setup.py", line 106, in run
        self.build_extension(ext)
      File "/Users/lewtun/git/giotto-tda/setup.py", line 148, in build_extension
        subprocess.check_call(['cmake', '--build', '.'] + build_args,
      File "/Users/lewtun/miniconda3/envs/gtda/lib/python3.8/subprocess.py", line 364, in check_call
        raise CalledProcessError(retcode, cmd)
    subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--config', 'Release', '--', '-j2']' returned non-zero exit status 2.
    ----------------------------------------
ERROR: Command errored out with exit status 1: /Users/lewtun/miniconda3/envs/gtda/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/Users/lewtun/git/giotto-tda/setup.py'"'"'; __file__='"'"'/Users/lewtun/git/giotto-tda/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps Check the logs for full command output.
MonkeyBreaker commented 4 years ago

@lewtun It's really strange it cannot find cmake/HelperBoost.cmake, it's like the file didn't exist at all, like the folder cmake was deleted, even more strange, why on the last case it didn't encounter this issue ...

Wait, how do you download giotto-tda library ? I have the feeling that you're not working on the master branch.

lewtun commented 4 years ago

hi @MonkeyBreaker yes you were right, my master branch was outdated - sorry!

here's the contents of flags.make for the two cmake files you shared:

Using cfeatures_CMakeLists.txt

# CMAKE generated file: DO NOT EDIT!
# Generated by "Unix Makefiles" Generator, CMake Version 3.18

# compile CXX with /Library/Developer/CommandLineTools/usr/bin/c++
CXX_DEFINES = -DBOOST_ALL_NO_LIB=1 -DBOOST_RESULT_OF_USE_DECLTYPE=1 -DBOOST_SYSTEM_NO_DEPRECATED=1 -Dgtda_bottleneck_EXPORTS

CXX_INCLUDES = -I/usr/local/include -I/Users/lewtun/git/giotto-tda/gtda/externals/hera -I/Users/lewtun/git/giotto-tda/gtda/externals/hera/bottleneck/include -isystem /Users/lewtun/git/giotto-tda/gtda/externals/pybind11/include -isystem /Users/lewtun/miniconda3/envs/gtda/include/python3.8

CXX_FLAGS = -DVERSION_INFO=\"0.2.2\" -O3 -DNDEBUG -flto=thin -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -fPIC -fvisibility=hidden -Wall -O3 -Wno-register -Wno-deprecated-register -std=gnu++14

Using checkEnable_CMakeLists.txt

# CMAKE generated file: DO NOT EDIT!
# Generated by "Unix Makefiles" Generator, CMake Version 3.18

# compile CXX with /Library/Developer/CommandLineTools/usr/bin/c++
CXX_DEFINES = -DBOOST_ALL_NO_LIB=1 -DBOOST_RESULT_OF_USE_DECLTYPE=1 -DBOOST_SYSTEM_NO_DEPRECATED=1 -Dgtda_bottleneck_EXPORTS

CXX_INCLUDES = -I/usr/local/include -I/Users/lewtun/git/giotto-tda/gtda/externals/hera -I/Users/lewtun/git/giotto-tda/gtda/externals/hera/bottleneck/include -isystem /Users/lewtun/git/giotto-tda/gtda/externals/pybind11/include -isystem /Users/lewtun/miniconda3/envs/gtda/include/python3.8

CXX_FLAGS = -DVERSION_INFO=\"0.2.2\" -O3 -DNDEBUG -flto=thin -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -fPIC -fvisibility=hidden -Wall -O3 -Wno-register -Wno-deprecated-register -std=gnu++14

In both cases, the installation did not succeed (as you expected). To be safe I reran your final suggestion

Maybe I found out the reason, see link.

If the fix resolves the issue, good, but I don't understand why everyone isn't affected by the issue.

After trying the CMake files in my previous answer, take the orginal CMakeList.txt file, without any modification, and add before the project(gtda_bindings LANGUAGES CXX) line (line 2), the following:

# Fix behavior of CMAKE_CXX_STANDARD when targeting macOS.
if (POLICY CMP0025)
  cmake_policy(SET CMP0025 NEW)
endif ()

but found the install still does not succeed. Contents of flags.make and the stack trace are below:

Contents of flags.make

# CMAKE generated file: DO NOT EDIT!
# Generated by "Unix Makefiles" Generator, CMake Version 3.18

# compile CXX with /Library/Developer/CommandLineTools/usr/bin/c++
CXX_DEFINES = -DBOOST_ALL_NO_LIB=1 -DBOOST_RESULT_OF_USE_DECLTYPE=1 -DBOOST_SYSTEM_NO_DEPRECATED=1 -Dgtda_bottleneck_EXPORTS

CXX_INCLUDES = -I/usr/local/include -I/Users/lewtun/git/giotto-tda/gtda/externals/hera -I/Users/lewtun/git/giotto-tda/gtda/externals/hera/bottleneck/include -isystem /Users/lewtun/git/giotto-tda/gtda/externals/pybind11/include -isystem /Users/lewtun/miniconda3/envs/gtda/include/python3.8

CXX_FLAGS = -DVERSION_INFO=\"0.2.2\" -O3 -DNDEBUG -flto=thin -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -fPIC -fvisibility=hidden -Wall -O3 -Wno-register -Wno-deprecated-register -std=gnu++14

Stack trace

  Running setup.py develop for giotto-tda
    ERROR: Command errored out with exit status 1:
     command: /Users/lewtun/miniconda3/envs/gtda/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/Users/lewtun/git/giotto-tda/setup.py'"'"'; __file__='"'"'/Users/lewtun/git/giotto-tda/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps
         cwd: /Users/lewtun/git/giotto-tda/
    Complete output (72 lines):
    running develop
    running egg_info
    writing giotto_tda.egg-info/PKG-INFO
    writing dependency_links to giotto_tda.egg-info/dependency_links.txt
    writing requirements to giotto_tda.egg-info/requires.txt
    writing top-level names to giotto_tda.egg-info/top_level.txt
    reading manifest file 'giotto_tda.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    writing manifest file 'giotto_tda.egg-info/SOURCES.txt'
    running build_ext
    -- The CXX compiler identification is AppleClang 11.0.3.11030032
    -- Detecting CXX compiler ABI info
    -- Detecting CXX compiler ABI info - done
    -- Check for working CXX compiler: /Library/Developer/CommandLineTools/usr/bin/c++ - skipped
    -- Detecting CXX compile features
    -- Detecting CXX compile features - done
    -- pybind11 v2.6.0 dev
    -- Found PythonInterp: /Users/lewtun/miniconda3/envs/gtda/bin/python (found version "3.8.5")
    -- Found PythonLibs: /Users/lewtun/miniconda3/envs/gtda/lib/libpython3.8.dylib
    -- BOOST_ROOT_PIPELINE:
    -- BOOST_ROOT:
    -- Found Boost: /usr/local/lib/cmake/Boost-1.72.0/BoostConfig.cmake (found suitable version "1.72.0", minimum required is "1.56")
    -- Boost_INCLUDE_DIR: /usr/local/include
    -- Boost_INCLUDE_DIRS: /usr/local/include
    -- Found OpenMP_CXX: -Xclang -fopenmp (found version "3.1")
    -- Found OpenMP: TRUE (found version "3.1")
    -- Configuring done
    -- Generating done
    -- Build files have been written to: /Users/lewtun/git/giotto-tda/build/temp.macosx-10.9-x86_64-3.8
    Scanning dependencies of target gtda_wasserstein
    [  4%] Building CXX object CMakeFiles/gtda_wasserstein.dir/gtda/externals/bindings/wasserstein_bindings.cpp.o
    /Users/lewtun/git/giotto-tda/gtda/externals/bindings/wasserstein_bindings.cpp:6:10: fatal error: 'wasserstein/include/wasserstein.h' file not found
    #include <wasserstein/include/wasserstein.h>
             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    Scanning dependencies of target gtda_cech_complex
    [  8%] Building CXX object CMakeFiles/gtda_cech_complex.dir/gtda/externals/bindings/cech_complex_bindings.cpp.o
    clang: warning: optimization flag '-frounding-math' is not supported [-Wignored-optimization-argument]
    clang: warning: argument unused during compilation: '-shared' [-Wunused-command-line-argument]
    1 error generated.
    make[2]: *** [CMakeFiles/gtda_wasserstein.dir/gtda/externals/bindings/wasserstein_bindings.cpp.o] Error 1
    make[1]: *** [CMakeFiles/gtda_wasserstein.dir/all] Error 2
    make[1]: *** Waiting for unfinished jobs....
    [ 12%] Linking CXX shared module ../../gtda/externals/modules/gtda_cech_complex.cpython-38-darwin.so
    [ 12%] Built target gtda_cech_complex
    make: *** [all] Error 2
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/Users/lewtun/git/giotto-tda/setup.py", line 154, in <module>
        setup(name=DISTNAME,
      File "/Users/lewtun/miniconda3/envs/gtda/lib/python3.8/site-packages/setuptools/__init__.py", line 165, in setup
        return distutils.core.setup(**attrs)
      File "/Users/lewtun/miniconda3/envs/gtda/lib/python3.8/distutils/core.py", line 148, in setup
        dist.run_commands()
      File "/Users/lewtun/miniconda3/envs/gtda/lib/python3.8/distutils/dist.py", line 966, in run_commands
        self.run_command(cmd)
      File "/Users/lewtun/miniconda3/envs/gtda/lib/python3.8/distutils/dist.py", line 985, in run_command
        cmd_obj.run()
      File "/Users/lewtun/miniconda3/envs/gtda/lib/python3.8/site-packages/setuptools/command/develop.py", line 38, in run
        self.install_for_development()
      File "/Users/lewtun/miniconda3/envs/gtda/lib/python3.8/site-packages/setuptools/command/develop.py", line 140, in install_for_development
        self.run_command('build_ext')
      File "/Users/lewtun/miniconda3/envs/gtda/lib/python3.8/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/Users/lewtun/miniconda3/envs/gtda/lib/python3.8/distutils/dist.py", line 985, in run_command
        cmd_obj.run()
      File "/Users/lewtun/git/giotto-tda/setup.py", line 108, in run
        self.build_extension(ext)
      File "/Users/lewtun/git/giotto-tda/setup.py", line 150, in build_extension
        subprocess.check_call(['cmake', '--build', '.'] + build_args,
      File "/Users/lewtun/miniconda3/envs/gtda/lib/python3.8/subprocess.py", line 364, in check_call
        raise CalledProcessError(retcode, cmd)
    subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--config', 'Release', '--', '-j2']' returned non-zero exit status 2.
    ----------------------------------------
ERROR: Command errored out with exit status 1: /Users/lewtun/miniconda3/envs/gtda/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/Users/lewtun/git/giotto-tda/setup.py'"'"'; __file__='"'"'/Users/lewtun/git/giotto-tda/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps Check the logs for full command output.
ulupo commented 4 years ago

@lewtun, could you pull the most recent version of master which includes the fixes in #451 , and try again? Or have you already done so and I'm confused?

lewtun commented 4 years ago

i'm working from commit f65f89947240dee62d9071789a8e148effe9d257 which looks to be the latest on master. sorry for the confusion šŸ˜¬

ulupo commented 4 years ago

Right. But this seems at odds with line

    -- pybind11 v2.6.0 dev

in your stack trace, doesn't it? Perhaps you can try to clean up any remnants of a previous pybind11 installation?

ulupo commented 4 years ago

I.e. I think the following line: https://github.com/giotto-ai/giotto-tda/blob/f65f89947240dee62d9071789a8e148effe9d257/setup.py#L114 might be responsible for your issue. So maybe you can try deleting gtda/externals/pybind11 and try again?

lewtun commented 4 years ago

Good catch @ulupo! Removing the pybind11 directory did the trick with 

# Fix behavior of CMAKE_CXX_STANDARD when targeting macOS.
if (POLICY CMP0025)
  cmake_policy(SET CMP0025 NEW)
endif ()

in the CMakeLists.txt file. I think adding this change should fix the problem for future noobs like myself šŸ™ƒ

ulupo commented 4 years ago

@lewtun that's great news! Are you convinced that the change to CMakeLists.txt was also necessary? I ask because we observed that an update to pybind11's master branch caused our Azure pipelines to fail, but this was single-handedly solved on the Azure machines by fixing to the 2.5 branch, without any changes to CMakeLists.txt.

lewtun commented 4 years ago

Even better catch šŸ˜ø !

I just did a clean build from commit 3bb84d905fb7b8294247bc04b7993a626a4aa3bb (without @MonkeyBreaker's proposed change to CMakeLists.txt) and the installation also works!

So after all this, I think we can safely close this issue. Thanks for all you help and sorry for taking up your time on this annoying topic

ulupo commented 4 years ago

Hoorray! Fixed by #451.