facebook / folly

An open-source C++ library developed and used at Facebook.
https://groups.google.com/forum/?fromgroups#!forum/facebook-folly
Apache License 2.0
28.13k stars 5.55k forks source link

coro/Task.h compile error on clang 10 #1353

Closed seelabs closed 3 months ago

seelabs commented 4 years ago

Compiling the following file results in a compile errors on clang 10 on linux (compiled with -stdlib=libc++)

#include <folly/experimental/coro/Task.h>
int
main(int argc, char** argv){}

The errors were:

ninja -v |& tee compile_log.txt
[1/2] ccache /home/swd/apps/clang-10.0/bin/clang++  -DFMT_LOCALE -isystem /home/swd/projs/common/boost_1_72_0-clang -isystem /usr/include/libiberty -isystem /home/swd/projs/common/folly-2020.04.13.00-clang/include -isystem /home/swd/projs/common/fmt-6.2-clang/include -g   -std=c++2a -stdlib=libc++ -MD -MT CMakeFiles/HelloFolly.dir/main.cpp.o -MF CMakeFiles/HelloFolly.dir/main.cpp.o.d -o CMakeFiles/HelloFolly.dir/main.cpp.o -c ../../main.cpp
FAILED: CMakeFiles/HelloFolly.dir/main.cpp.o 
ccache /home/swd/apps/clang-10.0/bin/clang++  -DFMT_LOCALE -isystem /home/swd/projs/common/boost_1_72_0-clang -isystem /usr/include/libiberty -isystem /home/swd/projs/common/folly-2020.04.13.00-clang/include -isystem /home/swd/projs/common/fmt-6.2-clang/include -g   -std=c++2a -stdlib=libc++ -MD -MT CMakeFiles/HelloFolly.dir/main.cpp.o -MF CMakeFiles/HelloFolly.dir/main.cpp.o.d -o CMakeFiles/HelloFolly.dir/main.cpp.o -c ../../main.cpp
In file included from ../../main.cpp:55:
In file included from /home/swd/projs/common/folly-2020.04.13.00-clang/include/folly/experimental/coro/Task.h:30:
In file included from /home/swd/projs/common/folly-2020.04.13.00-clang/include/folly/experimental/coro/CurrentExecutor.h:22:
In file included from /home/swd/projs/common/folly-2020.04.13.00-clang/include/folly/experimental/coro/ViaIfAsync.h:27:
In file included from /home/swd/projs/common/folly-2020.04.13.00-clang/include/folly/io/async/Request.h:23:
In file included from /home/swd/projs/common/folly-2020.04.13.00-clang/include/folly/synchronization/Hazptr.h:20:
/home/swd/projs/common/folly-2020.04.13.00-clang/include/folly/synchronization/HazptrDomain.h:490:34: error: use of overloaded operator '!=' is ambiguous (with operand types 'folly::SingletonThreadLocal<folly::hazptr_priv<atomic>, folly::HazptrTag, folly::detail::DefaultMake<folly::hazptr_priv<atomic> >, folly::HazptrTag>::Accessor::Iterator' and 'folly::SingletonThreadLocal<folly::hazptr_priv<atomic>, folly::HazptrTag, folly::detail::DefaultMake<folly::hazptr_priv<atomic> >, folly::HazptrTag>::Accessor::Iterator')
    for (hazptr_priv<Atom>& priv :
                                 ^
/home/swd/projs/common/folly-2020.04.13.00-clang/include/folly/synchronization/HazptrDomain.h:589:5: note: in instantiation of member function 'folly::hazptr_domain<std::atomic>::relaxed_cleanup' requested here
    relaxed_cleanup(); // calling regular cleanup may self deadlock
    ^
/home/swd/projs/common/folly-2020.04.13.00-clang/include/folly/synchronization/HazptrDomain.h:478:9: note: in instantiation of member function 'folly::hazptr_domain<std::atomic>::try_timed_cleanup' requested here
    if (try_timed_cleanup()) {
        ^
/home/swd/projs/common/folly-2020.04.13.00-clang/include/folly/synchronization/HazptrDomain.h:271:7: note: in instantiation of member function 'folly::hazptr_domain<std::atomic>::check_cleanup_and_reclaim' requested here
      check_cleanup_and_reclaim();
      ^
/home/swd/projs/common/folly-2020.04.13.00-clang/include/folly/synchronization/HazptrDomain.h:673:10: note: in instantiation of member function 'folly::hazptr_domain<std::atomic>::push_retired' requested here
  domain.push_retired(l, check);
         ^
/home/swd/projs/common/folly-2020.04.13.00-clang/include/folly/synchronization/HazptrObj.h:208:5: note: in instantiation of function template specialization 'folly::hazptr_domain_push_retired<atomic>' requested here
    hazptr_domain_push_retired(l, true, domain);
    ^
/home/swd/projs/common/folly-2020.04.13.00-clang/include/folly/synchronization/HazptrObj.h:196:7: note: (skipping 1 context in backtrace; use -ftemplate-backtrace-limit=0 to see all)
      push_to_retired(domain);
      ^
/home/swd/projs/common/folly-2020.04.13.00-clang/include/folly/synchronization/HazptrObjLinked.h:244:11: note: in instantiation of member function 'folly::hazptr_obj<atomic>::push_obj' requested here
    this->push_obj(domain); // defined in hazptr_obj
          ^
/home/swd/projs/common/folly-2020.04.13.00-clang/include/folly/concurrency/UnboundedQueue.h:644:10: note: in instantiation of member function 'folly::hazptr_obj_base_linked<folly::UnboundedQueue<folly::Function<void ()>, false, true, true, 8, 7, atomic>::Segment, atomic, std::__1::default_delete<folly::UnboundedQueue<folly::Function<void ()>, false, true, true, 8, 7, atomic>::Segment> >::retire' requested here
      s->retire(); // defined in hazptr_obj_base_linked
         ^
/home/swd/projs/common/folly-2020.04.13.00-clang/include/folly/concurrency/UnboundedQueue.h:668:5: note: in instantiation of member function 'folly::UnboundedQueue<folly::Function<void ()>, false, true, true, 8, 7, atomic>::reclaimSegment' requested here
    reclaimSegment(h);
    ^
/home/swd/projs/common/folly-2020.04.13.00-clang/include/folly/concurrency/UnboundedQueue.h:262:5: note: in instantiation of member function 'folly::UnboundedQueue<folly::Function<void ()>, false, true, true, 8, 7, atomic>::reclaimRemainingSegments' requested here
    reclaimRemainingSegments();
    ^
/home/swd/projs/common/folly-2020.04.13.00-clang/include/folly/executors/TimedDrivableExecutor.h:32:3: note: in instantiation of member function 'folly::UnboundedQueue<folly::Function<void ()>, false, true, true, 8, 7, atomic>::~UnboundedQueue' requested here
  ~TimedDrivableExecutor() noexcept {
  ^
/home/swd/projs/common/folly-2020.04.13.00-clang/include/folly/detail/Iterators.h:84:8: note: candidate function
  bool operator!=(D const& rhs) const {
       ^
/home/swd/projs/common/folly-2020.04.13.00-clang/include/folly/detail/Iterators.h:80:8: note: candidate function
  bool operator==(D const& rhs) const {
       ^
/home/swd/projs/common/folly-2020.04.13.00-clang/include/folly/detail/Iterators.h:80:8: note: candidate function (with reversed parameter order)
/home/swd/projs/common/folly-2020.04.13.00-clang/include/folly/detail/Iterators.h:105:8: note: candidate function [with D2 = folly::SingletonThreadLocal<folly::hazptr_priv<atomic>, folly::HazptrTag, folly::detail::DefaultMake<folly::hazptr_priv<atomic> >, folly::HazptrTag>::Accessor::Iterator]
  bool operator!=(D2 const& rhs) const {
       ^
/home/swd/projs/common/folly-2020.04.13.00-clang/include/folly/synchronization/HazptrDomain.h:491:10: note: in implicit call to 'operator!=' for iterator of type 'folly::SingletonThreadLocal<folly::hazptr_priv<atomic>, folly::HazptrTag, folly::detail::DefaultMake<folly::hazptr_priv<atomic> >, folly::HazptrTag>::Accessor'
         hazptr_priv_singleton<Atom>::accessAllThreads()) {
         ^~~~~~~~~~~~~~~~~~~~~
/home/swd/projs/common/folly-2020.04.13.00-clang/include/folly/SingletonThreadLocal.h:215:14: note: selected 'begin' function with iterator type 'folly::SingletonThreadLocal<folly::hazptr_priv<atomic>, folly::HazptrTag, folly::detail::DefaultMake<folly::hazptr_priv<atomic> >, folly::HazptrTag>::Accessor::Iterator'
    Iterator begin() const {
             ^
1 error generated.
ninja: build stopped: subcommand failed.

I was able to build a simple coroutine using the prereleased gcc-10 (after some minor edits to folly since coroutines are non-experimental in gcc-10), so this appears to be a clang 10 issue. I did not try with previous versions of clang.

seelabs commented 4 years ago

I was able to modify folly to successfully compile with clang-10 and the clang standard c++ library on linux. Here are the changes I had to make:

  1. Add throw() to memrchr in both portability/String.{h,cpp} (ref: https://github.com/facebook/folly/blob/master/folly/portability/String.cpp#L20)
  2. Comment out the #error about clock_gettime (Ref: https://github.com/facebook/folly/blob/master/folly/portability/Time.cpp#L278)
  3. Change the range based for loop in HazptrDomain to use a regular for loop and call with variables i and e and call i.operator!=(e) explicitly (Ref: https://github.com/facebook/folly/blob/master/folly/synchronization/HazptrDomain.h#L490)

With the above changes I am able to compile a run a simple program that uses folly's coro::Task and coro::AsyncGenerator.

My motivation for this is to experiment with coroutines. While gcc-10 supports coroutines, the compiler often ices. So folly needs to be built with clang and the clang standard library. If anyone else see this message, to make this work the following libraries also need to be compiled with the clang standard library since they are dependencies of folly:

  1. fmt
  2. gflags
  3. glog
  4. boost
Myz commented 4 years ago

You are compiling using -std=c++2a. Could you explain me in more detail how you modified https://github.com/facebook/folly/blob/master/folly/synchronization/HazptrDomain.h#L490 so that it compiles again?

seelabs commented 4 years ago

@Myz Inside the function relaxed_cleanup I made the following change:

    // swd hack
#if 0
    for (hazptr_priv<Atom>& priv :
         hazptr_priv_singleton<Atom>::accessAllThreads()) {
      priv.collect(h, t);
    }
#else
    {
        auto&& swd_temp = hazptr_priv_singleton<Atom>::accessAllThreads();
        for (auto swd_i = swd_temp.begin(), swd_e = swd_temp.end(); swd_i.operator!=(swd_e);++swd_i)
        {
            hazptr_priv<Atom>& priv = *swd_i;
            priv.collect(h, t);
        }
    }
#endif
Myz commented 4 years ago

Thanks a lot, like this I might be able to add those as patches for my .deb build pipeline.

Am Do., 10. Sept. 2020 um 22:39 Uhr schrieb Scott Determan < notifications@github.com>:

@Myz https://github.com/Myz Inside the function relaxed_cleanup I made the following change:

// swd hack

if 0

for (hazptr_priv<Atom>& priv :
     hazptr_priv_singleton<Atom>::accessAllThreads()) {
  priv.collect(h, t);
}

else

{
    auto&& swd_temp = hazptr_priv_singleton<Atom>::accessAllThreads();
    for (auto swd_i = swd_temp.begin(), swd_e = swd_temp.end(); swd_i.operator!=(swd_e);++swd_i)
    {
        hazptr_priv<Atom>& priv = *swd_i;
        priv.collect(h, t);
    }
}

endif

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/facebook/folly/issues/1353#issuecomment-690713706, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAFQPBTFBEVKHUKPMOUFPHDSFE2ORANCNFSM4MIXV5HQ .