search

RcppCore / RcppParallel

High-level functions for parallel programming with Rcpp
http://rcppcore.github.io/RcppParallel/
171 stars 58 forks source link

TBB autodetection was switched off #216

Open Enchufa2 opened 5 months ago

Enchufa2 commented 5 months ago

I didn't detect this because I had the old workaround in place, but after #178, I suppose that 640468165c05611ba81d76b757fd3f195a93b285 broke this?

kevinushey commented 5 months ago

Most likely a bug in the updated configure script? Can you confirm where libtbb is installed / available on the system you're testing with?

Enchufa2 commented 5 months ago

We have:

``` $ rpm -ql tbb-devel /usr/include/oneapi /usr/include/oneapi/tbb /usr/include/oneapi/tbb.h /usr/include/oneapi/tbb/blocked_range.h /usr/include/oneapi/tbb/blocked_range2d.h /usr/include/oneapi/tbb/blocked_range3d.h /usr/include/oneapi/tbb/blocked_rangeNd.h /usr/include/oneapi/tbb/cache_aligned_allocator.h /usr/include/oneapi/tbb/collaborative_call_once.h /usr/include/oneapi/tbb/combinable.h /usr/include/oneapi/tbb/concurrent_hash_map.h /usr/include/oneapi/tbb/concurrent_lru_cache.h /usr/include/oneapi/tbb/concurrent_map.h /usr/include/oneapi/tbb/concurrent_priority_queue.h /usr/include/oneapi/tbb/concurrent_queue.h /usr/include/oneapi/tbb/concurrent_set.h /usr/include/oneapi/tbb/concurrent_unordered_map.h /usr/include/oneapi/tbb/concurrent_unordered_set.h /usr/include/oneapi/tbb/concurrent_vector.h /usr/include/oneapi/tbb/detail /usr/include/oneapi/tbb/detail/_aggregator.h /usr/include/oneapi/tbb/detail/_aligned_space.h /usr/include/oneapi/tbb/detail/_allocator_traits.h /usr/include/oneapi/tbb/detail/_assert.h /usr/include/oneapi/tbb/detail/_attach.h /usr/include/oneapi/tbb/detail/_concurrent_queue_base.h /usr/include/oneapi/tbb/detail/_concurrent_skip_list.h /usr/include/oneapi/tbb/detail/_concurrent_unordered_base.h /usr/include/oneapi/tbb/detail/_config.h /usr/include/oneapi/tbb/detail/_containers_helpers.h /usr/include/oneapi/tbb/detail/_exception.h /usr/include/oneapi/tbb/detail/_export.h /usr/include/oneapi/tbb/detail/_flow_graph_body_impl.h /usr/include/oneapi/tbb/detail/_flow_graph_cache_impl.h /usr/include/oneapi/tbb/detail/_flow_graph_impl.h /usr/include/oneapi/tbb/detail/_flow_graph_indexer_impl.h /usr/include/oneapi/tbb/detail/_flow_graph_item_buffer_impl.h /usr/include/oneapi/tbb/detail/_flow_graph_join_impl.h /usr/include/oneapi/tbb/detail/_flow_graph_node_impl.h /usr/include/oneapi/tbb/detail/_flow_graph_node_set_impl.h /usr/include/oneapi/tbb/detail/_flow_graph_nodes_deduction.h /usr/include/oneapi/tbb/detail/_flow_graph_tagged_buffer_impl.h /usr/include/oneapi/tbb/detail/_flow_graph_trace_impl.h /usr/include/oneapi/tbb/detail/_flow_graph_types_impl.h /usr/include/oneapi/tbb/detail/_hash_compare.h /usr/include/oneapi/tbb/detail/_intrusive_list_node.h /usr/include/oneapi/tbb/detail/_machine.h /usr/include/oneapi/tbb/detail/_mutex_common.h /usr/include/oneapi/tbb/detail/_namespace_injection.h /usr/include/oneapi/tbb/detail/_node_handle.h /usr/include/oneapi/tbb/detail/_pipeline_filters.h /usr/include/oneapi/tbb/detail/_pipeline_filters_deduction.h /usr/include/oneapi/tbb/detail/_range_common.h /usr/include/oneapi/tbb/detail/_rtm_mutex.h /usr/include/oneapi/tbb/detail/_rtm_rw_mutex.h /usr/include/oneapi/tbb/detail/_scoped_lock.h /usr/include/oneapi/tbb/detail/_segment_table.h /usr/include/oneapi/tbb/detail/_small_object_pool.h /usr/include/oneapi/tbb/detail/_string_resource.h /usr/include/oneapi/tbb/detail/_task.h /usr/include/oneapi/tbb/detail/_task_handle.h /usr/include/oneapi/tbb/detail/_template_helpers.h /usr/include/oneapi/tbb/detail/_utils.h /usr/include/oneapi/tbb/detail/_waitable_atomic.h /usr/include/oneapi/tbb/enumerable_thread_specific.h /usr/include/oneapi/tbb/flow_graph.h /usr/include/oneapi/tbb/flow_graph_abstractions.h /usr/include/oneapi/tbb/global_control.h /usr/include/oneapi/tbb/info.h /usr/include/oneapi/tbb/memory_pool.h /usr/include/oneapi/tbb/mutex.h /usr/include/oneapi/tbb/null_mutex.h /usr/include/oneapi/tbb/null_rw_mutex.h /usr/include/oneapi/tbb/parallel_for.h /usr/include/oneapi/tbb/parallel_for_each.h /usr/include/oneapi/tbb/parallel_invoke.h /usr/include/oneapi/tbb/parallel_pipeline.h /usr/include/oneapi/tbb/parallel_reduce.h /usr/include/oneapi/tbb/parallel_scan.h /usr/include/oneapi/tbb/parallel_sort.h /usr/include/oneapi/tbb/partitioner.h /usr/include/oneapi/tbb/profiling.h /usr/include/oneapi/tbb/queuing_mutex.h /usr/include/oneapi/tbb/queuing_rw_mutex.h /usr/include/oneapi/tbb/rw_mutex.h /usr/include/oneapi/tbb/scalable_allocator.h /usr/include/oneapi/tbb/spin_mutex.h /usr/include/oneapi/tbb/spin_rw_mutex.h /usr/include/oneapi/tbb/task.h /usr/include/oneapi/tbb/task_arena.h /usr/include/oneapi/tbb/task_group.h /usr/include/oneapi/tbb/task_scheduler_observer.h /usr/include/oneapi/tbb/tbb_allocator.h /usr/include/oneapi/tbb/tbbmalloc_proxy.h /usr/include/oneapi/tbb/tick_count.h /usr/include/oneapi/tbb/version.h /usr/include/tbb /usr/include/tbb/blocked_range.h /usr/include/tbb/blocked_range2d.h /usr/include/tbb/blocked_range3d.h /usr/include/tbb/blocked_rangeNd.h /usr/include/tbb/cache_aligned_allocator.h /usr/include/tbb/collaborative_call_once.h /usr/include/tbb/combinable.h /usr/include/tbb/concurrent_hash_map.h /usr/include/tbb/concurrent_lru_cache.h /usr/include/tbb/concurrent_map.h /usr/include/tbb/concurrent_priority_queue.h /usr/include/tbb/concurrent_queue.h /usr/include/tbb/concurrent_set.h /usr/include/tbb/concurrent_unordered_map.h /usr/include/tbb/concurrent_unordered_set.h /usr/include/tbb/concurrent_vector.h /usr/include/tbb/enumerable_thread_specific.h /usr/include/tbb/flow_graph.h /usr/include/tbb/flow_graph_abstractions.h /usr/include/tbb/global_control.h /usr/include/tbb/info.h /usr/include/tbb/memory_pool.h /usr/include/tbb/mutex.h /usr/include/tbb/null_mutex.h /usr/include/tbb/null_rw_mutex.h /usr/include/tbb/parallel_for.h /usr/include/tbb/parallel_for_each.h /usr/include/tbb/parallel_invoke.h /usr/include/tbb/parallel_pipeline.h /usr/include/tbb/parallel_reduce.h /usr/include/tbb/parallel_scan.h /usr/include/tbb/parallel_sort.h /usr/include/tbb/partitioner.h /usr/include/tbb/profiling.h /usr/include/tbb/queuing_mutex.h /usr/include/tbb/queuing_rw_mutex.h /usr/include/tbb/rw_mutex.h /usr/include/tbb/scalable_allocator.h /usr/include/tbb/spin_mutex.h /usr/include/tbb/spin_rw_mutex.h /usr/include/tbb/task.h /usr/include/tbb/task_arena.h /usr/include/tbb/task_group.h /usr/include/tbb/task_scheduler_observer.h /usr/include/tbb/tbb.h /usr/include/tbb/tbb_allocator.h /usr/include/tbb/tbbmalloc_proxy.h /usr/include/tbb/tick_count.h /usr/include/tbb/version.h /usr/lib64/cmake/TBB /usr/lib64/cmake/TBB/TBBConfig.cmake /usr/lib64/cmake/TBB/TBBConfigVersion.cmake /usr/lib64/cmake/TBB/TBBTargets-relwithdebinfo.cmake /usr/lib64/cmake/TBB/TBBTargets.cmake /usr/lib64/libirml.so /usr/lib64/libtbb.so /usr/lib64/libtbbbind_2_5.so /usr/lib64/libtbbmalloc.so /usr/lib64/libtbbmalloc_proxy.so /usr/lib64/pkgconfig/tbb.pc /usr/lib64/pkgconfig/tbbmalloc.pc /usr/lib64/pkgconfig/tbbmalloc_proxy.pc /usr/share/doc/tbb-devel /usr/share/doc/tbb-devel/README.md ```
Enchufa2 commented 5 months ago

Well, the issue is that autodetection was disabled: https://github.com/RcppCore/RcppParallel/commit/d4315220e29f3e48cdc993f9c6e6cfbc605c88a4. Do you remember the reason?

Enchufa2 commented 5 months ago

And apart from this, I think that the issue with stan-dev/rstan/issues/1041 can be solved here much easily. The issue is that this part:

https://github.com/RcppCore/RcppParallel/blob/8ace3198c420a3d63e7c0cb06795ea8e17b304c1/R/tbb.R#L61-L73

~does not take into account if the system tbb is used via autodetection~. If this chunk adds the -DTBB_INTERFACE_NEW flag in this case, then rstan and family will compile just fine if I'm not mistaken.

EDIT: Sorry, I misread that: the TBB_INC variable is defined in the package namespace when autodetection is enabled. So it should work... The only point is to reenable autodetection.

kevinushey commented 4 months ago

Well, the issue is that autodetection was disabled: https://github.com/RcppCore/RcppParallel/commit/d4315220e29f3e48cdc993f9c6e6cfbc605c88a4. Do you remember the reason?

I do not -- it would've been helpful if I had actually documented why somewhere. 😞 My guess is that this was somehow CRAN-oriented; e.g. I wanted to make sure we always used the bundled TBB on CRAN, but even if that's true there's probably a better way to handle that.