[BUG]: bounds checking problem

0-wiz-0 commented 3 months ago

Required prerequisites

[X] Make sure you've read the documentation. Your issue may be addressed there.
[X] Search the issue tracker and Discussions to verify that this hasn't already been reported. +1 or comment there if it has.
[X] Consider asking first in the Gitter chat room or in a Discussion.

What version (or hash if on master) of pybind11 are you using?

2.13.1

Problem description

When compiling contourpy with bounds checking (-fstack-clash-protection) enabled, I get a compilation error (because contourpy by default builds with -Werror). That's with gcc 12.4.0 on NetBSD-10.99.11/amd64.

I reported this to the contourpy project but they said it's a problem in pybind11.

Here's the compilation output:

[12/14] c++ -Isrc/_contourpy.so.p -Isrc -I../src -I/usr/pkg/include/python3.12 -I/usr/pkg/lib/python3.12/site-packages/pybind11/include -I/usr/pkg/include -I/usr/include -fvisibility=hidden -fdiagnostics-color=always -DNDEBUG -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -Werror -std=c++17 -O3 -O2 -g -fstack-clash-protection -g -fPIC -DCONTOURPY_VERSION=1.2.1 -MD -MQ src/_contourpy.so.p/wra
p.cpp.o -MF src/_contourpy.so.p/wrap.cpp.o.d -o src/_contourpy.so.p/wrap.cpp.o -c ../src/wrap.cpp
FAILED: src/_contourpy.so.p/wrap.cpp.o
c++ -Isrc/_contourpy.so.p -Isrc -I../src -I/usr/pkg/include/python3.12 -I/usr/pkg/lib/python3.12/site-packages/pybind11/include -I/usr/pkg/include -I/usr/include -fvisibility=hidden -fdiagnostics-color=always -DNDEBUG -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -Werror -std=c++17 -O3 -O2 -g -fstack-clash-protection -g -fPIC -DCONTOURPY_VERSION=1.2.1 -MD -MQ src/_contourpy.so.p/wrap.cpp.o
In file included from /usr/include/g++/bits/forward_list.h:38,
                 from /usr/include/g++/forward_list:38,
                 from /scratch/graphics/py-contourpy/work/.buildlink/lib/python3.12/site-packages/pybind11/include/pybind11/detail/../detail/common.h:317,
                 from /scratch/graphics/py-contourpy/work/.buildlink/lib/python3.12/site-packages/pybind11/include/pybind11/detail/../attr.h:13,
                 from /scratch/graphics/py-contourpy/work/.buildlink/lib/python3.12/site-packages/pybind11/include/pybind11/detail/class.h:12,
                 from /scratch/graphics/py-contourpy/work/.buildlink/lib/python3.12/site-packages/pybind11/include/pybind11/pybind11.h:13,
                 from ../src/common.h:4,
                 from ../src/output_array.h:4,
                 from ../src/chunk_local.h:4,
                 from ../src/base.h:13,
                 from ../src/base_impl.h:4,
                 from ../src/wrap.cpp:1:
In static member function 'static _Up* std::__copy_move<_IsMove, true, std::random_access_iterator_tag>::__copy_m(_Tp*, _Tp*, _Up*) [with _Tp = pybind11::detail::type_info* const; _Up = pybind11::detail::type_info*; bool _IsMove = false]',
    inlined from '_OI std::__copy_move_a2(_II, _II, _OI) [with bool _IsMove = false; _II = pybind11::detail::type_info* const*; _OI = pybind11::detail::type_info**]' at /usr/include/g++/bits/stl_algobase.h:501:30,
    inlined from '_OI std::__copy_move_a1(_II, _II, _OI) [with bool _IsMove = false; _II = pybind11::detail::type_info* const*; _OI = pybind11::detail::type_info**]' at /usr/include/g++/bits/stl_algobase.h:528:42,
    inlined from '_OI std::__copy_move_a(_II, _II, _OI) [with bool _IsMove = false; _II = pybind11::detail::type_info* const*; _OI = pybind11::detail::type_info**]' at /usr/include/g++/bits/stl_algobase.h:535:31,
    inlined from '_OI std::copy(_II, _II, _OI) [with _II = pybind11::detail::type_info* const*; _OI = pybind11::detail::type_info**]' at /usr/include/g++/bits/stl_algobase.h:626:7,
    inlined from 'static _ForwardIterator std::__uninitialized_copy<true>::__uninit_copy(_InputIterator, _InputIterator, _ForwardIterator) [with _InputIterator = pybind11::detail::type_info* const*; _ForwardIterator = pybind11::detail::type_info**]' at /usr/include/g++/bits/stl_uninitialized.h:147:27,
    inlined from '_ForwardIterator std::uninitialized_copy(_InputIterator, _InputIterator, _ForwardIterator) [with _InputIterator = pybind11::detail::type_info* const*; _ForwardIterator = pybind11::detail::type_info**]' at /usr/include/g++/bits/stl_uninitialized.h:185:15,
    inlined from '_ForwardIterator std::__uninitialized_copy_a(_InputIterator, _InputIterator, _ForwardIterator, allocator<_Tp>&) [with _InputIterator = pybind11::detail::type_info* const*; _ForwardIterator = pybind11::detail::type_info**; _Tp = pybind11::detail::type_info*]' at /usr/include/g++/bits/stl_uninitialized.h:372:37,
    inlined from 'void std::vector<_Tp, _Alloc>::_M_assign_aux(_ForwardIterator, _ForwardIterator, std::forward_iterator_tag) [with _ForwardIterator = pybind11::detail::type_info* const*; _Tp = pybind11::detail::type_info*; _Alloc = std::allocator<pybind11::detail::type_info*>]' at /usr/include/g++/bits/vector.tcc:339:35,
    inlined from 'std::vector<_Tp, _Alloc>& std::vector<_Tp, _Alloc>::operator=(std::initializer_list<_Tp>) [with _Tp = pybind11::detail::type_info*; _Alloc = std::allocator<pybind11::detail::type_info*>]' at /usr/include/g++/bits/stl_vector.h:787:21,
    inlined from 'pybind11::detail::generic_type::initialize(const pybind11::detail::type_record&)::<lambda(pybind11::detail::internals&)>' at /scratch/graphics/py-contourpy/work/.buildlink/lib/python3.12/site-packages/pybind11/include/pybind11/pybind11.h:1434:75,
    inlined from 'decltype (cb(pybind11::detail::get_internals())) pybind11::detail::with_internals(const F&) [with F = generic_type::initialize(const pybind11::detail::type_record&)::<lambda(internals&)>]' at /scratch/graphics/py-contourpy/work/.buildlink/lib/python3.12/site-packages/pybind11/include/pybind11/detail/../detail/internals.h:626:14,
    inlined from 'void pybind11::detail::generic_type::initialize(const pybind11::detail::type_record&)' at /scratch/graphics/py-contourpy/work/.buildlink/lib/python3.12/site-packages/pybind11/include/pybind11/pybind11.h:1426:23:
/usr/include/g++/bits/stl_algobase.h:434:30: error: 'void* __builtin_memcpy(void*, const void*, long unsigned int)' forming offset 8 is out of the bounds [0, 8] of object '<anonymous>' with type 'pybind11::detail::type_info* const [1]' [-Werror=array-bounds]
  434 |             __builtin_memmove(__result, __first, sizeof(_Tp) * _Num);
      |             ~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/scratch/graphics/py-contourpy/work/.buildlink/lib/python3.12/site-packages/pybind11/include/pybind11/pybind11.h: In member function 'void pybind11::detail::generic_type::initialize(const pybind11::detail::type_record&)':
/scratch/graphics/py-contourpy/work/.buildlink/lib/python3.12/site-packages/pybind11/include/pybind11/pybind11.h:1434:75: note: '<anonymous>' declared here
 1434 |             internals.registered_types_py[(PyTypeObject *) m_ptr] = {tinfo};
      |                                                                           ^
cc1plus: all warnings being treated as errors
ninja: build stopped: subcommand failed.

I tried it also with pybind11 2.12.0, same problem there.

Reproducible example code

See above.

Is this a regression? Put the last known working version here if it is.

Not a regression

futu2 commented 2 months ago

Same problem.

[ 98%] Built target triton-translate
In file included from /nix/store/8hcsbp5h11lk9wwv5b8sw9kyw1g11zwf-gcc-12.4.0/include/c++/12.4.0/bits/stl_uninitialized.h:63,
                 from /nix/store/8hcsbp5h11lk9wwv5b8sw9kyw1g11zwf-gcc-12.4.0/include/c++/12.4.0/memory:65,
                 from /nix/store/pplybsx69vra3sj7l14x8xza4y9m8ndg-openai-triton-llvm-17.0.0-c5dede880d17/include/llvm/Support/Casting.h:20,
                 from /nix/store/pplybsx69vra3sj7l14x8xza4y9m8ndg-openai-triton-llvm-17.0.0-c5dede880d17/include/mlir/Support/LLVM.h:24,
                 from /nix/store/pplybsx69vra3sj7l14x8xza4y9m8ndg-openai-triton-llvm-17.0.0-c5dede880d17/include/mlir/IR/MLIRContext.h:12,
                 from /nix/store/pplybsx69vra3sj7l14x8xza4y9m8ndg-openai-triton-llvm-17.0.0-c5dede880d17/include/mlir/IR/DialectRegistry.h:16,
                 from /nix/store/pplybsx69vra3sj7l14x8xza4y9m8ndg-openai-triton-llvm-17.0.0-c5dede880d17/include/mlir/IR/Dialect.h:16,
                 from /nix/store/pplybsx69vra3sj7l14x8xza4y9m8ndg-openai-triton-llvm-17.0.0-c5dede880d17/include/mlir/IR/OpDefinition.h:22,
                 from /nix/store/pplybsx69vra3sj7l14x8xza4y9m8ndg-openai-triton-llvm-17.0.0-c5dede880d17/include/mlir/IR/Builders.h:12,
                 from /build/source/python/src/triton.cc:1:
In static member function ‘static _Up* std::__copy_move<_IsMove, true, std::random_access_iterator_tag>::__copy_m(_Tp*, _Tp*, _Up*) [with _Tp = pybind11::detail::type_info* const; _Up = pybind11::detail::type_info*; bool _IsMove = false]’,
    inlined from ‘_OI std::__copy_move_a2(_II, _II, _OI) [with bool _IsMove = false; _II = pybind11::detail::type_info* const*; _OI = pybind11::detail::type_info**]’ at /nix/store/8hcsbp5h11lk9wwv5b8sw9kyw1g11zwf-gcc-12.4.0/include/c++/12.4.0/bits/stl_algobase.h:501:30,
    inlined from ‘_OI std::__copy_move_a1(_II, _II, _OI) [with bool _IsMove = false; _II = pybind11::detail::type_info* const*; _OI = pybind11::detail::type_info**]’ at /nix/store/8hcsbp5h11lk9wwv5b8sw9kyw1g11zwf-gcc-12.4.0/include/c++/12.4.0/bits/stl_algobase.h:528:42,
    inlined from ‘_OI std::__copy_move_a(_II, _II, _OI) [with bool _IsMove = false; _II = pybind11::detail::type_info* const*; _OI = pybind11::detail::type_info**]’ at /nix/store/8hcsbp5h11lk9wwv5b8sw9kyw1g11zwf-gcc-12.4.0/include/c++/12.4.0/bits/stl_algobase.h:535:31,
    inlined from ‘_OI std::copy(_II, _II, _OI) [with _II = pybind11::detail::type_info* const*; _OI = pybind11::detail::type_info**]’ at /nix/store/8hcsbp5h11lk9wwv5b8sw9kyw1g11zwf-gcc-12.4.0/include/c++/12.4.0/bits/stl_algobase.h:626:7,
    inlined from ‘static _ForwardIterator std::__uninitialized_copy<true>::__uninit_copy(_InputIterator, _InputIterator, _ForwardIterator) [with _InputIterator = pybind11::detail::type_info* const*; _ForwardIterator = pybind11::detail::type_info**]’ at /nix/store/8hcsbp5h11lk9wwv5b8sw9kyw1g11zwf-gcc-12.4.0/include/c++/12.4.0/bits/stl_uninitialized.h:147:27,
    inlined from ‘_ForwardIterator std::uninitialized_copy(_InputIterator, _InputIterator, _ForwardIterator) [with _InputIterator = pybind11::detail::type_info* const*; _ForwardIterator = pybind11::detail::type_info**]’ at /nix/store/8hcsbp5h11lk9wwv5b8sw9kyw1g11zwf-gcc-12.4.0/include/c++/12.4.0/bits/stl_uninitialized.h:185:15,
    inlined from ‘_ForwardIterator std::__uninitialized_copy_a(_InputIterator, _InputIterator, _ForwardIterator, allocator<_Tp>&) [with _InputIterator = pybind11::detail::type_info* const*; _ForwardIterator = pybind11::detail::type_info**; _Tp = pybind11::detail::type_info*]’ at /nix/store/8hcsbp5h11lk9wwv5b8sw9kyw1g11zwf-gcc-12.4.0/include/c++/12.4.0/bits/stl_uninitialized.h:372:37,
    inlined from ‘void std::vector<_Tp, _Alloc>::_M_assign_aux(_ForwardIterator, _ForwardIterator, std::forward_iterator_tag) [with _ForwardIterator = pybind11::detail::type_info* const*; _Tp = pybind11::detail::type_info*; _Alloc = std::allocator<pybind11::detail::type_info*>]’ at /nix/store/8hcsbp5h11lk9wwv5b8sw9kyw1g11zwf-gcc-12.4.0/include/c++/12.4.0/bits/vector.tcc:339:35,
    inlined from ‘std::vector<_Tp, _Alloc>& std::vector<_Tp, _Alloc>::operator=(std::initializer_list<_Tp>) [with _Tp = pybind11::detail::type_info*; _Alloc = std::allocator<pybind11::detail::type_info*>]’ at /nix/store/8hcsbp5h11lk9wwv5b8sw9kyw1g11zwf-gcc-12.4.0/include/c++/12.4.0/bits/stl_vector.h:787:21,
    inlined from ‘void pybind11::detail::generic_type::initialize(const pybind11::detail::type_record&)’ at /nix/store/dkxf5a0ysb7cv7jp4acsyqyh6yp38xlw-python3.12-pybind11-2.12.0/include/pybind11/pybind11.h:1408:71:
/nix/store/8hcsbp5h11lk9wwv5b8sw9kyw1g11zwf-gcc-12.4.0/include/c++/12.4.0/bits/stl_algobase.h:434:30: error: ‘void* __builtin_memcpy(void*, const void*, long unsigned int)’ reading between 9 and 9223372036854775807 bytes from a region of size 8 [-Werror=stringop-overread]
  434 |             __builtin_memmove(__result, __first, sizeof(_Tp) * _Num);
      |             ~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /nix/store/dkxf5a0ysb7cv7jp4acsyqyh6yp38xlw-python3.12-pybind11-2.12.0/include/pybind11/functional.h:12,
                 from /build/source/python/src/triton.cc:48:
/nix/store/dkxf5a0ysb7cv7jp4acsyqyh6yp38xlw-python3.12-pybind11-2.12.0/include/pybind11/pybind11.h: In member function ‘void pybind11::detail::generic_type::initialize(const pybind11::detail::type_record&)’:
/nix/store/dkxf5a0ysb7cv7jp4acsyqyh6yp38xlw-python3.12-pybind11-2.12.0/include/pybind11/pybind11.h:1408:71: note: at offset [0, 7] into source object ‘<anonymous>’ of size 8
 1408 |         internals.registered_types_py[(PyTypeObject *) m_ptr] = {tinfo};
      |                                                                       ^
At global scope:
cc1plus: note: unrecognized command-line option ‘-Wno-covered-switch-default’ may have been intended to silence earlier diagnostics
cc1plus: all warnings being treated as errors
make[2]: *** [CMakeFiles/triton.dir/build.make:93: CMakeFiles/triton.dir/python/src/triton.cc.o] Error 1
make[2]: Leaving directory '/build/source/python/build/cmake.linux-x86_64-cpython-3.12'
make[1]: *** [CMakeFiles/Makefile2:1778: CMakeFiles/triton.dir/all] Error 2
make[1]: Leaving directory '/build/source/python/build/cmake.linux-x86_64-cpython-3.12'
make: *** [Makefile:149: all] Error 2

smuzaffar commented 2 months ago

We also get this arry-bounds[a] error when building onnxruntime and contourpy

[a]

  .../gcc/12.4.1-0eebe235f0134c8f4787c4c07b632d10/include/c++/12.4.1/bits/stl_algobase.h:434:30: error: 'void* __builtin_memcpy(void*, const void*, long unsigned int)' forming offset 8 is out of the bounds [0, 8] of object '<anonymous>' with type 'pybind11::detail::type_info* const [1]' [-Werror=array-bounds]
    434 |             __builtin_memmove(__result, __first, sizeof(_Tp) * _Num);

jmeyers314 commented 2 months ago

I see the same error at https://dev.azure.com/conda-forge/feedstock-builds/_build/results?buildId=981519&view=results Seems to be limited to linux with py3.11 and py3.12. (MacOS x86 & ARM with python 3.9 through 3.12 are fine, as is linux with py3.9 and py3.10).

tkittel commented 2 months ago

Ouch, we are also bitten by this. Does anyone know of a workaround until this is fixed (i.e. a compiler flag?).

jmeyers314 commented 2 months ago

Adding -Wno-stringop-overread fixed (well, worked around) the error for my build.

I see both -Werror=array-bounds and -Werror=stringop-overread in the compile outputs above though, so maybe something from -Warray-bounds would also be required in some cases?

tkittel commented 2 months ago

Thanks @jmeyers314 ! -Wno-array-bounds and -Wno-stringop-overread were both needed as a workaround in my case.

mytbk commented 1 month ago

This is a bug in GCC 12 with GCC 12.4.0 libstdc++. Yeah, I also first found this by compiling pybind. https://gcc.gnu.org/bugzilla/show_bug.cgi?id=115824

BobbyRBruce commented 1 month ago

I created a PR which fixes this issue. It's a bit of a work around but given the issue is with gcc 12, I don't see a better way. https://github.com/pybind/pybind11/pull/5355

pybind / pybind11