search

pybind / pybind11

Seamless operability between C++11 and Python
https://pybind11.readthedocs.io/
Other
15.09k stars 2.05k forks source link

[BUG]: Compilation failure with GCC 10.2 on pybind11 2.13.0 (typing.h, line 209) #5201

Closed zhqrbitee closed 5 days ago

zhqrbitee commented 6 days ago

Required prerequisites

What version (or hash if on master) of pybind11 are you using?

2.13.0

Problem description

Environment:

Compiler: GCC 10.2 C++ Language Dialect: C++20 (std=c++20)

Error Message:

In file included from my_venv/include/pybind11/pybind11.h:19,
                 from my_venv/include/pybind11/numpy.h:12
my_venv/include/pybind11/typing.h:219:49: error: class template argument deduction failed:
  219 | struct handle_type_name<typing::Literal<Literals...>> {
      |                                                 ^~~
compilation terminated due to -Wfatal-errors.

It seems the error just comes from including pybind11 headers and it comes after upgrading to 2.13. 

#if defined(__cpp_nontype_template_parameter_class)
template <typing::StringLiteral... Literals>
struct handle_type_name<typing::Literal<Literals...>> {
    static constexpr auto name = const_name("Literal[")
                                 + pybind11::detail::concat(const_name(Literals.name)...)
                                 + const_name("]");
};
template <typing::StringLiteral StrLit>
struct handle_type_name<typing::TypeVar<StrLit>> {
    static constexpr auto name = const_name(StrLit.name);
};
#endif

I see it tries to use the macro __cpp_nontype_template_parameter_class to enable some template codes. However not sure why the macro check is success, but GCC 10.2 fails to compile the codes.

Reproducible example code

No response

Is this a regression? Put the last known working version here if it is.

2.12.0

henryiii commented 6 days ago

class template argument deduction failed

I assume it's trying to deduce something which means it's failing on a usage of it (and we test on GCC 10.2.1), so it's probably a regression for some usage. Do you know where this template is being used in your code?

zhqrbitee commented 6 days ago

Hi @henryiii ,

Thanks for your reply. This is something looks very weird to me. Based on my observation, as long as I include the pybind11/pybind11.h, it produces above compile failure.

I can reproduce using the simple example from the docs:

#include "pybind11/pybind11.h"

int add(int i, int j) {
    return i + j;
}

PYBIND11_MODULE(example, m) {
    m.doc() = "pybind11 example plugin"; // optional module docstring

    m.def("add", &add, "A function that adds two numbers");
}

and compile with the command:

// $CXX point our GCC 10.2 bin
$CXX -O3 -Wall -shared -std=c++20 -fPIC $(python3 -m pybind11 --includes) example.cpp -o example.cpython-39m-x86_64-linux-gnu.so

It gives a bit more message this case (maybe because I didn't use -Wfatal-errors), but I still get no clue:

In file included from my__venv/lib/python3.9/site-packages/pybind11/include/pybind11/pybind11.h:19,
                 from example.cpp:1:
my__venv/lib/python3.9/site-packages/pybind11/include/pybind11/typing.h:219:49: error: class template argument deduction failed:
  219 | struct handle_type_name<typing::Literal<Literals...>> {
      |                                                 ^~~
my__venv/lib/python3.9/site-packages/pybind11/include/pybind11/typing.h:219:49: error: no matching function for call to 'StringLiteral(StringLiteral<...auto...>)'
my__venv/lib/python3.9/site-packages/pybind11/include/pybind11/typing.h:104:15: note: candidate: 'template<long unsigned int N> StringLiteral(const char (&)[N])-> pybind11::typing::StringLiteral<N>'
  104 |     constexpr StringLiteral(const char (&str)[N]) { std::copy_n(str, N, name); }
      |               ^~~~~~~~~~~~~
my__venv/lib/python3.9/site-packages/pybind11/include/pybind11/typing.h:104:15: note:   template argument deduction/substitution failed:
my__venv/lib/python3.9/site-packages/pybind11/include/pybind11/typing.h:219:49: note:   mismatched types 'const char [N]' and 'StringLiteral<...auto...>'
  219 | struct handle_type_name<typing::Literal<Literals...>> {
      |                                                 ^~~
my__venv/lib/python3.9/site-packages/pybind11/include/pybind11/typing.h:103:8: note: candidate: 'template<long unsigned int N> StringLiteral(pybind11::typing::StringLiteral<N>)-> pybind11::typing::StringLiteral<N>'
  103 | struct StringLiteral {
      |        ^~~~~~~~~~~~~
my__venv/lib/python3.9/site-packages/pybind11/include/pybind11/typing.h:103:8: note:   template argument deduction/substitution failed:
my__venv/lib/python3.9/site-packages/pybind11/include/pybind11/typing.h:219:49: note:   mismatched types 'pybind11::typing::StringLiteral<N>' and 'StringLiteral<...auto...>'
  219 | struct handle_type_name<typing::Literal<Literals...>> {
      |                                                 ^~~
my__venv/lib/python3.9/site-packages/pybind11/include/pybind11/typing.h:219:52: error: template argument 1 is invalid
  219 | struct handle_type_name<typing::Literal<Literals...>> {
      |                                                    ^~
my__venv/lib/python3.9/site-packages/pybind11/include/pybind11/typing.h:225:41: error: class template argument deduction failed:
  225 | struct handle_type_name<typing::TypeVar<StrLit>> {
      |                                         ^~~~~~
my__venv/lib/python3.9/site-packages/pybind11/include/pybind11/typing.h:225:41: error: no matching function for call to 'StringLiteral(StringLiteral<...auto...>)'
my__venv/lib/python3.9/site-packages/pybind11/include/pybind11/typing.h:104:15: note: candidate: 'template<long unsigned int N> StringLiteral(const char (&)[N])-> pybind11::typing::StringLiteral<N>'
  104 |     constexpr StringLiteral(const char (&str)[N]) { std::copy_n(str, N, name); }
      |               ^~~~~~~~~~~~~
my__venv/lib/python3.9/site-packages/pybind11/include/pybind11/typing.h:104:15: note:   template argument deduction/substitution failed:
my__venv/lib/python3.9/site-packages/pybind11/include/pybind11/typing.h:225:41: note:   mismatched types 'const char [N]' and 'StringLiteral<...auto...>'
  225 | struct handle_type_name<typing::TypeVar<StrLit>> {
      |                                         ^~~~~~
my__venv/lib/python3.9/site-packages/pybind11/include/pybind11/typing.h:103:8: note: candidate: 'template<long unsigned int N> StringLiteral(pybind11::typing::StringLiteral<N>)-> pybind11::typing::StringLiteral<N>'
  103 | struct StringLiteral {
      |        ^~~~~~~~~~~~~
my__venv/lib/python3.9/site-packages/pybind11/include/pybind11/typing.h:103:8: note:   template argument deduction/substitution failed:
my__venv/lib/python3.9/site-packages/pybind11/include/pybind11/typing.h:225:41: note:   mismatched types 'pybind11::typing::StringLiteral<N>' and 'StringLiteral<...auto...>'
  225 | struct handle_type_name<typing::TypeVar<StrLit>> {
      |                                         ^~~~~~
my__venv/lib/python3.9/site-packages/pybind11/include/pybind11/typing.h:225:47: error: template argument 1 is invalid
  225 | struct handle_type_name<typing::TypeVar<StrLit>> {
      |                                               ^~

BTW, we have another CI that run in macOS with Apple-clang and that seems OK to compile.

zhqrbitee commented 6 days ago

After a few more tries. the error seems only happen when I enable -std=c++20 (this is the standard we use in our environment), if I low down it to -std=c++17, it can be compiled successful, e.g.

// $CXX point our GCC 10.2 bin
$CXX -O3 -Wall -shared -std=c++17 -fPIC $(python3 -m pybind11 --includes) example.cpp -o example.cpython-39m-x86_64-linux-gnu.so
henryiii commented 6 days ago

We might be missing a c++20 test on this compiler.

rwgk commented 6 days ago

It seems very likely to me that you're stumbling over a bug in that old GCC release. We're testing with C++20 using a variety of more modern C++ compilers (clang, gcc, msvc).

A simple, pragmatic workaround would be to add to the ifdef here:

https://github.com/pybind/pybind11/blob/57287b578d58af016cd571f6eacee19300d37f2d/include/pybind11/typing.h#L225

... && !defined(...conditions for GCC 10...)

~But that wouldn't be covered by our GitHub Actions, which brings me to the question:~

~What platform do you have, and how exactly did you get your GCC 10? (Could we reproduce that with a GHA job, with a reasonable effort?)~

@InvincibleRMC for awareness (and ideas).

rwgk commented 6 days ago

Oops, sorry, I just realized I mixed up clang 10 vs gcc 10 in my mind: we actually do have gcc 10 testing, e.g. here:

https://github.com/pybind/pybind11/actions/runs/9687720852/job/26732870800

It should be super easy to add gcc 10 C++20 testing along with the ifdef workaround. (Unless we're stumbling over other (presumed) bugs in the compiler.)

zhqrbitee commented 6 days ago

Hi @rwgk @henryiii ,

Thanks very much for your quick response. I forked a repo on my side and tried different version of gcc: https://github.com/zhqrbitee/pybind11/actions/runs/9692860216/job/26747094324.

I short, GCC 9, GCC 10.1, 10.2 have this issue with c++20. GCC 10.3 (and above) seems to have fixed it.

InvincibleRMC commented 5 days ago

@rwgk Do you want me to implement the suggest changes or are you on it?

rwgk commented 5 days ago

@rwgk Do you want me to implement the suggest changes or are you on it?

It'd be helpful if you could send the PR. Thanks!