pybind / pybind11

Seamless operability between C++11 and Python
https://pybind11.readthedocs.io/
Other
15.58k stars 2.09k forks source link

[smart_holder] Make smart holder type caster of `unique_ptr` accept `automatic_reference` #4775

Closed wangxf123456 closed 1 year ago

wangxf123456 commented 1 year ago

Description

When calling a Python callback function that consumes std::unique_ptr in C++, we see RuntimeError: Invalid return_value_policy for unique_ptr. This is because when calling Python functions manually from C++ , we use return_value_policy::automatic_reference for function arguments, and the type caster of std::unique_ptr rejects this return value policy.

Pybind11 always override return_value_policy::automatic and return_value_policy::automatic_reference with return_value_policy::move for std::unique_ptr: https://github.com/pybind/pybind11/blob/b6444460eeddc2965ab1a49c6c50c83073779489/include/pybind11/cast.h#L1138, so return_value_policy::automatic and return_value_policy::automatic_reference does not make any difference for std::unique_ptr. We should make the type caster accept both.

rwgk commented 1 year ago

Ignoring a test_iostream flake and Clang Dev failures fixed by PR #4767 (currently only on master).