Single precision float treated as doubles, np.float32 incompatible?

[aldanor]

(on 2.4.3) Looks like there's a mix of a few issues in the example below:

• First, least precision floats would be picked if you don't carefully order your overloads (as @ax3l mentioned in #1512), sounds like that's something that may be done automatically
• There's no builtin float32, so both floats and doubles show up as 'float' on Python side; this would have been fine unless...
• ... passing np.float32() actually worked. You may expect it would have worked but because it doesn't inherit from builtin.float it won't. Is it worth special-casing scalar float32 case?

I do sort of understand where this is coming from, but someone who doesn't know py11 internals may assume that "well, np.float32[:] arrays work, and np.float64[:] arrays work, and I can pass np.float64 and builtin.float for double-type arguments, so np.float32 should work as well?", but then it gets converted to a different type altogether.

#include <pybind11/pybind11.h>

namespace py = pybind11;

auto f_double(double x) { return "double"; }
auto f_float(float x) { return "float"; }
auto f_int(int x) { return "int"; }

PYBIND11_MODULE(py11, m) {
    // double/float: these two overloads have to be ordered this way... (see #1512)
    m.def("f", &f_double, py::arg("x"));
    m.def("f", &f_float, py::arg("x"));
    // int
    m.def("f", &f_int, py::arg("x"));
}

Let's give this a spin:

>>> f?
Overloaded function.
1. f(x: float) -> str  # <-- ?
2. f(x: float) -> str
3. f(x: int) -> str

>>> f(1)
'int'  # ok

>>> f(1.)
'double'  # note: would have been float if two overloads were swapped

>>> f(np.float64(1.))
'double'  # so far so good

>>> f(np.float32(1.))
<ipython-input-7-4d9dada8d141>:1: DeprecationWarning: an integer is required (got type numpy.float32).  Implicit conversion to integers using __int__ is deprecated, and may be removed in a future version of Python.
  f(np.float32(1.))
'int'  # not so good

[ax3l]

Just to complete the issue's documentation: what versions of Python and Numpy are you using? In principle, numpy scalars are quite broken/very complicated in Python2 as well as prior to Numpy 1.15.

Nevertheless, for numpy scalars such as np.float32 you have to implement py::buffer (or py::array) overloads. Those types are fundamentally arrays/buffers with 1 element in size and 0D dimensionality. https://github.com/openPMD/openPMD-api/blob/0.10.3-alpha/src/binding/python/RecordComponent.cpp#L438-L530 https://github.com/openPMD/openPMD-api/blob/0.10.3-alpha/src/binding/python/Attributable.cpp#L266-L315

[aldanor]

I've already fixed it myself, was just lazy to push a PR here :) If there's any interest, I can push it.

No, you don't need to implement buffer APIs. NumPy provides C API for extracting scalars out of PyObject directly.

So, something along the lines of py::numpy_scalar<int64_t> or py::numpy_scalar<float> which will be super strict and only accepts numpy scalars of the types requested.

[aldanor]

@ax3l Ok, pushed a PR - #2060