[BUG]: `pybind11::format_descriptor<float16_t>::format()` throws `RuntimeError: NumPy type info missing for Dh` but should return `e`

dastrobu commented 6 months ago

Required prerequisites

[X] Make sure you've read the documentation. Your issue may be addressed there.
[X] Search the issue tracker and Discussions to verify that this hasn't already been reported. +1 or comment there if it has.
[X] Consider asking first in the Gitter chat room or in a Discussion.

What version (or hash if on master) of pybind11 are you using?

2.11.1

Problem description

when constructing a buffer format for a dtype float16 array, I get:

RuntimeError: NumPy type info missing for Dh. Here is a full function to reproduce:

std::string float16_format() {
  return pybind11::format_descriptor<float16_t>::format();
}

float16_t is defined as typedef __fp16 float16_t in arm_fp16.h running on MacOS.

According to the python buffer protocol specs float16 should be represented as 'e'.

Numpy and tensor flow are following this convention:

>>> memoryview(np.ones(3, dtype=np.float16)).format
'e'
>>> memoryview(tf.ones(3, dtype=tf.float16)).format
'e'

Reproducible example code

std::string float16_format() {
  return pybind11::format_descriptor<float16_t>::format();
}

Is this a regression? Put the last known working version here if it is.

Not a regression

dastrobu commented 6 months ago

relates to #1908

swillisart commented 5 months ago

Hello! Yes, it's true half floats are not currently available. You can see extra information here: https://github.com/pybind/pybind11/discussions/4627 and a workaround from the issue here: https://github.com/pybind/pybind11/issues/1776

None of the core devs have responded to the discussion but I'm still very interested in bringing support of the half-float type from OpenEXR / Imath directly. The question is how they want to proceed with it.

pybind / pybind11