Issue with `mi.MemoryStream`

[dvicini]

Something appears to go wrong when writing bytes into the mi.MemoryStream in Python. I have some code that uses the memory stream to load EXR files. However, it appears that writing data of type bytes to the MemoryStream is somehow not quite working. Here is what I have:

import mitsuba as mi
import drjit as dr
import numpy as np

mi.set_variant('llvm_ad_rgb')

filename = 'image.exr'
mi.Bitmap(np.random.rand(128,128,3).astype(np.float32)).write(filename)

stream = mi.MemoryStream()
with open(filename, 'rb') as f:
  buffer = f.read()
  print('N loaded bytes:', len(buffer))
  stream.write(buffer)

stream.seek(0)
print(stream)
mi.Bitmap(stream)

This prints

N loaded bytes: 197029
MemoryStream[
  host_byte_order = little-endian,
  byte_order = little-endian,
  can_read = 1,
  can_write = 1,
  owns_buffer = 1,
  capacity = 512,
  pos = 0,
  size = 5
]

Traceback (most recent call last):
  File ".../mitsuba3/build/exr_loading.py", line 19, in <module>
    mi.Bitmap(stream)
RuntimeError: Cannot read image file "MemoryStream[
  host_byte_order = little-endian,
  byte_order = little-endian,
  can_read = 1,
  can_write = 1,
  owns_buffer = 1,
  capacity = 512,
  pos = 0,
  size = 5
]". The file format version number's flag field contains unrecognized flags.

As you can see, somehow only the first 5 bytes get written. Do I need to pass something else to memory stream rather than Python bytes?

The code runs fine using the last pre-nanobind version

[dvicini]

I think I found the issue. The binding:

.def("write", [](Stream &s, nb::bytes b) {
    std::string data(b.c_str());
    s.write(data.c_str(), data.size());
}, D(Stream, write))

should be

.def("write", [](Stream &s, nb::bytes b) {
    s.write(b.c_str(), b.size());
}, D(Stream, write))

Otherwise the string conversion goes wrong, due to the way null-termination characters are handled. In PyBind11, the py::bytes type had a specialized conversion to get a std::string, but directly converting b.c_str() to string will stop at the termination character

[rtabbara]

Yep, you're right! I just discovered this too and put up a PR #1255