constantinpape / z5

Lightweight C++ and Python interface for datasets in zarr and N5 format
MIT License
111 stars 27 forks source link

bad_alloc when running C++ example (writeSubarray, bufView = view) #213

Open bartoszukm opened 1 year ago

bartoszukm commented 1 year ago

Hello,

I just wanted to start using z5 library in my C++ project. I copied a code from example:

#include "json.hpp"
#include "xtensor/xarray.hpp"

// factory functions to create files, groups and datasets
#include "z5/factory.hxx"
// handles for z5 filesystem objects
#include "z5/filesystem/handle.hxx"
// io for xtensor multi-arrays
#include "z5/multiarray/xtensor_access.hxx"
// attribute functionality
#include "z5/attributes.hxx"

int main() {

  // get handle to a File on the filesystem
  z5::filesystem::handle::File f("data.zr");
  // if you wanted to use a different backend, for example AWS, you
  // would need to use this instead:
  // z5::s3::handle::File f;

  // create the file in zarr format
  const bool createAsZarr = true;
  z5::createFile(f, createAsZarr);

  // create a new zarr dataset
  const std::string dsName = "data";
  std::vector<size_t> shape = { 1000, 1000, 1000 };
  std::vector<size_t> chunks = { 100, 100, 100 };
  auto ds = z5::createDataset(f, dsName, "float32", shape, chunks);

  // write array to roi
  z5::types::ShapeType offset1 = { 50, 100, 150 };
  xt::xarray<float>::shape_type shape1 = { 150, 200, 100 };
  xt::xarray<float> array1(shape1, 42.0);
  z5::multiarray::writeSubarray<float>(ds, array1, offset1.begin());

  // read array from roi (values that were not written before are filled with a fill-value)
  z5::types::ShapeType offset2 = { 100, 100, 100 };
  xt::xarray<float>::shape_type shape2 = { 300, 200, 75 };
  xt::xarray<float> array2(shape2);
  z5::multiarray::readSubarray<float>(ds, array2, offset2.begin());

  // get handle for the dataset
  const auto dsHandle = z5::filesystem::handle::Dataset(f, dsName);

  // read and write json attributes
  nlohmann::json attributesIn;
  attributesIn["bar"] = "foo";
  attributesIn["pi"] = 3.141593;
  z5::writeAttributes(dsHandle, attributesIn);

  nlohmann::json attributesOut;
  z5::readAttributes(dsHandle, attributesOut);

  return 0;
}

and unfortunately I get std::bad_alloc and Signal: SIGABRT (Aborted) in a line z5::multiarray::writeSubarray<float>(ds, array1, offset1.begin());.

As I debugged the code, it is a line 366 in xtensor_access.hxx:

// could also implement smart view for this,
// but this would be kind of hard and premature optimization
bufView = view;

I use xtensor-0.24.2, xtensor-blas-0.20.0 and xtl-0.7.4 in my project. The z5 is in version z5-2.0.16. g++ (Ubuntu 10.3.0-1ubuntu1~20.04) 10.3.0.

I tried slightly change a main code, as createAsZarr = false or put more threads in writeSubarray. The outcome remains the same.

I have not linked against the compression codecs, but I believe it is not a problem here.

Could you please help me or give any guidance what is the nature of this problem?