Support for array_view/span-like arrays?

[Weeena]

If I have a class that holds an array represented as a pointer and its length like e.g.

template <typename T>
struct PodArray
{
    T* parray_;
    std::size_t size_;
};

PONDER_TYPE(PodArray<uint8_t>)

then I would like to be able to declare an array property for this class e.g. in the following way

    ponder::Class::declare<PodArray<uint8_t>>()
        .property("Array", &PodArray::parray_, &PodArray::size_);

If I get it right then the resulting array property would (in the first place) have to be non-dynamic since in your current implementation ArrayProperty::setSize() does an element-wise insertElement() to grow the array.

I made first attempts to implement such functionality, but since I am no C++ template expert this is hard for me because of all the compile time distinction of cases with PropertyFactory[n], Accessor[n] and the like.

When I think of an array represented as a pointer and its length I think of it in terms of an array_view<T> or a span<T>. Maybe a possible implementation for this issue could be based on a specialization of ArrayMapper for an array_view<T>. Unfortunately, a standardized class array_view is currently not yet available. Nevertheless, there are current implementations, like this one on GitHub: array_view.

---

Side note for future extension: Since our domain is image processing, I would of course be interested in an ArrayProperty for arrays with more than one dimension (for image processing, particularly, a support for two-dimensional arrays would be welcome). To illustrate this point, I give a code example:

template <typename T>
struct Image
{
    T* pdata_;
    std::size_t width_;
    std::size_t height_;
};

PONDER_TYPE(Image<uint8_t>)
...
    ponder::Class::declare<Image<uint8_t>>()
        .property("Image", &PodArray::parray_, &PodArray::width_, &PodArray::height_);
...
   arrayProperty.get(obj, 2, 5); // get image value at position [2][5]

The class span<T> would be suited for multi-dimensional arrays (there, the number of dimensions is called rank). span<T> is part of the GSL (Guidelines Support Library Guidelines) and maintained by Microsoft (you find it here on GitHub: GSL). Alas, I could imagine that to support an ArrayProperty for more than one dimension the effort would be high (and maybe in confict with the concepts so far). Also, I do not know in which state the implementation of span<T> is (its quite huge, more than 2000 LOC) and how platform independent it is.

---

As always, I would be interested in your opinion and further plans. If you prefer I could try to continue with my attempts to implement such functionality (for the one-dimensional case).

[billyquith]

I think perhaps you are trying to extend the functionality of Ponder when you can use the existing functionality and use a slightly different approach.

template <typename T>
struct PodArray
{
    T* parray_;
    std::size_t size_;

    void setSize(std::size_t newSize);
};

typedef PodArray<uint8_t> Array8u;

PONDER_TYPE(Array8u);

ponder::Class::declare<Array8u>()
        .property("size", &Array8u::size_, &Array8u::setSize); // get/set

i.e. Ponder doesn't manipulate the array, your code does. Ponder just reflects the API. How the code behaves is up to you. You just write a class in C++, test the behaviour and then expose it to Ponder. You don't use Ponder to implement your behaviour if that makes sense. Think of Ponder as layer you just add on top of your code. I hope I have understood the question.

There are many ways to implement containers, and the way they grow/shrink/are managed depends on the use case. This should not be built into Ponder.

[billyquith]

Side note for future extension: Since our domain is image processing, I would of course be interested in an ArrayProperty for arrays with more than one dimension (for image processing, particularly, a support for two-dimensional arrays would be welcome).

Similar notes. Write the API for the class, then expose it. I think using properties here is perhaps causing complications. Properties are just shorthand for get/set functions.

template <typename T>
struct Image
{
    T* pdata_;
    std::size_t width_;
    std::size_t height_;

    void setPixel(std::size_t x, std::size_t y, T value);
    T getPixel(std::size_t x, std::size_t y) const;
};

PONDER_TYPE(Image<uint8_t>)

ponder::Class::declare<Image<uint8_t>>()
    .function("get", &Image<uint8_t>::getPixel)
    .function("set", &Image<uint8_t>::setPixel);

obj.get(2, 5); // get image value at position [2][5]

[billyquith]

Closing due to no response.

[Weeena]

This comment might get lengthy. Fortunately, it has a happy ending :-).

---

To clarify what I wanted to achieve with my issues regarding arrays (this issue #32 and additionally issue #31): I wanted to implement an (as much as possible) generic serialization and deserialization for pondered objects. I'd like to do this much the same as in your XML example code (found in .\include\ponder-xml). For arrays, however, I'd like something more efficient than accessing the values individually. Rather, I'd like to get access to the array data with a reference or a pointer and treat the data as a buffer.

So, I want to show some of my attempts. Hopefully, someone finds this helpful. One obvious was this:

template <typename T>
struct PodArray
{
    T* parray_;
    std::size_t size_;

    void setSize(std::size_t newSize);
    const T* getDataByPtr() { return parray_; };
    const T& getDataByRef() { return *parray_; };
};

typedef PodArray<uint8_t> Array8u;

PONDER_TYPE(Array8u);

ponder::Class::declare<Array8u>()
        .constructor()
        .property("Size", &Array8u::size_, &Array8u::setSize)  // get/set
        .property("Data", &Array8u::parray_); //<== compiler error

This leads to compiler errors for the property "Data". Equally:

        .property("DataByPtr", &T::getDataByPtr); //<== compiler error

Again compiler errors for property "DataByPtr". Next try:

        .property("DataByRef", &T::getDataByRef);

No compiler error. But if you try to access this property

        const Array8u& dataByRef = obj.get("DataByRef");

you get a runtime error. It seems that, since the return value is passed with a ponder::Value, it is not possible to pass a reference to a POD with ponder::Value.

Ok, we still have additional ideas. Let's try it such:

struct VectorWrapper
{
    std::vector<uint8_t> vector_;
    const std::vector<uint8_t>* getDataByPtr() const { return &vector_; };
    const std::vector<uint8_t>& getDataByRef() const { return vector_;  };
};

PONDER_TYPE(std::vector<uint8_t>)
PONDER_TYPE(VectorWrapper)
...
ponder::Class::declare<std::vector<uint8_t>>()
        .constructor()
        .constructor<std::size_t>();
ponder::Class::declare<VectorWrapper>()
        .constructor()
        .property("DataByPtr", &VectorWrapper::getDataByPtr) //<== compiler errors with ponder::ArrayPropertyImpl
        .property("DataByRef", &VectorWrapper::getDataByRef) //<== compiler errors with ponder::ArrayPropertyImpl
        .property("Data", &VectorWrapper::vector_);

The difference to the PodArray example is now that we declared std::vector<uint8_t> to ponder in the hope that it could be treated like a ponder::UserObject. Alas, this is not the case. The properties "DataByPtr" and "DataByRef" lead to compiler errors with ponder::ArrayPropertyImpl, and the "Data" property leads to compiler errors once the property is tried to accessed:

            const std::vector<uint8_t>& data = obj.get("Data"); //<== compiler errors with ponder::ValueImpl

My conclusion: I cannot use array types like std::vector in the same manner as a ponder::UserObject because std::vector and the like are treated in ponder in a special way, namely as arrays for its ponder::ArrayProperty.

---

Finally, I think I have found a satisfactory solution for my purpose: I can use a gsl::span. This is a wrapper for contiguous arrays. Its implementation can be found here: https://github.com/Microsoft/GSL/tree/master/gsl.

To show how it could work, I give some example code:

#include "span.h"

PONDER_TYPE(gsl::span<uint8_t>)

ponder::Class::declare<gsl::span<uint8_t>>();

struct VectorWrapper
{
    std::vector<uint8_t> vector_;
    gsl::span<uint8_t> getDataBySpan() { return gsl::span<uint8_t>(vector_); };
};

PONDER_TYPE(VectorWrapper)

ponder::Class::declare<VectorWrapper>()
        .constructor()
        .property("DataBySpan", &VectorWrapper::getDataBySpan);

Then, I can gain access to the array data by asking for a span

        gsl::span<uint8_t> dataBySpan = obj.get("DataBySpan");

and the using the access function gsl::span::data() which returns a pointer to the array data.

One more note: In my initial comment, I wrote that our domain is image processing, hence I am interested in arrays with more than one dimension (for image processing in particular I need two-dimensional arrays). The good news: gsl::span supports multiple dimensions. Futhermore: Often in image processing, not the entire image shall be accessed, rather only a "region of interest" (some rectangular sub-image). Another good news: gsl::span also provides a gsl::strided_span which can be used for this purpose.

[billyquith]

Thanks for the work you've put in. I'll try and revisit this topic when I have more time. In the meantime I'll reopen the ticket so it doesn't get forgotten. 🐘

[billyquith]

Reopened.

[billyquith]

Related to return policies #51.