Can ponder be used to build a generic data repository?

[Weeena]

We consider building a generic data repository (GDR) in which entries are stored in the following manner:

struct Entry {
    const ponder::Class& metaClass;
    void* p;
}

If we have created some object of class T (with T being a class declared to ponder)

T* pObject = ...

we would store it in the GDR like this:

someEntry.p = pObject;
someEntry.metaClass = ponder::classByObject(*pObject);

If that would work we could examine our GDR in a very generic way by initializing a ponder::UserObject for each entry and working with that ponder::UserObject through the reflection capabilities of ponder.

Alas, for such a generic data repository to work some obstacles would have to be overcome.

First of all, there currently is no functionality to initialize a ponder::UserObject from a ponder::Class and a void* pointer. (I'd guess this would be doable.)

Second, we would also like to store PODs, std::string, std::array, std::vector and the like in the GDR. For those types, there is no direct support in ponder, i.e. it is not possible to simply declare those types to ponder and initialize a ponder::UserObject for it. One way would be to wrap those types into custom classes like

template <typename T>
struct PodWrapper
{
    T value;
};
PONDER_TYPE(PodWrapper<uint8_t>)
PONDER_TYPE(PodWrapper<uint16_t>)
...
ponder::Class::declare<PodWrapper<uint8_t>>("PodWrapper<uint8_t>")
    .constructor()
    .property("Value", &PodWrapper<uint8_t>::value);
ponder::Class::declare<PodWrapper<uint16_t>>("PodWrapper<uint16_t>")
    .constructor()
    .property("Value", &PodWrapper<uint16_t>::value);
...    

but maybe there is some better way.

I would be glad to hear your opinion on the subject of this issue.

[billyquith]

I can understand the rationale for this. It is a bit inefficient to have to store UserObjects. They are really only required when you need to use a metaclass API. I have a similar issue with the Lua binding I am working on in the Script branch. I am thinking a similar structure would be more efficient way of storing an object instance. However, reconstructing a UserObject looks somewhat inefficient at the moment due to the various features it offers, like parent-child. I was wondering about having different versions of UserObject, perhaps a simple one and then a hierarchy one.

[billyquith]

Second, we would also like to store PODs, std::string, std::array, std::vector and the like in the GDR. For those types, there is no direct support in ponder, i.e. it is not possible to simply declare those types to ponder and initialize a ponder::UserObject for it. One way would be to wrap those types into custom classes

Well they aren't PODs, they are templated containers. PODs have no constructors.

We could almost do with a meta-metaclass. I.e. you declare a template container type and then the types it should implement and the metaclasses are created accordingly.

Class::instance_of<
    MetaClass::declare<std::vector>()
        .constructor()
        .constructor(std::size_t)
        .function("append", &std;:vector::push_back),
    int8_t, int32_t >();

This would save multiple declarations. But the same thing could be achieved probably more simply using a template declaration function.

template <typename T>
void declare_vector() {
    Class::declare<std::vector<T>>()
        .constructor()
        .constructor(std::size_t)
        .function("append", &std;:vector<T>::push_back);
}

void declare() {
    declare_vector<int8_t>();
    declare_vector<int32_t>();
}

[Weeena]

store UserObejcts...

Yes, that would be a possibility. It costs some extra memory, though. Couriously enough, I haven't thought of this (maybe I'm too efficiency minded)...

Well they aren't PODs...

I meant "we would also like to store PODs as well as std::string, std::array, std::vector and the like...".

using a template declaration function

... for std::vector: If I get it right, with this declaration I cannot access the data contained in that vector like I could if the std::vector would be a member of a user defined class. If a have a class (I apologize for not using a template)

struct VectorWrapper
{
    std::vector<uint8_t> vector_;
};
PONDER_TYPE(VectorWrapper)
...
    ponder::Class::declare<VectorWrapper>("VectorWrapper")
        .constructor()
        .property("Vector", &VectorWrapper::vector_);

I can get me a ponder::ArrayProperty and manipulate vector_ with its generic API. Alas, with

    ponder::Class::declare<std::vector<uint8_t>>()
        .constructor()
        .constructor(std::size_t)
        .function("append", &std;:vector<T>::push_back);

I cannot as simply access the data in a generic manner. This is a kind of asymmetry between user objects holding data objects and the data objects itself (which may be PODs or e.g. containers).

[billyquith]

The thing declaring the vector is a template function, not a struct container. The template argument is the type for the vector. They should have unique names because the contained types are different (untested).

[Weeena]

Regarding the idea that an object of type T could be stored with a void* pointer and its Class

struct Entry {
    const ponder::Class& metaClass;
    void* p;
}

and that it would be nice if a UserObject could be initialized from these (see my first comment in this issue).

I would argue that such an initialization makes sense. I can initialize a UserObject from a typed object

UserObject (const T &object)

why then not from a void* pointer and its Class?

For testing purposes, I tried this:

class.hpp:

    ....
    typedef UserObject (*Caster)(void*);
    Caster m_caster;

public:
    ...
    UserObject get(void* pointer) const;

class.inl:

namespace detail
{
    template <typename T>
    UserObject get(void* pointer)
    {
        return UserObject(*static_cast<T*>(pointer));
    }
}

template <typename T>
inline ClassBuilder<T> Class::declare(const std::string& name)
{
    ...
    newClass.m_caster = &detail::get<T>;
    ...
}

class.cpp:

UserObject Class::get(void* pointer) const
{
    return m_caster(pointer);
}

With this, I can get a UserObject from a void* pointer and a Class in the following way:

UserObject obj = metaClass.get(p);

(I am not sure if get() is the right name for this function. I thought about cast() but I am not sure either. I also did neither check if there could be a better implementation nor did I carry out extensive testing.)

If you think such a functionality would fit into your concept of ponder I would be glad if you support such a functionality in one of the next releases of ponder.

[billyquith]

For the second part you will have to instance a template and make a type. You cannot declare a template since that is a compile-time concept that generates source code. We can only reflect what is compiled. E.g.

template <typename T>
void declare_vector() {
    Class::declare<std::vector<T>>()
        .constructor()
        .constructor(std::size_t)
        .function("append", &std;:vector<T>::push_back);
}

void declare() {
    declare_vector<int8_t>();
    declare_vector<int32_t>();
}

[billyquith]

I'll close this for now as I think both parts are answered.

[billyquith]

Sorry work was so slow on this. I had IT problems last week and wasted a lot of time. 😒

[Weeena]

You don't have to be sorry, I think you still react very fast. I am sorry for you that you had to waste time.

In some respects I am slow. We are still trying to sculpture ponder such that we can best use it for our application. Maybe there are more issues to come...

[billyquith]

It's all good feedback. You have pointed out some genuine bugs and usability problems. I haven't really used Ponder that much TBH. It is sort of a side project. I do intend to use it a lot more in the future so I'd encounter the same problems. It is useful having multiple people critiquing it. 👍