Converting from a bsoncxx::value to a specified type

[Lazrius]

Good morning!

Sorry for another thing to bother you over, but I was wondering how possible it would be to use the BSON converter to convert between a bsoncxx::value to a struct/class. In our project we are storing data inside of a Mongo database using the mongocxx driver, which depends upon libbson.

When we get a document via the driver we get a bsoncxx::view or bsoncxx::value back, and I noticed when reading through the docs that it's possible to implement a custom from_bson method. I understand that the BSON library here is using the C driver, but I was wondering if it would be possible somehow to convert between the bsoncxx:value we get and the reflect-cpp interface in order to serialize the class into our desired type.

There are somethings on the class that exist that I doubt reflect-cpp has any context over, like bsoncxx::oid _id (which most documents in the db have). I find the mongocxx docs are not the best so was hoping you might have some insight if this is possible or potential steps to try and implement it - I know given its a different library/dependency it cannot be implemented into this library itself, but maybe you have an idea of what needs to be done on our side. I imagine this is not a unique issue ^^

[liuzicheng1987]

Hi, don’t apologise…issues and comments are often just as helpful as code contributions.

As far as an oid is concerned we could simply treat that as an integer.

I agree that the documentation is not the best…implementing the libbson interface was a bit of a nightmare. Is it somehow possible to get the underlying libbson type from bsoncxx::view or bsoncxx::value?

[liuzicheng1987]

I think it is possible to return the underlying bytes from bsoncxx::value or view using .data().

https://github.com/mongodb/mongo-cxx-driver/blob/master/src/bsoncxx/include/bsoncxx/v_noabi/bsoncxx/document/value.hpp

You could then use that to construct a libbson value without copying the either by using this:

https://mongoc.org/libbson/current/bson_init_static.html

or by constructing a BSON value the way we do it here:

https://github.com/getml/reflect-cpp/blob/main/include/rfl/bson/read.hpp

It should then be fairly easy to write a rfl::bson::read function that takes bsoncxx::value or bsoncxx::view as an input. I agree that this is not an issue that only you would have.

[liuzicheng1987]

@Lazrius, here's a feature branch:

https://github.com/getml/reflect-cpp/tree/f/bsoncxx

I have added a function rfl::bson::read_bsoncxx(document_or_view). Could you confirm that it works?

https://github.com/getml/reflect-cpp/blob/f/bsoncxx/include/rfl/bson/read.hpp

[liuzicheng1987]

@Lazrius , I hadn't heard back from you, but I want to draft a new release, so I going for this solution for now:

https://github.com/getml/reflect-cpp/commit/7f2be9b7cbe8e967bad667dbe7fe5f1e94a77ed2

I am rather certain that this will work now:

read<YourStruct>(bsoncxx_value_or_view.data(), bsoncxx_value_or_view.length());

But I haven't tested it myself, so I don't want to document it as the proposed solution for this problem. If you could give it a try and then give me feedback whether it worked then I would explicitly put this in the documentation.

[Lazrius]

So sorry for not getting back to you. I've been busy with a different project and haven't a chance to test it yet. I will try my best to test it next weekend when I have some more time! Thank you again for your work.

[liuzicheng1987]

OK, no problem.

[Lazrius]

Very sorry for the delay, I've been away for a while. While testing it I found that bsoncxx::oid was not a recognised type. Is this a thing I'll have to implement myself?

I tested with this structure:

struct Account
{
        std::string _id;
        std::vector<bsoncxx::oid> characters; // Character is another document type that has an bsoncxx::oid _id field
        bool banned = false;
        int64 scheduledUnbanDate = 0;
        int64 cash = 0;
        std::optional<std::vector<std::string>> gameRoles;
        std::optional<std::vector<std::string>> webRoles;
        std::optional<std::string> hashedToken;
        std::optional<std::vector<byte>> salt;
        std::optional<std::string> username;
        std::optional<std::string> passwordHash;
};

And called it like so:

    auto accountsCollection = db.GetCollection("accounts");
    if (!accountsCollection.has_value())
    {
        return;
    }

    const auto accountBson = accountsCollection->GetItemByIdRaw(id);
    if (!accountBson.has_value())
    {
        // TODO: Create Account
        return;
    }

    auto& accountRaw = accountBson.value();
    auto account = rfl::bson::read<Account>(accountRaw.view().data(), accountRaw.view().length());
    if (account.error().has_value())
    {
        return;
    }

[liuzicheng1987]

@Lazrius , no problem about the delay.

I think what I could to is to support the bson_oid_t. That wouldn't be hard.

Looking at the code, it seems impossible for me to support the bsoncxx class without explicitly having bsoncxx as a dependency. But we're using the C library and it seems like overkill to be introducing a dependency for just this one use case.

Would that be an acceptable solution to the problem?

[Lazrius]

That makes sense and sounds like a good solution. Out of curiosity, how hard would it be for us to implement the mongocxx version within our project? I assume its just a template definition somewhere?

If a bother, I am fairly sure that I can easily handle it with the following and storing them as bson_oid_t.

    // Transform our std::vector into a bson array
    auto idArr = bsoncxx::builder::basic::array{};
    for (auto id : account.characters) // characters has been changed into a std::vector<bson_oid_t>
    {
        idArr.append(bsoncxx::oid(reinterpret_cast<const char*>(id.bytes), bsoncxx::oid::size()));
    }

[liuzicheng1987]

@Lazrius for discussions on OID support, please refer to issue #69 . I believe this to be a separate issue from bsoncxx::value and bsoncxx::view.

[Lazrius]

Understood! Thank you very much for your help. This issue can be closed now I think.