Performance problem with sol::make_object

[maxdemarzi]

I have a set of Nodes, these nodes have properties which have a key (std::string) and value (Variant property_type_t) .

To convert I am iterating through the properties in the map and calling PropertyToSolObject on each value.

  sol::table Shard::PropertiesToSolObject(const std::map<std::string, property_type_t>& properties) {
    sol::table property_map = lua.create_table();
    for (const auto& [_key, value] : properties) {
      property_map[_key] = PropertyToSolObject(value);
    }
    return property_map;
  }

The definition of PropertyToSolObject is below and uses sol::make_object

sol::object Shard::PropertyToSolObject(const property_type_t value) const {

    switch (value.index()) {
      case 0:
        return sol::lua_nil;
      case 1:
        return sol::make_object(lua.lua_state(), get<bool>(value));
      case 2:
        return sol::make_object(lua.lua_state(), get<int64_t>(value));
      case 3:
        return sol::make_object(lua.lua_state(), get<double>(value));
      case 4:
        return sol::make_object(lua.lua_state(), get<std::string>(value));
      case 5:
        return sol::make_object(lua.lua_state(), sol::as_table(get<std::vector<bool>>(value)));
      case 6:
        return sol::make_object(lua.lua_state(), sol::as_table(get<std::vector<int64_t>>(value)));
      case 7:
        return sol::make_object(lua.lua_state(), sol::as_table(get<std::vector<double>>(value)));
      case 8:
        return sol::make_object(lua.lua_state(), sol::as_table(get<std::vector<std::string>>(value)));
      default:
          return sol::lua_nil;
      }
    return sol::lua_nil;
  }

Converting 100k nodes each with 5 properties or so is taking about 100ms which is 10x more than it takes to gather the data.

Is there a higher performance way of doing this?

The code is on https://github.com/ragedb/ragedb/blob/main/src/graph/lua/Helpers.cpp I am following examples found on:

• https://github.com/ThePhD/sol2/issues/1035
• https://github.com/ThePhD/sol2/issues/86

[maxdemarzi]

Can I do something like https://github.com/ThePhD/sol2/issues/1012 to create the table to return instead of using sol::make_object?

Also related, new_table(int sequence_hint = 0, int map_hint = 0); doesn't really tell me what the hints mean. I have a problem now where I try to return a map of with large integer keys (3437773498) and it blows my memory. I think this is allocating a giant array section. How do I tell Sol to tell Lua to create a table that is allocates nothing for the array section and instead use the map section?

[maxdemarzi]

So it turns out I was creating objects for no good reason. We can just return a sol::table directly without as_table and create it right then and there. The properties returned were "property_type_t" which is a variant which Sol2 already supports, so just setting it directly worked fine.

   sol::table Shard::NodesGetPropertyViaLua(std::vector<uint64_t> ids, const std::string& property) {
    sol::table properties = lua.create_table(0, ids.size());
    for (const auto& [id, value] : NodesGetPropertyPeered(ids, property).get0()) {
        properties.set(id, value);
    }
    return properties;
}

I also tried just returning raw userdata but surprisingly it made no difference in the speed of my application:

    std::map<uint64_t, property_type_t> Shard::NodesGetPropertyViaLuaRaw(std::vector<uint64_t> ids, const std::string& property) {
    return NodesGetPropertyPeered(ids, property).get0();
}

Oh, and the large integer key issue was a bug in my json encoding, nothing to do with Sol/Lua/LuaJIT. Closing issue.