stephenberry / glaze

Extremely fast, in memory, JSON and interface library for modern C++
MIT License
1.25k stars 124 forks source link

Support for any type? #24

Closed paulocoutinhox closed 1 year ago

paulocoutinhox commented 2 years ago

Hi,

There is a way to have something like this as i do in Kotlin?

class Param(val n: String, val v: Any)

val request = Request(
    "test1",
    Param("username", "paulo"),
    Param("password", "123456"),
    Param("remember", true)
)

val gson = Gson()
val str = gson.toJson(request)

Thanks.

stephenberry commented 2 years ago

We are planning to add support for std::variant, which will allow you to write out generic structures like this.

For example, you'll be able to do the following:

std::map<std::string, std::variant<std::string, int, bool>> request;

request["username"] = "paulo";
request["password"] = "123456";
request[remember] = true;

auto str = glz::write_json(request);

I'll keep this issue open until this has been added. Thanks for your question!

mwalcott3 commented 2 years ago

std::any support is incredibly unlikely. std::variant write support is incredibly easy but reading is somewhat problematic.

Glaze essentially does a typed json parse and doesn't have good support for generic json parsing. Generally, libs with support for generic json store values in an intermediate struct that can be queried for criteria and then converted to the appropriate type. But that is not an option in this lib. Since glaze stores data directly in user-defined structures, it needs to know what type it is parsing. Not having an intermediate representation was a performance consideration to avoid needles allocations and copying but impacts certain usability cases like this.

Variant support will most likely have similar limitations to libs that function similarly like daw_json_link https://github.com/beached/daw_json_link/blob/release/docs/cookbook/variant.md where the variant can at most contain one each of the fundamental json types like numeric, string, object (Or user-defined struct), and array. This allows the lib to efficiently determine the type with the first char of the value before anything needs to be stored.

For the broader case of all variants if a type field was added we could determine the type from that field and parse to it. Ive seen some libs like daw_json_link and json_struct do this to support variants and polymorphism. However, while this is valid json its not exactly standard. For instance daw_json_link uses "type" (In their example but support any field) and dotnets json serializer uses "__type" https://learn.microsoft.com/en-us/dotnet/framework/wcf/feature-details/stand-alone-json-serialization. Typed json would be nice but its not really a thing.

Alternatively, types could be inferred by iterating through a variant and backtracking when it fails but this is an incredibly expensive operation and a very bad idea. I don't think Ive seen any libs take this approach largely because it's a terrible idea.

Probably not what you are looking for but the raw_json type can be used for some use cases of generic json values in a limited fashion where you dont know the type up front but can figure it out. I don't think there are any docs on it and I believe it may be in the detail namespace. This doesn't really offer untyped parsing but allows you to delay the full parsing till you know the type.

If you are less interested in serialization/deserialization and more interested in generic json parsing simdjson is probably a better bet. It uses an iterator approach to avoid having to use an intermediate struct when using there on-demand option.

mwalcott3 commented 1 year ago

Doesn't directly address what the OP was looking for but @stephenberry added custom variant read-write support via type tags in #80.

stephenberry commented 1 year ago

The requested example is now supported and part of our test suite:

std::map<std::string, std::variant<std::string, int, bool>> request;

      request["username"] = "paulo";
      request["password"] = "123456";
      request["remember"] = true;

      auto str = glz::write_json(request);

      expect(str == R"({"password":"123456","remember":true,"username":"paulo"})");

Object variant handling with type deduction is also now supported and documented in the Wiki: Variant Handling