dropbox / djinni

A tool for generating cross-language type declarations and interface bindings.
Apache License 2.0
2.88k stars 488 forks source link

map<string, any>? #371

Closed nmtitov closed 6 years ago

nmtitov commented 6 years ago

Is it possible to make maps with values of any types and pass them from C++ to Swift & Java somehow?

For example, can I receive a typical JSON dictionary in C++ and pass it back and forth to a Swift client?

artwyman commented 6 years ago

C++ doesn't have a native "any" type, nor do the other languages you mention as far as I know (though I'm less certain there). There wouldn't be any way to generically marshal any type across languages regardless, without using some sort of serialization framework (java.io.Serializable, protobufs, JSON).

For JSON your easy options are either to serialize to a string, or to build an interface an/or record which can handle the types you need. If you have a preferred JSON-supporting type in each language, you could also implement it as a custom external type, but then you'd need to implement your own marshaling across languages. For more info on that see https://github.com/dropbox/djinni#modularization-and-library-support

nmtitov commented 6 years ago

@artwyman Just to make it clear for myself: in Swift and Objective-C [string: Any] is a native thing, in C++ 17 we have std::variant, in Java there is a Map<String, Object>. These features don't help anyhow? By any type I mean only boolean, numeric, string and map type, that would be enough for me.

artwyman commented 6 years ago

Those types could be helpful, but don't solve the whole problem, and don't have any uniformity or interoperability between languages. std::variant can't be recursive, so I don't think it can actually represent JSON without adding pointers to the mix. The problem with an "any" type like java.lang.Object in the context of bridging between languages is that it's too general. It opens the question of what the bridge should do with types it doesn't understand.

In general, any/variant types in type-safe languages depend on knowing the type (or querying it) each time the object is used. That means the code using the object is generally going to have to handle only a specific set of types, not all of them. There are some operations like copy/clone which might be done in a polymorphic way, but that kind of polymorphism doesn't translate across languages. That means the marshaling in Djinni would be a case of "using" the object, and would have to know the type and/or query for a specific set of handled types. It's not feasible to marshal any arbitrary type.

For the special case of JSON, there would be a fixed (though recursive) set of types, so it would be feasible to do. Given that, I'd tend to prefer something more specific to JSON with more type-safety, rather than a generic "any" type. You can see an example of the JSON type we use in our C++ code at Dropbox here: https://github.com/dropbox/json11

You could use your preferred JSON type in each language, and write marshaling for it as an external type as I mentioned before. Personally if I were going to tackle that, I'd still probably marshal by writing a JSON string and re-parsing it on the other end. That's mostly for simplicity and ease of implementation. It would be possible to recursively traverse the JSON object and marshal all the individual fields, but for most use cases I'd guess that the additional trips through the bridge (particularly in JNI - the bridge for ObjC++ is much lighter) would add enough overhead to outweigh the cost of JSON parsing. Your mileage may vary depending on the size and composition of your JSON objects.

Alternately, you could leave your JSON data in one language, and use an interface to bridge individual methods to access the data. At that point, the interface to your JSON object isn't much different from the interface to a database, or any other sort of storage. Whether that's a good idea is dependent on your use case.

nmtitov commented 6 years ago

Great insight! Thank you.

I've been using json11 for a while and it's a great library. I'll try to pass data as JSON, maybe serializing/deserializing overhead won't be that noticeable.