Support for other JSON serialization libraries

[FeldrinH]

It would be great to have this client support other JSON serialization libraries. Ideally the API would allow users to supply their own JSON serialization logic.

In particular there are two issues I have with Gson that the ability to use other libraries would resolve:

• As far as I know Gson breaks Kotlin's null safety and default value handling (see https://medium.com/@jurajkunier/kotlinx-json-vs-gson-4ba24a21bd73).
• Existing custom serialization logic implemented using some other JSON serialization library has to be ported to Gson, which depending on the specific features needed may be quite complicated.

[phughk]

Hey thanks for highlighting! This may require a bit more tinkering. The way I see it, there would need to be multiple modules. A core module that handles the logic, and then separate "provided" modules that would be tied to a serialisation library.

So this ticket is about moving the design of the surrealdb.java project into multi module, where there is a serialisation module that handles just the serialisation asepct.

It would be good to understand how that glues together - including scouting out how other projects do this.

I personally, don't have as much time as I would like to do this, but if anyone would like a look, please share! I have added the help wanted tag to this 🙏

[mnbjhu]

I think it's worth noting that, whilst providing a serialization strategy which uses kotlinx is possible, there would be performance losses compared to using it in kotlin.

Kotlinx serialization is branded as a reflection-free library. It can do this by exposing most of it's API as inline functions with reified generic type parameters. The significance of this is that the generic type of the function must be known at compile time. While in Kotlin you can call these functions with a generic argument as long as it's also reified (example in the kotlin driver: https://github.com/mnbjhu/surrealdb-kotlin-driver/blob/master/src/commonMain/kotlin/uk/gibby/driver/rpc/functions/Create.kt), there's not a way to do this in Java.

This would mean you'd have to either:

• Allow the user to pass a KSerializer<T> rather than using the type argument or Class (requiring changes to the API).
• Use the reflective methods for retrieving the KSerializer (jvm tab: https://kotlinlang.org/api/kotlinx.serialization/kotlinx-serialization-core/kotlinx.serialization/serializer.html)

[FeldrinH]

I think out of those two options I would prefer the ability to pass in KSerializer using a new API. I think that as long as that API is cleanly separated adding new API-s shouldn't be too bad.

Though now that a separate Kotlin driver exists I think the best approach would be to push for the Kotlin driver to become more official and explicitly not support Kotlin specific serialization methods in the Java driver. (A separate Kotlin driver is also useful because Kotlin can run on platforms other than the JVM, which a Java-based driver can't support.) (Note that when this issue was created the Kotlin driver did not exist.)

[mnbjhu]

I'm not sure I can do anything to make the driver official, although I will create a PR to add it to awesome-surreal. However if you have any features you'd like or notice any issues feel free to raise it on the repo. Any feedback would be amazing!

[phughk]

@mnbjhu regarding making the driver official: we aren't directly trying to have a kotlin driver, so we could officially send people who want a Kotlin-only driver to your repo. It certainly seems to be the preferred implementation at the moment.

The way we see the driver going is to have the possibility to support the surrealdb binary inside of it. That means you could run surrealdb natively, in embedded mode with any and all operations on top of it. Plus we would like the java driver to be as compatible as possible with JVM variations (platform, arch, language invoked from). So it is unlikely we would go down the Kotlin route. I don't know enough about Kotlin - it is a possibility.

RE: replaceable JSON libraries - we could definitely do this. I haven't done that in the past, but I imagine its enough to have an interface for creating/consuming JSON. Then have various providers that activate when a library is available.

[mnbjhu]

@phughk That'd be great!

Support for embedded SurrealDB is also something I have planned so I'd be interested in how you plan to approach it.

I'd like to add that while there is a seperate Kotlin driver, it currently works by using common kotlin libraries (mostly ktor). Ideally the library would instead use the native libraries and expose a target agnostic multiplatform API. This would mean smaller bundles (particularly implortant for web) and presumably better performance. While the jvm target will probably be the least affected by changing to use the native library, it's still by far the most used kotlin target