vert-x3 / vertx-lang-kotlin

Vert.x for Kotlin
Apache License 2.0
296 stars 68 forks source link

json builder processes binary data differently from JsonObject class #202

Closed Ultranium closed 3 years ago

Ultranium commented 3 years ago

Version

Vert.x 4.1.0, Kotlin 1.5.10, JDK 16.0.1

Context

It seems that the json builder uses a Base64 algorithm that is different from the one used in the JsonObject itself, and it leads to problems when used with MongoClient (

Steps to reproduce

val binaryValue = byteArrayOf(58, -104, 93, -89, 79, -30, 37, -78, 4, 92, 23, 45, 107, -45, -112, -67, -123, 95,
    8, 110, 62, -99, 82, 91, 70, -65, -30, 69, 17, 67, 21, 50)

val jsonObject1 = JsonObject()
    .put("myValue", binaryValue)

val jsonObject2 = json {
    obj(
        "myValue" to binaryValue
    )
}

println(jsonObject1)
println(jsonObject2)

The output is:

{"myValue":"Ophdp0_iJbIEXBcta9OQvYVfCG4-nVJbRr_iRRFDFTI"}
{"myValue":"Ophdp0/iJbIEXBcta9OQvYVfCG4+nVJbRr/iRRFDFTI="}

The first approach works with MongoClient.insert() just fine. The second one (with json builder) fails if the final Base64 data contains slashes with this exception:

Stack trace ``` java.lang.IllegalArgumentException: Illegal base64 character 2f at java.util.Base64$Decoder.decode0(Base64.java:847) ~[?:?] at java.util.Base64$Decoder.decode(Base64.java:566) ~[?:?] at java.util.Base64$Decoder.decode(Base64.java:589) ~[?:?] at io.vertx.core.json.JsonObject.getBinary(JsonObject.java:308) ~[mongotest-1.0.0-SNAPSHOT-fat.jar:?] at io.vertx.ext.mongo.impl.codec.json.JsonObjectCodec.writeBinary(JsonObjectCodec.java:216) ~[mongotest-1.0.0-SNAPSHOT-fat.jar:?] at io.vertx.ext.mongo.impl.codec.json.AbstractJsonCodec.writeValue(AbstractJsonCodec.java:96) ~[mongotest-1.0.0-SNAPSHOT-fat.jar:?] at io.vertx.ext.mongo.impl.codec.json.AbstractJsonCodec.lambda$writeDocument$1(AbstractJsonCodec.java:277) ~[mongotest-1.0.0-SNAPSHOT-fat.jar:?] at io.vertx.ext.mongo.impl.codec.json.JsonObjectCodec.lambda$forEach$0(JsonObjectCodec.java:124) ~[mongotest-1.0.0-SNAPSHOT-fat.jar:?] at java.lang.Iterable.forEach(Iterable.java:75) ~[?:?] at io.vertx.ext.mongo.impl.codec.json.JsonObjectCodec.forEach(JsonObjectCodec.java:123) ~[mongotest-1.0.0-SNAPSHOT-fat.jar:?] at io.vertx.ext.mongo.impl.codec.json.JsonObjectCodec.forEach(JsonObjectCodec.java:25) ~[mongotest-1.0.0-SNAPSHOT-fat.jar:?] at io.vertx.ext.mongo.impl.codec.json.AbstractJsonCodec.writeDocument(AbstractJsonCodec.java:274) ~[mongotest-1.0.0-SNAPSHOT-fat.jar:?] at io.vertx.ext.mongo.impl.codec.json.AbstractJsonCodec.encode(AbstractJsonCodec.java:29) ~[mongotest-1.0.0-SNAPSHOT-fat.jar:?] at org.bson.codecs.BsonDocumentWrapperCodec.encode(BsonDocumentWrapperCodec.java:63) ~[mongotest-1.0.0-SNAPSHOT-fat.jar:?] at org.bson.codecs.BsonDocumentWrapperCodec.encode(BsonDocumentWrapperCodec.java:29) ~[mongotest-1.0.0-SNAPSHOT-fat.jar:?] at com.mongodb.internal.connection.SplittablePayload$WriteRequestEncoder.encode(SplittablePayload.java:200) ~[mongotest-1.0.0-SNAPSHOT-fat.jar:?] at com.mongodb.internal.connection.SplittablePayload$WriteRequestEncoder.encode(SplittablePayload.java:187) ~[mongotest-1.0.0-SNAPSHOT-fat.jar:?] at org.bson.codecs.BsonDocumentWrapperCodec.encode(BsonDocumentWrapperCodec.java:63) ~[mongotest-1.0.0-SNAPSHOT-fat.jar:?] at org.bson.codecs.BsonDocumentWrapperCodec.encode(BsonDocumentWrapperCodec.java:29) ~[mongotest-1.0.0-SNAPSHOT-fat.jar:?] at com.mongodb.internal.connection.BsonWriterHelper.writeDocument(BsonWriterHelper.java:77) ~[mongotest-1.0.0-SNAPSHOT-fat.jar:?] at com.mongodb.internal.connection.BsonWriterHelper.writePayload(BsonWriterHelper.java:59) ~[mongotest-1.0.0-SNAPSHOT-fat.jar:?] at com.mongodb.internal.connection.CommandMessage.encodeMessageBodyWithMetadata(CommandMessage.java:162) ~[mongotest-1.0.0-SNAPSHOT-fat.jar:?] at com.mongodb.internal.connection.RequestMessage.encode(RequestMessage.java:138) ~[mongotest-1.0.0-SNAPSHOT-fat.jar:?] at com.mongodb.internal.connection.CommandMessage.encode(CommandMessage.java:59) ~[mongotest-1.0.0-SNAPSHOT-fat.jar:?] at com.mongodb.internal.connection.InternalStreamConnection.sendAndReceiveAsync(InternalStreamConnection.java:393) ~[mongotest-1.0.0-SNAPSHOT-fat.jar:?] at com.mongodb.internal.connection.UsageTrackingInternalConnection.sendAndReceiveAsync(UsageTrackingInternalConnection.java:145) ~[mongotest-1.0.0-SNAPSHOT-fat.jar:?] at com.mongodb.internal.connection.DefaultConnectionPool$PooledConnection.sendAndReceiveAsync(DefaultConnectionPool.java:527) ~[mongotest-1.0.0-SNAPSHOT-fat.jar:?] at com.mongodb.internal.connection.CommandProtocolImpl.executeAsync(CommandProtocolImpl.java:77) ~[mongotest-1.0.0-SNAPSHOT-fat.jar:?] at com.mongodb.internal.connection.DefaultServer$DefaultServerProtocolExecutor.executeAsync(DefaultServer.java:273) ~[mongotest-1.0.0-SNAPSHOT-fat.jar:?] at com.mongodb.internal.connection.DefaultServerConnection.executeProtocolAsync(DefaultServerConnection.java:218) ~[mongotest-1.0.0-SNAPSHOT-fat.jar:?] at com.mongodb.internal.connection.DefaultServerConnection.commandAsync(DefaultServerConnection.java:135) ~[mongotest-1.0.0-SNAPSHOT-fat.jar:?] at com.mongodb.internal.operation.MixedBulkWriteOperation.executeCommandAsync(MixedBulkWriteOperation.java:440) ~[mongotest-1.0.0-SNAPSHOT-fat.jar:?] at com.mongodb.internal.operation.MixedBulkWriteOperation.executeBatchesAsync(MixedBulkWriteOperation.java:348) ~[mongotest-1.0.0-SNAPSHOT-fat.jar:?] at com.mongodb.internal.operation.MixedBulkWriteOperation.access$1000(MixedBulkWriteOperation.java:76) ~[mongotest-1.0.0-SNAPSHOT-fat.jar:?] at com.mongodb.internal.operation.MixedBulkWriteOperation$2$1.call(MixedBulkWriteOperation.java:228) ~[mongotest-1.0.0-SNAPSHOT-fat.jar:?] at com.mongodb.internal.operation.OperationHelper.validateWriteRequests(OperationHelper.java:269) ~[mongotest-1.0.0-SNAPSHOT-fat.jar:?] at com.mongodb.internal.operation.MixedBulkWriteOperation$2.call(MixedBulkWriteOperation.java:211) ~[mongotest-1.0.0-SNAPSHOT-fat.jar:?] at com.mongodb.internal.operation.OperationHelper$9.onResult(OperationHelper.java:733) ~[mongotest-1.0.0-SNAPSHOT-fat.jar:?] at com.mongodb.internal.operation.OperationHelper$9.onResult(OperationHelper.java:730) ~[mongotest-1.0.0-SNAPSHOT-fat.jar:?] at com.mongodb.internal.connection.DefaultServer$1.onResult(DefaultServer.java:116) ~[mongotest-1.0.0-SNAPSHOT-fat.jar:?] at com.mongodb.internal.connection.DefaultServer$1.onResult(DefaultServer.java:105) ~[mongotest-1.0.0-SNAPSHOT-fat.jar:?] at com.mongodb.internal.async.ErrorHandlingResultCallback.onResult(ErrorHandlingResultCallback.java:48) ~[mongotest-1.0.0-SNAPSHOT-fat.jar:?] at com.mongodb.internal.connection.DefaultConnectionPool$2.onResult(DefaultConnectionPool.java:229) ~[mongotest-1.0.0-SNAPSHOT-fat.jar:?] at com.mongodb.internal.connection.DefaultConnectionPool$2.onResult(DefaultConnectionPool.java:210) ~[mongotest-1.0.0-SNAPSHOT-fat.jar:?] at com.mongodb.internal.connection.DefaultConnectionPool$PooledConnection$1.onResult(DefaultConnectionPool.java:447) ~[mongotest-1.0.0-SNAPSHOT-fat.jar:?] at com.mongodb.internal.connection.DefaultConnectionPool$PooledConnection$1.onResult(DefaultConnectionPool.java:438) ~[mongotest-1.0.0-SNAPSHOT-fat.jar:?] at com.mongodb.internal.connection.UsageTrackingInternalConnection$1.onResult(UsageTrackingInternalConnection.java:66) ~[mongotest-1.0.0-SNAPSHOT-fat.jar:?] at com.mongodb.internal.connection.UsageTrackingInternalConnection$1.onResult(UsageTrackingInternalConnection.java:58) ~[mongotest-1.0.0-SNAPSHOT-fat.jar:?] at com.mongodb.internal.connection.InternalStreamConnection$1$1.onResult(InternalStreamConnection.java:189) ~[mongotest-1.0.0-SNAPSHOT-fat.jar:?] at com.mongodb.internal.connection.InternalStreamConnection$1$1.onResult(InternalStreamConnection.java:175) ~[mongotest-1.0.0-SNAPSHOT-fat.jar:?] at com.mongodb.internal.connection.InternalStreamConnectionInitializer.completeConnectionDescriptionInitializationAsync(InternalStreamConnectionInitializer.java:215) ~[mongotest-1.0.0-SNAPSHOT-fat.jar:?] at com.mongodb.internal.connection.InternalStreamConnectionInitializer.access$100(InternalStreamConnectionInitializer.java:44) ~[mongotest-1.0.0-SNAPSHOT-fat.jar:?] at com.mongodb.internal.connection.InternalStreamConnectionInitializer$1$1.onResult(InternalStreamConnectionInitializer.java:92) ~[mongotest-1.0.0-SNAPSHOT-fat.jar:?] at com.mongodb.internal.connection.InternalStreamConnectionInitializer$1$1.onResult(InternalStreamConnectionInitializer.java:86) ~[mongotest-1.0.0-SNAPSHOT-fat.jar:?] at com.mongodb.internal.async.ErrorHandlingResultCallback.onResult(ErrorHandlingResultCallback.java:48) ~[mongotest-1.0.0-SNAPSHOT-fat.jar:?] at com.mongodb.internal.connection.SaslAuthenticator.verifySaslClientComplete(SaslAuthenticator.java:171) ~[mongotest-1.0.0-SNAPSHOT-fat.jar:?] at com.mongodb.internal.connection.SaslAuthenticator.access$500(SaslAuthenticator.java:43) ~[mongotest-1.0.0-SNAPSHOT-fat.jar:?] at com.mongodb.internal.connection.SaslAuthenticator$Continuator.onResult(SaslAuthenticator.java:267) ~[mongotest-1.0.0-SNAPSHOT-fat.jar:?] at com.mongodb.internal.connection.SaslAuthenticator$Continuator.onResult(SaslAuthenticator.java:247) ~[mongotest-1.0.0-SNAPSHOT-fat.jar:?] at com.mongodb.internal.connection.CommandHelper$1.onResult(CommandHelper.java:59) ~[mongotest-1.0.0-SNAPSHOT-fat.jar:?] at com.mongodb.internal.connection.CommandHelper$1.onResult(CommandHelper.java:53) ~[mongotest-1.0.0-SNAPSHOT-fat.jar:?] at com.mongodb.internal.connection.InternalStreamConnection$2$1.onResult(InternalStreamConnection.java:463) ~[mongotest-1.0.0-SNAPSHOT-fat.jar:?] at com.mongodb.internal.connection.InternalStreamConnection$2$1.onResult(InternalStreamConnection.java:440) ~[mongotest-1.0.0-SNAPSHOT-fat.jar:?] at com.mongodb.internal.connection.InternalStreamConnection$MessageHeaderCallback$MessageCallback.onResult(InternalStreamConnection.java:745) ~[mongotest-1.0.0-SNAPSHOT-fat.jar:?] at com.mongodb.internal.connection.InternalStreamConnection$MessageHeaderCallback$MessageCallback.onResult(InternalStreamConnection.java:712) ~[mongotest-1.0.0-SNAPSHOT-fat.jar:?] at com.mongodb.internal.connection.InternalStreamConnection$5.completed(InternalStreamConnection.java:582) ~[mongotest-1.0.0-SNAPSHOT-fat.jar:?] at com.mongodb.internal.connection.InternalStreamConnection$5.completed(InternalStreamConnection.java:579) ~[mongotest-1.0.0-SNAPSHOT-fat.jar:?] at com.mongodb.internal.connection.AsynchronousChannelStream$BasicCompletionHandler.completed(AsynchronousChannelStream.java:250) ~[mongotest-1.0.0-SNAPSHOT-fat.jar:?] at com.mongodb.internal.connection.AsynchronousChannelStream$BasicCompletionHandler.completed(AsynchronousChannelStream.java:233) ~[mongotest-1.0.0-SNAPSHOT-fat.jar:?] at sun.nio.ch.Invoker.invokeUnchecked(Invoker.java:127) ~[?:?] at sun.nio.ch.Invoker.invokeUnchecked(Invoker.java:282) ~[?:?] at sun.nio.ch.WindowsAsynchronousSocketChannelImpl$ReadTask.completed(WindowsAsynchronousSocketChannelImpl.java:581) ~[?:?] at sun.nio.ch.Iocp$EventHandlerTask.run(Iocp.java:387) ~[?:?] at sun.nio.ch.AsynchronousChannelGroupImpl$1.run(AsynchronousChannelGroupImpl.java:112) ~[?:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) ~[?:?] at java.lang.Thread.run(Thread.java:831) [?:?] ```
vietj commented 3 years ago

can you provide a fix ?

Ultranium commented 3 years ago

@vietj Just created a pull request.

vietj commented 3 years ago

thanks!