search

hpgrahsl / kryptonite-for-kafka

Kryptonite for Kafka is a client-side 🔒 field level 🔓 cryptography library for Apache Kafka® offering a Kafka Connect SMT, ksqlDB UDFs, and a standalone HTTP API service. It's an ! UNOFFICIAL ! community project
83 stars 6 forks source link

Option to use funqy-http-kryptonite with single field values without requiring to wrap strings in quotes #21

Closed joshuagrisham-karolinska closed 6 months ago

joshuagrisham-karolinska commented 8 months ago

In the end this is a bit of a "nice to have" but it would be quite helpful and less confusing for us if there was a way to use the funqy-http-kryptonite with single value strings without having to wrap the value in quotes, and to receive the value back without it being wrapped in quotes.

E.g. if I run an instance of the server and try to encrypt a value like this:

$ curl --data "test" http://localhost:9080/encrypt/value

Then the response is an empty HTTP 400 error and there is an exception logged in the container:

2024-03-07 15:43:55,364 ERROR [io.qua.funqy] (vert.x-eventloop-thread-3) Failed to unmarshal input: com.fasterxml.jackson.core.JsonParseException: Unrecognized token 'test': was expecting (JSON String, Number, Array, Object or token 'null', 'true' or 'false')
 at [Source: (io.netty.buffer.ByteBufInputStream); line: 1, column: 5]
        at com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:2418)
        at com.fasterxml.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:759)
        at com.fasterxml.jackson.core.json.UTF8StreamJsonParser._reportInvalidToken(UTF8StreamJsonParser.java:3693)
        at com.fasterxml.jackson.core.json.UTF8StreamJsonParser._reportInvalidToken(UTF8StreamJsonParser.java:3666)
        at com.fasterxml.jackson.core.json.UTF8StreamJsonParser._matchToken2(UTF8StreamJsonParser.java:3007)
        at com.fasterxml.jackson.core.json.UTF8StreamJsonParser._matchTrue(UTF8StreamJsonParser.java:2941)
        at com.fasterxml.jackson.core.json.UTF8StreamJsonParser._nextTokenNotInObject(UTF8StreamJsonParser.java:878)
        at com.fasterxml.jackson.core.json.UTF8StreamJsonParser.nextToken(UTF8StreamJsonParser.java:793)
        at com.fasterxml.jackson.databind.ObjectReader._initForReading(ObjectReader.java:357)
        at com.fasterxml.jackson.databind.ObjectReader._bindAndClose(ObjectReader.java:2095)
        at com.fasterxml.jackson.databind.ObjectReader.readValue(ObjectReader.java:1481)
        at io.quarkus.funqy.runtime.bindings.http.VertxRequestHandler.lambda$handle$2(VertxRequestHandler.java:107)
        at io.vertx.core.impl.future.FutureImpl$1.onSuccess(FutureImpl.java:91)
        at io.vertx.core.impl.future.FutureBase.lambda$emitSuccess$0(FutureBase.java:54)
        at io.vertx.core.impl.EventLoopContext.execute(EventLoopContext.java:86)
        at io.vertx.core.impl.DuplicatedContext.execute(DuplicatedContext.java:163)
        at io.vertx.core.impl.future.FutureBase.emitSuccess(FutureBase.java:51)
        at io.vertx.core.impl.future.FutureImpl.tryComplete(FutureImpl.java:211)
        at io.vertx.core.impl.future.PromiseImpl.tryComplete(PromiseImpl.java:23)
        at io.vertx.core.http.impl.HttpEventHandler.handleEnd(HttpEventHandler.java:79)
        at io.vertx.core.http.impl.Http1xServerRequest.onEnd(Http1xServerRequest.java:581)
        at io.vertx.core.http.impl.Http1xServerRequest.lambda$pendingQueue$1(Http1xServerRequest.java:130)
        at io.vertx.core.streams.impl.InboundBuffer.handleEvent(InboundBuffer.java:239)
        at io.vertx.core.streams.impl.InboundBuffer.write(InboundBuffer.java:129)
        at io.vertx.core.http.impl.Http1xServerRequest.handleEnd(Http1xServerRequest.java:562)
        at io.vertx.core.impl.EventLoopContext.execute(EventLoopContext.java:76)
        at io.vertx.core.impl.DuplicatedContext.execute(DuplicatedContext.java:153)
        at io.vertx.core.http.impl.Http1xServerConnection.onEnd(Http1xServerConnection.java:191)
        at io.vertx.core.http.impl.Http1xServerConnection.onContent(Http1xServerConnection.java:181)
        at io.vertx.core.http.impl.Http1xServerConnection.handleOther(Http1xServerConnection.java:161)
        at io.vertx.core.http.impl.Http1xServerConnection.handleMessage(Http1xServerConnection.java:149)
        at io.vertx.core.net.impl.ConnectionBase.read(ConnectionBase.java:157)
        at io.vertx.core.net.impl.VertxHandler.channelRead(VertxHandler.java:153)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
        at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93)
        at io.netty.handler.codec.http.websocketx.extensions.WebSocketServerExtensionHandler.channelRead(WebSocketServerExtensionHandler.java:99)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
        at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
        at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346)
        at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
        at io.vertx.core.http.impl.Http1xOrH2CHandler.end(Http1xOrH2CHandler.java:61)
        at io.vertx.core.http.impl.Http1xOrH2CHandler.channelRead(Http1xOrH2CHandler.java:38)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
        at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
        at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
        at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
        at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
        at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788)
        at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724)
        at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562)
        at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
        at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.base/java.lang.Thread.run(Thread.java:829)

But if we instead wrap the value in quotes then it works (though the value we receive back is also wrapped in quotes):

$ curl --data "\"test\"" http://localhost:9080/encrypt/value
"JwFp/j8gmvvsp15jTZOGdSp89FQzUmevY0ERBzBIYNvw3UG8MXYaDDCybXktdGVzdC1rZXktdrFrsQ=="

I can understand that it seems to be expecting "valid json" as the input and returning "valid json" as the output, but honestly if we had a way avoid wrapping the values in quotes and then that the value we receive back were also not wrapped in quotes, then it would remove pre processing and post processing that we need to do every time we are using this service currently 😎

Maybe it could be a separate endpoint to specify we are sending a string e.g. /encrypt/string or /encrypt/value/string which could handle the pre- and postprocessing logic for us? Or any other idea would be welcome, of course!

hpgrahsl commented 8 months ago

Hi @joshuagrisham-karolinska.

Your observation is correct. The fact that the provided API is expecting valid JSON makes it necessary to escape strings in the way you show it above. The JSON standard describes valid tokens and in case such tokens are used "standalone" a string token has to be explicitly quoted. Handling this in a different way would break the JSON standard semantics and potentially also existing tooling that is used against the API could break if this gets changed. I hope this explains why I'm rather hesitant to do that.

Irrespective of this, I'd be interested to learn what you are doing with the API and how you are using it? In my experience and based on the usage of others, it's very rarely if at all necessary to actually encrypt / decrypt single JSON tokens. Do you really have that requirement? Also, could you maybe instead make use of other API endpoints (e.g. a map with just one entry, or using the value with config endpoint) to achieve your goals?

hpgrahsl commented 6 months ago

Closing this for now due to lack of activity. Feel free to re-open if more discussion is appreciated.