Open JamesChenX opened 2 years ago
After research, we can write a plugin for protoc
to add our own code for proto models:
ExtendedString
that stores the raw UTF8 bytes for MongoDB and UTF16/LATIN1 bytes[] for turms-plugin if they are enabledBut it needs about 5 days to finish and test, we just keep it simple and unchanged currently in v1.0
For user texts:
ByteBuf
first when receivingcom.google.protobuf.CodedInputStream
to decode theByteBuf
toTurmsRequest
proto model, and usescom.google.protobuf.CodedInputStream.UnsafeDirectNioDecoder#readStringRequireUtf8
to decode thestring
data, and then usescom.google.protobuf.Utf8.UnsafeProcessor#decodeUtf8Direct
to decode.char[]
, and copy the direct buffer to the heap. copy but it's reasonable.char[]
will be passed tonew String(resultArr, 0, resultPos)
. So copy again.CreateMessageRequest
, turms-service will callString#getBytes
to write the bytes todirect buffer
to flush theMessage
record to MongoDB. So copy again.String#getBytes
to remove non-printable characters of user input texts to ensure it's safe to log them, and then write the finalByteBuffer
to append to console or the log file. So copy twice.~~ We don't log the user input texts for security and better performance currently. ...If we can avoid using
String
and just usebyte[]
, we can save a lot of memory.