playframework / play-plugins

CachePlugin
444 stars 161 forks source link

String Redis cache value is limited to length < 2^16 #182

Open ildar15 opened 5 years ago

ildar15 commented 5 years ago

String Redis cache is using DataInputStream / DataOutputStream readUTF / write UTF for string serialization. This is stepping on Java's inherent limitation of these functions, namely, they use 2 byte header and cannot process strings longer than 2^16 bytes see for example https://www.drillio.com/en/2009/java-encoded-string-too-long-64kb-limit/ I created a test case which exercises that limitation: simply using string of 70K bytes will fail string test.

ildar15 commented 5 years ago

This issue can be fixed in two way: 1) the easy way to allow large strings into the cache is simply use String as an Object. The drawback of this approach is that other tools besides Java (Scala) will not be able to access redis value directly. 2) another approach will be just to avoid using writeUTF and convert String to byte array directly. I tested number

Number 1 certainly works and I can submit patch for that solution. I will be testing second approach, which is more favorable (flexible) in my opinion. One can certainly try to support both provided we have correct default.

ildar15 commented 5 years ago

I forked the project here

https://github.com/ildar15/play-plugins

and implemented the solution to this problem. The writeUTF was eliminated, and the value is stored as text, without length header. The result is transparent to the client, but will invalidate existing string caches. You will need to skip the first two bytes of the returned result if you need to read old cache values.