tagomoris
commented
8 years ago @andreaseger What's happen when you downgrade (revert) update of mongodb driver?
Closed andreaseger closed 8 years ago
tagomoris
commented
8 years ago @andreaseger What's happen when you downgrade (revert) update of mongodb driver?
andreaseger
commented
8 years ago When we downgrade mongo to 2.1.2 which also downgrades bson from 4.1.x to 3.2.x msgpack is working again.
Problem is we really should update to the latest mongo driver because of compatibility and bugfixes (mostly related to filling our mongo logs)
iconara
commented
8 years ago Some information about Java version, and what the differences are between your local system and the system where you see the error, would be helpful.
andreaseger
commented
8 years ago our production system has the following java version
java version "1.8.0_45"
Java(TM) SE Runtime Environment (build 1.8.0_45-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.45-b02, mixed mode)locally I have
openjdk version "1.8.0_92"
OpenJDK Runtime Environment (build 1.8.0_92-b14)
OpenJDK 64-Bit Server VM (build 25.92-b14, mixed mode)Maybe I find some time tomorrow to run some more tests regarding this directly on one of our production boxes.
andreaseger
commented
8 years ago OK Sorry it took me a while to get back to this but here is what I found out and now I kinda understand why a mongo update was able to influence msgpack behaviour:
during requests in question we load some entities from mongodb.
Lets say they look like this:
{
"position":{
"fastest_mi": 1462790020205,
"fastest_km": 1462790086203,
"fastest5k" : "none"
}
}In the previous version of mongo/bson(3.x) the numbers here were - as one would expect - of type Fixnum and msgpack had no issue packing them.
Now with the newer version of mongo/bson(4.x) these numbers are all of a sudden of type Bignum although they would fit perfectly into a Fixnum.
Given that the promotion and demotion of Fixnums to Bignums should happen automatically I assume this has maybe something to do with bson using plain java and maybe on the context switch something isn't working as expected. I'll open another issue with the bson/mongo people and ask why this is the case.
But anyway I'm guessing somewhere in the packing logic of msgpack this causes some edge case where the logic expects that the Bignum really is a Bignum and has a certain size.
iconara
commented
8 years ago There's definitely a bug in msgpack-ruby here, it's just a question of what input that causes it. You should never get that kind of exception, if there's something that goes wrong in the encoding you should get a nice Ruby error, not a Java exception from deep down in ByteBuffer.
What would be very helpful at this point is to get a piece of code that reliably triggers the bug.
andreaseger
commented
8 years ago java/ruby interop is interesting ;) I managed to create a small gist to demonstrate the issue https://gist.github.com/andreaseger/eceb57dd80b6f9104eba452c553408db
obviously you want to run it with jruby - I'm not sure of another way to create a tiny Bignum
iconara
commented
8 years ago I ran your code and can confirm I get the error. I will take a look and try to fix it.
tagomoris
commented
8 years ago Here's another one:
MessagePack.pack((1 << 64).coerce(0)[0])
Java::JavaLang::IndexOutOfBoundsException:
from java.nio.Buffer.checkBounds(Buffer.java:567)v0.7.6 should solve this problem.
We have a rather big application running on jruby(1.7.15) which uses msgpack to send messages to rabbitmq.
We just wanted to update out mongodb driver which also updates the bson gem and we suddenly see lots of
Java::JavaLang::IndexOutOfBoundsExceptionfromjava.nio.Buffer.checkBounds(Buffer.java:567)which as you can see below in the - rather long - stacktrace is called from within msgpack.We are using msgpack version
0.7.4but looking at the changelog and commit for0.7.5nothing of relevance changed.I'm also not able to reproduce this on my local setup - msgpack is just able to pack whatever I throw at it. Has anyone maybe an insight why updating bson from v3 to v4 can have an impact on how msgpack operates? (could also be in the mongo gem but that doesn't have any native java code)
I opened the issue here and not in the mongo or bson gem as the error clearly comes from msgpack and not the other gems.