search

reverb / wordnik-oss

Wordnik Open Source Software
168 stars 39 forks source link

Unexpected EOF #20

Open ubiquitousthey opened 11 years ago

ubiquitousthey commented 11 years ago

I am getting the following exception on all my oplog files when I use the ReplayUtil.

java.io.IOException: unexpected EOF at org.bson.BasicBSONDecoder$BSONInput._need(BasicBSONDecoder.java:327) at org.bson.BasicBSONDecoder$BSONInput.readInt(BasicBSONDecoder.java:340) at org.bson.BasicBSONDecoder._decode(BasicBSONDecoder.java:74) at org.bson.BasicBSONDecoder.decode(BasicBSONDecoder.java:57) at org.bson.BasicBSONDecoder.readObject(BasicBSONDecoder.java:42) at com.wordnik.system.mongodb.ReplayUtil.run(ReplayUtil.java:154) at com.wordnik.system.mongodb.ReplayUtil.main(ReplayUtil.java:70)

I am restoring a snapshot and then replaying five oplogs. I have programmatically verified that the data is being restored without gaps, so I am confident the data is there.

Have you seen that? Do you have any suggestions for me?

Here is a link to one of the oplogs. http://static.integrityops.net.s3.amazonaws.com/1381330862_oplog.bson.gz

fehguy commented 11 years ago

I've only seen that when the data is corrupt, but if that's on every line, I expect it's not the case. Can you see what is in the oplog being replied? You can write the file as JSON instead of BSON to see.

Are you on windows or a mac?

ubiquitousthey commented 11 years ago

I think I have narrowed it down. When I replay it as a .bson file it works without error, but when I replay it as a .bson.gz file, I get the error. For now, I'll gunzip the files manually.

There must be something about the GZipInputStream / BufferedInputStream combination is not feeding the data to the BSONDecoder right.