search

zbjornson / MongoDBLink

MongoDB driver for Mathematica
http://zbjornson.github.io/MongoDBLink
MIT License
17 stars 5 forks source link

Speed up writes by moving serialization to Java #12

Closed zbjornson closed 5 years ago

platomaniac commented 8 years ago

Let me know when you have the serialization. I will anyways come and check for myself...in case I get no notification :)

platomaniac commented 7 years ago

Hi! Any guess when the serialization thingy could be available. Will help my project a lot.

zbjornson commented 7 years ago

@platomaniac I have most of it done (https://github.com/zbjornson/MongoDBLink/commit/90329f2b565110f5110c0c134c22a8a8dfe772d1) and it benchmarks about 20x faster. I have a bit more work to do on it still but should finish up in the next few days.

zbjornson commented 7 years ago

More left on this than I'd hoped. Down to 3 (out of 47) test failures.

zbjornson commented 7 years ago

@platomaniac all tests are passing. Do you feel like trying it out before I merge it into master please?

tmp = URLSave["https://raw.githubusercontent.com/zbjornson/MongoDBLink/java-serialization/MongoDBLink.zip"];
dest = FileNameJoin[{$AddOnsDirectory, "Applications"}];
ExtractArchive[tmp, dest];
DeleteFile[tmp];
Print["Installed MongoDBLink to " <> dest]

InsertDocument will now also do bulk inserts:

docs = {"a" -> #, "b" -> 2 #} &/@ Range[5];
InsertDocument[coll, docs]
platomaniac commented 7 years ago

@zbjornson I am giving it a proper test-drive today. Will let you know in next 5-10 hrs. I am excited to test it. Gr8 job man 👍

zbjornson commented 7 years ago

Thanks!

BTW you'll probably need to delete your existing version before upgrading: SystemOpen[FileNameJoin[{$AddOnsDirectory, "Applications"}]] and delete the MongoDBLink folder.

platomaniac commented 7 years ago

Hey did that already... ;) by the way you tested with latest MongoDB...?

zbjornson commented 7 years ago

I'm on 3.2 locally but I upgraded the java driver so it should work with 2.6 through 3.4, and java 6 and later.

zbjornson commented 7 years ago

@platomaniac how did the test go?

zbjornson commented 7 years ago

@platomaniac one more ping before I go ahead and merge the java-serialization branch. Please let me know if you found any issues with it.

platomaniac commented 7 years ago

@zbjornson hey sorry for late answer. There was no issue with the serialization for me. It can be merged I suppose.