search

ScottPJones / LibBSON.jl

libbson bindings for the Julia programming language
Other
9 stars 16 forks source link

BSONOID drops references #1

Closed JeffBezanson closed 9 years ago

JeffBezanson commented 9 years ago

This type allocates a julia Array, but stores it as a Ptr. The data is still reachable through the Ptr, but the Array object is not referenced and so will be freed by the GC. This is a serious memory bug.

This is a good example of why in v0.4 we're deprecating converting to Ptr; it makes it too easy to accidentally write a type like this without any warning.

ghost commented 9 years ago

Thanks for pointing this out Jeff. I understand the error but I am not sure about my fix for it simply because I have very little experience working with Julia. I am going to set up a pull request for what I think is a good fix, if you have a sec would you might taking a look? I'm hoping it will be obvious for you if it addresses the issue.

szalmaf commented 9 years ago

@JeffBezanson @pzion, has this problem been solved since March? It sounds like a pretty serious issue.

ghost commented 9 years ago

I'm guessing that it hasn't been fixed, as I opened a pull request for what I thought might fix it but then got a bunch of errors from CI (at least that's what I can piece together, it was a while ago).

szalmaf commented 9 years ago

It seems like you have fixed this memory leak problem, as far as I can tell now, by introducing references, not just pointers to Julia objects.

I am having another problem now: I cannot pull more than a certain amount of data from a Mongo database, say I can pull 350 documents, but I cannot pull 351 documents. It is really mysterious, my Jupyter session hangs when I for-loop iterate over a cursor with more than 350 documents in this case.

szalmaf commented 9 years ago

Update: Apparently, the for loop does not work when you have a println in it in Jupyter in the above case. So the problem is not related to either LibBSON.js or Mongo.jl. Sorry for the inadvertent spamming.

ghost commented 9 years ago

I put in fixes last night for this issue -- I had a consistent repro on my system so it was easy to fix.