tr8dr / .Net-Bridge

.NET Bridge allowing Python and R to access .NET libraries, running either locally or remotely
Apache License 2.0
28 stars 16 forks source link

_cache_io dictionary gets corrupted #15

Open daveyboy103 opened 2 years ago

daveyboy103 commented 2 years ago

Sometimes the internal cache dictionaries for ObjectId and objects gets corrupted and the program tries to access a key that does not exist. Currently only fixable by restarting the server.

CLRObjectProxy.cs

tr8dr commented 2 years ago

I am no longer maintaining this (aside from a JVM version). Is this from R or python? I think there are scenarios in R where it can release an object early, where it should not. If you don't mind objects piling up, you could change the code to cause the Release() function in CLRObjectProxy to do nothing.

I don't see any flaw in the C# logic. The object IDs will be unique (and increasing). The only way the object reference disappears is from the release call from R. Have seen this behavior in R myself (back when I was using R - have since switched to python).

Could alternatively create a list or queue of objects pending release, and delete the oldest ones (to avoid the R problem). The logic would be as follows: In the Release function do:

create a member variable called _pending_delete or something like that

_pending_delete.Add (obj);
if (_pending_delete.Length > somethreshold)
    <delete 1/2 of that threshold starting from oldest>
   <trim list or queue to remove the items no longer pending>

Somewhat more complex would be to:

daveyboy103 commented 2 years ago

Hi Jonathan,

Thanks for getting back to me,

Looking at some methods for accessing .NEt stuff from Python and came across this. Works pretty well and I have addressed a few threading issues too.

The version fix would be useful to have integrated if you get a few minutes to do so.

I think what I was seeing was a concurrency issue as I was hitting to from several different sources local and remote. Making it a ConcurrentDictionary seems to have sorted that out.

I will create a PR for it too.

Great work by the way, Very inventive and very solid.

Do you mind if I run with this?

David

On 29 Nov 2021, at 15:26, Jonathan Shore @.***> wrote:

I am no longer maintaining this (aside from a JVM version). Is this from R or python? I think there are scenarios in R where it can release an object early, where it should not. If you don't mind objects piling up, you could change the code to cause the Release() function in CLRObjectProxy to do nothing.

Could also create a list or queue of objects pending release, and delete the oldest ones (to avoid the R problem). The logic would be as follows: In the Release function do:

create a member variable called _pending_delete or something like that

_pending_delete.Add (obj); if (_pending_delete.Length > somethreshold) <delete 1/2 of that threshold starting from oldest>

I don't see any flaw in the C# logic. The object IDs will be unique (and increasing). The only way the object reference disappears is from the release call from R. Have seen this behavior in R myself (back when I was using R - have since switched to python). — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub , or unsubscribe . Triage notifications on the go with GitHub Mobile for iOS or Android .
tr8dr commented 2 years ago

Please do, feel free to submit fixes or enhancements - I will merge. I can just add you as permissioned for the project. I have improved the python side of this considerably, but against the JVM (alas have not had the time to release publicly).

daveyboy103 commented 2 years ago

Many thanks Jonathan

Not on the Java side myself, more from the .NET side

David

On 29 Nov 2021, at 17:21, Jonathan Shore @.***> wrote:

Please do, feel free to submit fixes or enhancements - I will merge. I can just add you as permissioned for the project. I have improved the python side of this considerably, but against the JVM (alas have not had the time to release publicly).

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/tr8dr/.Net-Bridge/issues/15#issuecomment-981847381, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABJJRINLOQ6KKGR7D4QG2Q3UOOZCPANCNFSM5I7HKZ2Q. Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

tr8dr commented 2 years ago

Added you to the repo -- feel free to fix, add functionality, etc.

daveyboy103 commented 2 years ago

Thanks

On 29 Nov 2021, at 17:25, Jonathan Shore @.***> wrote:

Added you to the repo -- feel free to fix, add functionality, etc.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/tr8dr/.Net-Bridge/issues/15#issuecomment-981850348, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABJJRILEJNQNMD3FH6LES53UOOZO3ANCNFSM5I7HKZ2Q. Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.