search

soravux / scoop

SCOOP (Scalable COncurrent Operations in Python)
https://github.com/soravux/scoop
GNU Lesser General Public License v3.0
625 stars 88 forks source link

SCOOP Memory Leak 1 #65

Open maharjun opened 7 years ago

maharjun commented 7 years ago

Since the termination signal via sendReply is not given upon the completion of local futures, consider the following situation.

  1. Root process spawns many futures
  2. Root process calls 'sendFuture' on these processes thereby creating entries in the broker process
  3. Root process fetches one of these futures later as part of 'execQueue.pop()'
  4. Root process executes said future and completes it
  5. Now being a local future that has completed, sendReply is not called and future is returned directly to parents executing coroutine. This means that STATUS_DONE is not sent to the broker process for the current future thereby not deleting it from the assigned_tasks queue. The only way this is handled currently is by the periodic STATUS_UPDATE which has its own set of issues (see #64 )

I would suggest that the STATUS_DONE be sent by all futures when they are completed and their result has been returned to the origin process.

soravux commented 7 years ago

Sorry you had this issue. I sadly don't have the time to fix it currently. I'll accept any pull request fixing memory accumulation problems.