Memory leak

[ZhiWeiCui]

Hi:

Thank you so much for providing an amazing package.

Use this package according to the method you said. When the route method is executed multiple times, it is found that the memory keeps increasing. I don't know if you have encountered this situation.

My test code is as follows. When I put router = OSRM(OSRM_DATASET.encode('utf-8'), contraction=True) in the for loop, the memory is stable. I don't want to instantiate OSRM in every loop based on performance. Would you like to ask if I have other options ?

Looking forward to your reply!

from pyosrm import PyOSRM, Status
from dpsolver.Util import randomly_location
import subprocess, os, re

from osrmcpy import OSRM, Coordinate

def convert_size(size):
    if size <1024:
        return size
    elif (size >= 1024) and (size < (1024 * 1024)):
        return "%.2f KB"%(size/1024)
    elif (size >= (1024*1024)) and (size < (1024*1024*1024)):
        return "%.2f MB"%(size/(1024*1024))
    else:
        return "%.2f GB"%(size/(1024*1024*1024))

def process_info():
    pid = os.getpid()
    res = subprocess.getstatusoutput('ps aux|grep ' + str(pid))[1].split('\n')[0]

    p = re.compile(r'\s+')
    l = p.split(res)
    info = {'user': l[0],
            'pid': l[1],
            'cpu': l[2],
            'mem': l[3],
            'vsa': convert_size(int(l[4])*1024),
            'rss': convert_size(int(l[5])*1024),
            'start_time': l[6]}
    return info

# location
location_start = randomly_location(10000)
location_end = randomly_location(10000)

# osrmcpy
DATA_DIR = '/mnt/d/Data/osrm/bicycle/ch'
OSRM_DATASET = os.path.join(DATA_DIR, 'china-latest.osrm')
router = OSRM(OSRM_DATASET.encode('utf-8'), contraction=True)

# pyosrm
# router = PyOSRM(use_shared_memory=True, algorithm='CH')

for start, end in zip(location_start.values(), location_end.values()):
    result = router.route([start[::-1], end[::-1]])
    print(process_info())

[cayetanobv]

Thanks, @ZhiWeiCui. I have tried to reproduce it and you are right. I need more time to investigate the problem. If you have any new info on this, please let me know. Thanks again.

[ZhiWeiCui]

Thank you for your reply. These are two related issues I submitted in Project-OSRM/osrm-backend and daniel-j-h/libosrmc. I hope they can be helpful to solve this problem. https://github.com/daniel-j-h/libosrmc/issues/23
https://github.com/Project-OSRM/osrm-backend/issues/6051

[cayetanobv]

Thank you for your reply. These are two related issues I submitted in Project-OSRM/osrm-backend and daniel-j-h/libosrmc. I hope they can be helpful to solve this problem. daniel-j-h/libosrmc#23 Project-OSRM/osrm-backend#6051

Thank you!

[ZhiWeiCui]

Hi cayetanobv, is there any progress? I am looking forward to you to resolve this issue as soon as possible.

[cayetanobv]

Thank you for your interest @ZhiWeiCui but unfortunately, I hadn't enough time now to solve the problem. I'm almost sure that the problem is with route destructor after each iteration. I didn't notice this problem because I don't need to compute routes massively; to solve huge cost matrices (tipically used in VRP problems), is more convenient to use table method. Do you really need the geometry of all these routes? If not, you should use table instead route.

[ZhiWeiCui]

Thank you for taking time out of your busy schedule to reply me. My test results show that table and route have the same problem. I am not sure if this problem is caused by osrmcpy or libosrmc. I have some experience in python development, but I don't know anything about C++.