mcneel / compute.rhino3d

REST geometry server based on RhinoCommon and headless Rhino
Other
284 stars 182 forks source link

compute.geometry.exe not releasing memory #468

Open jeffswilkinson opened 2 years ago

jeffswilkinson commented 2 years ago

The compute.geometry.exe process spawned by rhino compute running under IIS does not release memory and eventually crashes the server. I have coded a workaround by checking for the working set size each time I send a grasshopper request to the compute node. When the working set gets over 10GB I kill the process and restart the app pool. image

jeffswilkinson commented 2 years ago

I am using the following settings in web.config because it was spawning 4 processes but it would only every use one and it never recycled any of the processes or shut them down after. Also, my python application which is doing the request to the compute node (running locally) was timing out waiting for the compute.geometry.exe to startup. In this configuration, it still doesnt spawn the compute.geometry.exe process until you connect with valid API connection, so I am having to do a request to the /healthcheck as a way to get it to spawn the process and then do a sleep timer for 20 seconds to wait for it to initialize before it will start taking requests.

<aspNetCore processPath=".\rhino.compute.exe" arguments="--port 80 --childcount 1 --spawn-on-startup" stdoutLogEnabled="true" stdoutLogFile="..........\logs\LogFiles\W3SVC1\"

This is an example of what I am having to do to get this to work for me (python).

code from my application I have to run everything before I process a grasshopper script:

geometry_instance,compute_recycle = b2f_grasshopper.check_geometry_instance()
if geometry_instance == 0 or compute_recycle == True:
                    b2f_grasshopper.recycle_rhino_compute()

code from b2f_grasshopper.py

compute_geometry = "compute.geometry.exe"
rhino_compute = "RhinoComputeAppPool"

def stop_compute_geometry():
    for proc in psutil.process_iter():
        if proc.name() == compute_geometry:
            proc.kill()
def check_geometry_instance():
    geometry_instance = 0
    total_compute_memory = 0
    compute_recycle = False
    for proc in psutil.process_iter():
        if proc.name() == compute_geometry:
            geometry_instance += 1
            mem_info = proc.memory_info()
            total_compute_memory += mem_info.wset
            if mem_info.wset > 10000000000:
                compute_recycle = True
    if total_compute_memory > 15000000000:
       compute_recycle = True
    return geometry_instance, compute_recycle

def recycle_rhino_compute():
    stop_compute_geometry()
    if pool.exists(rhino_compute):
        pool.restart(rhino_compute)
    else:
        pool.start(rhino_compute)
    compute_rhino3d.Util.url = b2f_constants.RHINO3D_URL
    compute_rhino3d.Util.apiKey = b2f_constants.RHINO3D_KEY
    health_url = compute_rhino3d.Util.url + "healthcheck"
    headers = {
        'RhinoComputeKey': f'{b2f_constants.RHINO3D_KEY}'
    }
    try:
        health_check = requests.get(health_url, headers=headers, timeout=30)
        time.sleep(20)
    except:
        return False
    else:
        if health_check.text == 'Healthy':
            return True
sanchez commented 1 year ago

Possible duplicate of #396 ?