Expose multiprocessing pool

[LiAI-tech]

Is there an existing issue for this?

• [X] I have searched the existing issues

Describe the bug

D:\miniconda3\envs\sanic\python.exe D:/PythonProjects/untitled/11.py
[2023-06-21 09:28:51 +0800] [16972] [INFO] Sanic v22.9.1
[2023-06-21 09:28:51 +0800] [16972] [INFO] Goin' Fast @ http://localhost:9000
[2023-06-21 09:28:51 +0800] [16972] [INFO] mode: production, single worker
[2023-06-21 09:28:51 +0800] [16972] [INFO] server: sanic, HTTP/1.1
[2023-06-21 09:28:51 +0800] [16972] [INFO] python: 3.8.15
[2023-06-21 09:28:51 +0800] [16972] [INFO] platform: Windows-10-10.0.22621-SP0
[2023-06-21 09:28:51 +0800] [16972] [INFO] packages: sanic-routing==22.8.0, sanic-ext==22.9.1
[2023-06-21 09:28:53 +0800] [22192] [INFO] Sanic Extensions:
[2023-06-21 09:28:53 +0800] [22192] [INFO]   > injection [0 added]
[2023-06-21 09:28:53 +0800] [22192] [INFO]   > openapi [http://127.0.0.1:9000/docs]
[2023-06-21 09:28:53 +0800] [22192] [INFO]   > http 
[2023-06-21 09:28:53 +0800] [22192] [INFO]   > templating [jinja2==3.1.2]
[2023-06-21 09:28:53 +0800] [22192] [INFO] Starting worker [22192]
[1, 2, 3, 4, 7]
[2023-06-21 09:29:13 +0800] [22192] [ERROR] Exception occurred while handling uri: 'http://127.0.0.1:9000/is_prime'
Traceback (most recent call last):
  File "handle_request", line 94, in handle_request
  File "D:\PythonProjects\untitled\11.py", line 44, in api_is_prime
    future = process_pool.submit(is_prime, number_list)
  File "D:\miniconda3\envs\sanic\lib\concurrent\futures\process.py", line 645, in submit
    self._start_queue_management_thread()
  File "D:\miniconda3\envs\sanic\lib\concurrent\futures\process.py", line 584, in _start_queue_management_thread
    self._adjust_process_count()
  File "D:\miniconda3\envs\sanic\lib\concurrent\futures\process.py", line 608, in _adjust_process_count
    p.start()
  File "D:\miniconda3\envs\sanic\lib\multiprocessing\process.py", line 118, in start
    assert not _current_process._config.get('daemon'), \
AssertionError: daemonic processes are not allowed to have children

Code snippet

from sanic import Sanic
from concurrent.futures import ProcessPoolExecutor
import json
import math

process_pool = ProcessPoolExecutor()
app = Sanic("Extract")

# 判断是否是素数
def is_prime(n):
    if n < 2:
        return False
    if n == 2:
        return True
    if n % 2 == 0:
        return False
    sqrt_n = int(math.floor(math.sqrt(n)))

    for i in range(3, sqrt_n + 1, 2):
        if n % i == 0:
            return False
    return True

@app.post("/is_prime")
def api_is_prime(request):
    number_list = request.json["numbers"]
    print(number_list)

    # rsts = process_pool.map(is_prime, number_list)
    future = process_pool.submit(is_prime, number_list)
    rst = future.result()

    rst = json.dumps(
        dict(zip(number_list, rst))
    )
    return rst

if __name__ == '__main__':
    # process_pool = ProcessPoolExecutor()
    app.run(host="localhost", port=9000)

Expected Behavior

{"1": false, "2": true, "4": false, "5": true}

How do you run Sanic?

Sanic CLI

Operating System

windows

Sanic Version

22.9.1

Additional context

No response

[ahopkins]

I think the error is self-explanatory. You cannot open another process pool endlessly and nested. Either you need to drop Sanic into single process mode, or work with the existing tools that Sanic provides to build on top of that.

[james-che]

hi，Is this problem solved? @LiAI-tech

[vvaradarajan]

@ahopkins -- Found the same issue. Appreciate an explanation of this (i.e. what is the issue if both Sanic and the server python code, both use multiprocessing pools?)

[ahopkins]

I am working on a change that would expose a pool. Until then, maybe this will help: https://sanic.dev/en/guide/running/manager.html#on-the-fly-process-management