squeaky-pl / japronto

Screaming-fast Python 3.5+ HTTP toolkit integrated with pipelining HTTP server based on uvloop and picohttpparser.
MIT License
8.61k stars 581 forks source link

i used aiomysql with singleton mode #177

Closed junbaibai0719 closed 2 years ago

junbaibai0719 commented 4 years ago

when i set multi process, only one run.

code

import aiomysql
import threading

class DbUtil(object):
    _instance_lock = threading.Lock()
    def __init_(self):
        pass

    async def __new__(cls, *args, **kwargs):
        if not hasattr(DbUtil, "_instance"):
            with DbUtil._instance_lock:
                if not hasattr(DbUtil, "_instance"):
                    DbUtil._instance = await aiomysql.create_pool(host='127.0.0.1', port=3306,
                           user='root', password='ll980810',
                           db='mysql')            
        return DbUtil._instance

from japronto import Application
import aiomysql
import asyncio
import traceback
import os

async def hello(request):
    res = "hello world"
    try:
        pool = await DbUtil()
        print(os.getpid())
        async with pool.acquire() as conn:
            async with conn.cursor() as cur:
                await cur.execute("SELECT * from user;")
                data = await cur.fetchone()
                res = data[1]
    except Exception as e:
        print(e)
        traceback.print_exc()
    return request.Response(text=res)

def index(request):
    print(os.getpid())
    return request.Response(text="hello world")

app = Application()

app.router.add_route('/hello', hello)
app.router.add_route('/', index)

app.run(port=8081, worker_num=6, debug=False)

while i request 127.0.0.1:8081/, proccessor id

27719 27716 27710 27722 27719 27716 27710 27722 27719 27710 27716 27722 27719 27710 27716

while i request 127.0.0.1:8081/hello, proccessor id

27711 27711 27711 27711 27711 27711 27711 27711 27711 27711 27711 27711 27711 27711 27711 27711

squeaky-pl commented 4 years ago

I didn't run the code but this most probably happens because _instance_lock = threading.Lock() gets created in the master server process and then gets shared between all the spawned children processes.

You can try removing with DbUtil._instance_loc: and see what happens.

junbaibai0719 commented 4 years ago

I see!I always thought the lock would be created in children process,It is actually created when the class is defined. I use asyncio.Lock and succeed. If using threading.Lock,the children process will blocking themselves, because DbUtil._instance = await aiomysql.create_pool.