Open MarkReedZ opened 4 months ago
could you provide a full reproduction example?
@trim21 it's a concurrcy bug that not easy to repoduce all the time. It could be happen when your program in high loads.
@trim21 it's a concurrcy bug that not easy to repoduce all the time. It could be happen when your program in high loads.
I understand, can you provide a minimal example?
@trim21 I think the first example could reproduce this issue. But you may need try it multiple times
@trim21 I think the first example could reproduce this issue. But you may need try it multiple times
sadly I can't.
import asyncmy # 0.2.9
import fastapi
app = fastapi.FastAPI()
@app.on_event("startup")
async def startup_event():
app.mysql_pool = await asyncmy.create_pool(
autocommit=True,
minsize=4,
maxsize=20,
...
)
@app.get("/test")
async def test():
async with app.mysql_pool.acquire() as conn:
async with conn.cursor() as cur:
await cur.execute("...")
with uvicorn a:app --no-access-log
and hey -z 3m -c 16 http://127.0.0.1:8000/test
on win10/ubuntu
Confirm. 0.2.7 same problem. 0.2.5 stable
In my case using SQLAlchemy and FastAPI it always throws this error for this specific handler when querying for a large amount of rows. This still works locally for small dataset (dev database). Code obfuscated for legal reasons:
async def public_sitemap():
async with get_async_session() as session:
query = (
select(
SAC.name, # type: ignore
SAC.s.label('cs'), # type: ignore
SAC.ca.label('cca'), # type: ignore
SAC.ua.label('cua'), # type: ignore
SA.s.label('as'), # type: ignore
SAR.y,
SA.ca,
SA.ua
)
.select_from(SAC)
.join(SACM)
.join(SA)
.join(SAR)
.where(filter_cond())
)
categories = {}
for row in (await session.execute(query)).fetchall():
if row.cs not in categories:
categories[row.cs] = {
"n": row.n,
...
"as": []
}
categories[row.cs]["as"].append({
"t": row.t,
...
})
return {
"categories": list(categories.values())
}
While other handlers with similar or even heavier SQLAlchemy usage perform without any issue. I switched to aiomysql
and it doesn't seem to error out anymore.
Running wrk against a web page with a simple select caused my logs to fill up with errors. Aiomysql has no problem with the same exact code. I'll create a simple test later to reproduce this.
Using create_pool
The page handler is just a simple select
The wrk cmd