search

eth-sri / lmql

A language for constraint-guided and efficient LLM programming.
https://lmql.ai
Apache License 2.0
3.64k stars 197 forks source link

Fix batched openai request worker race #340

Closed kentjhall closed 5 months ago

kentjhall commented 6 months ago

This is my current local fix for addressing the async batch openai assertion failure I've experienced. See Issue #339.

benwhalley commented 6 months ago

Thanks for this. Can confirm it fixed the same issue for me.

lbeurerkellner commented 5 months ago

Thanks for this, looks good to me.