search

guidance-ai / guidance

A guidance language for controlling large language models.
MIT License
19.14k stars 1.04k forks source link

[Bug] Make `_self_call_placeholder_` thread-safe #1034

Closed hudson-ai closed 1 month ago

hudson-ai commented 2 months ago

Puts _self_call_placeholder_ in threading.local() to prevent other threads from tricking the decorator into thinking we're in a recursive call.

Small note: the thread-local storage is in the wrapped function's closure, not an attribute on the function itself.

Tests would be nice, but annoyingly I can only stochastically reproduce the issue as it is due to a race condition. Increasing the number of threads and/or jobs might make the bug occur with higher probability though -- happy to add @mmoskal's repro if we're all okay with a slightly stochastic test...

Closes https://github.com/guidance-ai/guidance/issues/1031

hudson-ai commented 2 months ago

Ping @mmoskal (for some reason, I can't mark you as a reviewer... not in the org?)

codecov-commenter commented 2 months ago

:warning: Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 61.47%. Comparing base (edd8b87) to head (e406a70). Report is 1 commits behind head on main.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #1034 +/- ## ========================================== - Coverage 70.16% 61.47% -8.69% ========================================== Files 62 62 Lines 4471 4473 +2 ========================================== - Hits 3137 2750 -387 - Misses 1334 1723 +389 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

paulbkoch commented 1 month ago

LGTM. Really nice find and solution @hudson-ai.