search

guardrails-ai / guardrails

Adding guardrails to large language models.
https://www.guardrailsai.com/docs
Apache License 2.0
3.87k stars 289 forks source link

Fix typing for streaming #507

Closed CalebCourier closed 1 week ago

CalebCourier commented 9 months ago

Returning a union from the __call__ method is the best solution we have with the current implementation, but this is still problematic. Besides unhelpful intellisense, users using pyright or mypy for typing will have to cast their result and know what to cast it too. Because the return type of __call__ is currently dependent on the value of the stream kwarg, not just its type, it is not possible to properly overload the method signatures to account for it.

I think the better option would be for streaming to only be available through its own guard method i.e. guard.stream. This way there's no question what the return type would be. This would also put it more in line with our existing pattern since guard.parse is also a distinct method.

_Originally posted by @CalebCourier in https://github.com/guardrails-ai/guardrails/pull/497#discussion_r1424188374_

thekaranacharya commented 9 months ago

Great point @CalebCourier , we can add this in the next iteration of streaming alongwith reask and async support.

github-actions[bot] commented 3 weeks ago

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 14 days.

github-actions[bot] commented 1 week ago

This issue was closed because it has been stalled for 14 days with no activity.