fix(weave): Handle exceptions raised by scoring funcs during evaluation

wandb / weave

Weave is a toolkit for developing AI-powered applications, built by Weights & Biases.

https://wandb.me/weave

Apache License 2.0

604 stars 43 forks source link

fix(weave): Handle exceptions raised by scoring funcs during evaluation #1866

Open andrewtruong opened 2 weeks ago

andrewtruong commented 2 weeks ago

Resolves:

https://wandb.atlassian.net/browse/WB-18523

This PR changes Evaluation.evaluate to not crash the user script if an exception is raised in a scoring function. Instead, the error is captured and relayed to the user as part of the eval

circle-job-mirror[bot] commented 2 weeks ago

Preview this PR with FeatureBee: https://beta.wandb.ai/?betaVersion=aca96989d06d25f00637c83622510cfbb31fd4ef