[python] Empty failure message

[pamelafox]

I received an empty message on a failure, which is difficult to debug. I probably messed up something about using TypedDict, so I'll keep studying the examples, but I figure it should always show some sort of message, right?

Code is at: https://gist.github.com/pamelafox/ec8e229a774c5d7920a3b03bf38d1d07

[DanielRosenwasser]

It's unrelated to what you're running into, but one thing that is going to go wrong is that you probably meant to write

-         for question in result.value.questions:
+         for question in result.value["questions"]:
              print(question)

since the types are declared as TypedDicts - if you don't want that, you can always declare them as dataclasses instead.

[DanielRosenwasser]

The actual issue that's taking place is that the request is timing out since the underlying HTTP library has a timeout of 5s. The empty error message is due to the fact that the thrown ReadTimeout has no information. I can probably raise the default models to timeout after a longer duration.

This all adds some reckoning with TypeChat - how should failures be surfaced?

In the TypeScript version, a model can return a Success or an Error, or it can throw an exception in the event of a timeout or other events!

In TypeChat for Python, currently we only return Success and Failure. In the case of a timeout, the internal exception is completely swallowed up and converted into a failure.

I'll note that both of these are subtly wrong in behavior because when the translator receives a Error/Failure from a model, it may retry with the given error message. In the TypeScript version, if a non-200 status code is received too many times, the translator will retry even though the model already may have retried.

I'll try to chat with @ahejlsberg and others about this.

I was wrong about most of this.

[DanielRosenwasser]

https://github.com/microsoft/TypeChat/pull/226 increases the timeout. Still unclear whether we should just throw on most things instead.