search

replicate / replicate-go

Go client for Replicate
https://replicate.com
Apache License 2.0
65 stars 9 forks source link

When ctx is canceled, StreamPrediction will throw an error #42

Open yan5xu opened 7 months ago

yan5xu commented 7 months ago

If the ctx used by StreamPrediction is canceled, StreamPrediction will throw an error. The expectation is that StreamPrediction stops processing without throwing an error.

mattt commented 6 months ago

Hi @yan5xu. Thanks for reporting this. It was our intention for context cancellation to return without an error. Can you share what error is being produced?

yan5xu commented 6 months ago

Hi @yan5xu. Thanks for reporting this. It was our intention for context cancellation to return without an error. Can you share what error is being produced?

lol, below is the error msg I collected in the log

failed to send request: Get \"https://streaming-api.svc.us.c.replicate.net/v1/streams/fge5z76wnqszkj5rqi3cxr4w6bwirlju6vxfvqxeoaiqnrij7sga\": context canceled

mattt commented 6 months ago

@yan5xu Thank you for clarifying. If the error happens because the context is canceled for the original HTTP request for the stream, I think it's reasonable to bubble that up. And on reflection, I think it's less confusing if context errors are always returned, even when used to cancel a stream. Would you agree with that?

In the meantime, my recommendation would be to check for context cancellation errors and handle those differently:

if errors.Is(err, context.Canceled) {
    // ...
}