replicate / cog

Containers for machine learning
https://cog.run
Apache License 2.0
7.45k stars 512 forks source link

How to stream via webhooks? #1716

Open schananas opened 1 month ago

schananas commented 1 month ago

Based on documentation, if I set webook filter with "output", If I set Iterator[str] as return type and yield strings, I would expect that my webhook is called on each yield, but that is not the case, yielded output is included on the completion webhook call. As well I didnt set webhook filter "logs" but logs are still included in the end webhook call.

schananas commented 1 month ago

My call:

echo '{
  "input": {},
  "webhook":"http://localhost:8080",
  "webhook_events_filter": ["start", "output" , "completed"],
  "stream": true
}' | curl -X POST -H "Content-Type: application/json" -H "Prefer: respond-async" --data-binary @- http://localhost:5000/predictions

my code:

    def predict(
        self,
  ...
    ) -> Iterator[str]:

yield 'test'

...prediction

I get 'test' in output field, but after whole prediction has finished.