Open gthaker opened 1 month ago
Hi @gthaker. I'm sorry that you're occasionally having issues running meta/meta-llama-3-70b-instruct
. Can you share a link to or logs for any failed predictions?
replicate.exceptions.ModelError
indicates a problem with the model, so there's nothing to be done with the client itself. What you could try to do is wrap your call in a try/catch
and implement retry logic to suit your needs.
Can I send you the link in a private message? Prob. best not share the query with everyone.
GHT
On Sun, Jun 2, 2024 at 5:49 AM Mattt @.***> wrote:
Hi @gthaker https://github.com/gthaker. I'm sorry that you're occasionally having issues running meta/meta-llama-3-70b-instruct. Can you share a link to or logs for any failed predictions?
replicate.exceptions.ModelError indicates a problem with the model, so there's nothing to be done with the client itself. What you could try to do is wrap your call in a try/catch and implement retry logic to suit your needs.
— Reply to this email directly, view it on GitHub https://github.com/replicate/replicate-python/issues/306#issuecomment-2143777969, or unsubscribe https://github.com/notifications/unsubscribe-auth/AC7QKFDOBCNV7UHGBW5IAJLZFLTEHAVCNFSM6AAAAABIUKYSPWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNBTG43TOOJWHE . You are receiving this because you were mentioned.Message ID: @.***>
Have the same issue here. The error still exists not only for meta/meta-llama-3-70b-instruct, but also for model mixtral-8x7b-instruct-v0.1.
I get this error. If I rerun it usually goes thru fine but dies again down the road. (I am trying 300 total question from a professional exam.) I had no trouble running this with llama-2-70b model, i did it many times.