[Inaccuracy] Insufficient max_tokens_to_sample for the Anthropic streaming example in bedrock_basics

aws-samples / amazon-bedrock-workshop

This is a workshop designed for Amazon Bedrock a foundational model service.

https://catalog.us-east-1.prod.workshops.aws/workshops/a4bdb007-5600-4368-81c5-ff5b4154f518/en-US/20-intro

MIT No Attribution

1.27k stars 545 forks source link

[Inaccuracy] Insufficient max_tokens_to_sample for the Anthropic streaming example in bedrock_basics #236

Closed jicowan closed 2 months ago

jicowan commented 2 months ago

The example says to create a 500 word paragraph but max_tokens is set to 512. If you assume that 1 token~=3/4 words, then 512 tokens would only be 384 words.

w601sxs commented 2 months ago

I just tested this today @jicowan -- changed!