With our summarizer now processing four times more tokens than the previous implementation, it is possible to summarize more reviews, generating more meaningful results.
We should scrape to the limits of our token count (while calculating the limit with https://github.com/dqbd/tiktoken) and feed that to GPT3.5
With our summarizer now processing four times more tokens than the previous implementation, it is possible to summarize more reviews, generating more meaningful results.
We should scrape to the limits of our token count (while calculating the limit with https://github.com/dqbd/tiktoken) and feed that to GPT3.5