This dataset has the following issues: the multi cluster extraction node was outputting too many topics and subtopics, this made the merge cluster extraction node fall over.
The merge cluster extraction node would lead to openai timing out, and with an increased timeout the output was truncated.
Think about adding clearer warnings when outputs get too big. Also experiment with continuation calls to OpenAI.
lightningorb
commented
5 months ago
https://talktothecity.org/report/wishing-pond-fork
This dataset has the following issues: the multi cluster extraction node was outputting too many topics and subtopics, this made the
merge cluster extractionnode fall over.The
merge cluster extractionnode would lead to openai timing out, and with an increased timeout the output was truncated.Think about adding clearer warnings when outputs get too big. Also experiment with continuation calls to OpenAI.