Upon using the ChatGPTTagger on more nodes, I ran into 2 kinds of runtime issues (so far):
The completion response would occasionally max out the context window and get truncated. This issue is addressed by the first commit (Handle truncated GPT responses) by adding to the config a parameter to leave wiggle room for the completion tokens.
⚠️ The completion response would occasionally be missing some input nodes (which is slightly concerning). The current implementation would check for this and fail on "ID mismatch". The second commit (Warn instead of failing on node id mismatch) changes that behavior to warn instead of failing and suggests to the user to rerun the node tagging.
Notes about this second commit:
Upon re-running the tagging, only the mismatched nodes will be fetched (since the successful ones are all cached)
In the case where several completions are requested for each prompt (new default as per #15), a node has to be present in all the completions "choices" to be considered valid (i.e. if an input node is missing from any completion, it will be considered invalid and not cached)
@dhimmel @eric-czech Does this sound like a reasonable behavior?
Hi,
Upon using the ChatGPTTagger on more nodes, I ran into 2 kinds of runtime issues (so far):
Notes about this second commit:
@dhimmel @eric-czech Does this sound like a reasonable behavior?