Error Streaming Response - API Error. (Bug with long files without paragraph spacing)

[avataraustin]

So If I ask a question of my files using language to trigger search of my embedded Smart Connections files I get an API Error. Smart Connections Error Streaming Response.

plugin:smart-connections:2696 CustomEvent {isTrusted: false, data: `{\n  "error": {\n    "message": "This model's maximu…es",\n    "code": "context_length_exceeded"\n  }\n}\n`, source: ScStreamer, detail: null, type: 'error', …}

plugin:smart-connections:2734 Uncaught (in promise) CustomEvent {isTrusted: false, data: `{\n  "error": {\n    "message": "This model's maximu…es",\n    "code": "context_length_exceeded"\n  }\n}\n`, source: ScStreamer, detail: null, type: 'error', …}

After testing using a question meant to trigger embedding query of other documents I knew used to work, it worked fine. The issue seems to be only occurring when Smart Connections queries a folder I have containing a bunch of really long markdown files that don't have spaces in the text. They are long transcript files without paragraph spacing. This seems to confuse Smart Connections and cause it to fail.

[avataraustin]

I ended up removing the new .md files of the transcripts that didn't have any punctuation and I processed them all adding sentence structure to the files. I then readded the files and forced a refresh of my embeddings in SC. Unfortunately I still get an error about it trying to use too much context length tokens. I'm using 3.5turbo16k but it was trying to use 20k tokens. If I initiate a smart chart that I know will target other notes it works fine but these new longer files apparently use up too much context length? Does each of my markdown files need to be less than 16k tokens or is there something I could do to my files that would help it function without needing to drastically edit my files? Or is it something else entirely?

[avataraustin]

FYI , It seems the issue goes away if the markdown file contains headings to split up the text into more manageable chunks. I wrote a script that at least takes the .txt transcript files I was starting with and converts them into markdown files with periods and clusters them into manageable headings in the new markdown file. It isn't perfect but it seems to work for letting Smart Connections use the data now.

If anyone has a similar need, script -> Transcript_processing

[brianpetro]

@avataraustin thanks for sharing that and bringing this my attention.

It's somewhat of a known issue, but it helps to have the issue clarified and recorded like this.

I anticipate this being fixed in the next major release. But until then, your script could be a big help for other users facing the same issue.

Thanks for your contribution

🌴