Autocomplete error!

[yusylams]

I keep getting autocomplete error: Error: AbortError when using the plugin. It just doesn't work for me.

[buszk]

@yusylams Thank you for your interest in the project. Before I try to triage the bug. Could you provide more details such as version, whether you configured a valid openai api token, detailed steps to reproduce, and maybe logs from developer tools (View -> Toggle Developer Tool).

[buszk]

Hi @yusylams, I haven't heard back from you since last contact. Have you solved the issue yourself?

[yusylams]

Hi, I wasn't able to solve the issue. I just couldn't figure it out at the time so I gave up on it ig.

[yusylams]

I get this error when translating long mandarin texts into english using the plugin. It works fine whenever I select a really short part of it for translation but gives me the error for anything longer than. What steps do I need to take after toggling developer tools in order to provide you with the information you need to look into my issue?

[buszk]

@yusylams I see. So the plugin has trouble handling long input text. I will better surface the error, and add ability to handle long inputs.

[buszk]

Here is my reproduced result on v0.5.0.

• When long input text is passed to LLM, a notice shows up stating the token number is more than maximum allowed.

There is some intricacy on splitting the input text, as it cannot be generalize to all action type. Suppose I split on paragraphs for the examples. I can then ask llm to translate each paragraph and put the result together. However, it won't work for summarization because the context must be the whole documents.

I suggest you upgrade to v0.5.0 and use gpt-3.5.-16k model that allows 16,000 tokens for now.

[treyharris]

/I’m getting the same issue on v0.5.2. This is with an Obsidian file containing 913 words. It does not occur with your included “Summarize Document” action using OpenAl GPT-3.5-turbo-preview on the document.

But with my action (which is basically just a text processor—I could do it with regexes, but I was being lazy) using OpenAl GPT-3.5-turbo-preview on the very same 913-word document, I get 525–540 words through (it’s different every time I run it) before it aborts with:

If I switch my action to using OpenAl GPT-3.5-16k, I get the error above after only 226-300 words are processed (again, it differs from run to run).

I tried using OpenAI GPT-4-32k, and I got:

I’m only at Tier 1, so I suspect I don’t have access to that model yet.

I switched to OpenAl GPT-4-turbo-preview, and it only gives me 150–230 words. (Btw, in my OpenAI dashboard, it calls the model GPT-4-1106-preview.)

Switching to selection mode and selecting all does not seem to change anything; however, I can find where it stopped, select from there to the end, and repeat, building up the output after 3 or so runs.

In case it’s of interest: the usage dashboard says each run is using the identical number of context tokens, 965—which is approximately the 913 words in the document, plus the words in prompt, as would be expected. The usage dashboard reports generated tokens varying from run to run, as I observed.

For reference, my action is as follows:

Action Name: Convert Markdown to Comments format

LLM Model Selection: OpenAl GPT-3.5-16k | GPT-3.5-turbo-preview | GPT-4-turbo-preview

Prompt:

Please take the following text in Markdown and give me a version where text in italics is changed to underscore-quoted, and boldface is changed to asterisk-surrounded. For example: `This is *italics* and this is **boldface**` should become `This is _italics_ and this is *boldface*`.

or, wrapped for clarity:

Please take the following text in Markdown and give me a version where text in italics is changed to underscore-quoted, and boldface is changed to asterisk-surrounded. For example: This is *italics* and this is **boldface** should become This is _italics_ and this is *boldface*.

Output format:

```txt
{{result}}

**Modal title:** `Check result`

**Input selection:** Select the whole document | Input selected text by cursor

**Output location:** Insert at the beginning of the document

[buszk]

@treyharris There is a 20 sec timeout. Does this match your experience?

[treyharris]

@treyharris There is a 20 sec timeout. Does this match your experience?

Ah indeed! That would explain the otherwise slightly counterintuitive results—they’re not mapping to the sophistication/quality of the model but to the raw speed of output—and the randomness since wall-clock time can be influenced by a variety of factors.

Perhaps this could be made a tunable parameter on a per-action basis? From your examples I take it you find GPT integration most useful for reduction tasks (like summarizing), but I use it frequently for mapping tasks (like format-modification) tasks and expansion tasks (like filling in an outline), and it seems more likely to run into this if you have a rough cutoff on the quantity of generated tokens.

[buszk]

The timeout is added for condition with bad network connection so that a user don't wait forever. It's surely flawed unable to handle long input/output.

There are some options for fixing this bug:

• Customizable as an action attribute
• Dynamically proportional to input length.
• For streaming mode, timeout if the first token didn't come within 20 sec, instead of timeout on last token.

I think the last is very feasible and should be how I imagine it to work, although it's not really extensible for non-streaming mode. I'll try to implement it shortly.

[buszk]

@treyharris @yusylams Could you check if version 0.5.3 solves the issue?

[treyharris]

@treyharris @yusylams Could you check if version 0.5.3 solves the issue?

Works perfectly, thanks!