IndexError: list index out of range ()

[johnd0e]

With my custom model the first chunk always contains empty choices list. There is no problems with other utilities, but shell_gpt does not expect such case:

│ D:\OpenAI\shell_gpt\venv\lib\site-packages\sgpt\handlers\handler.py:119 in get_completion        │
│                                                                                                  │
│   116 │   │   │   return                                                                         │
│   117 │   │                                                                                      │
│   118 │   │   for chunk in self.client.chat.completions.create(**kwargs, stream=True):           │
│ ❱ 119 │   │   │   delta = chunk.choices[0].delta                                                 │
│   120 │   │   │   if delta.function_call:                                                        │
│   121 │   │   │   │   if delta.function_call.name:                                               │
│   122 │   │   │   │   │   func_call["name"] = delta.function_call.name                           │
│                                                                                                  │
│ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │
│ │             chunk = ChatCompletionChunk(                                                     │ │
│ │                     │   id='',                                                               │ │
│ │                     │   choices=[],                                                          │ │
│ │                     │   created=0,                                                           │ │
│ │                     │   model=None,                                                          │ │
│ │                     │   object=None,                                                         │ │
│ │                     │   system_fingerprint=None,                                             │ │
│ │                     │   prompt_filter_results=[                                              │ │
│ │                     │   │   {                                                                │ │
│ │                     │   │   │   'content_filter_results': {                                  │ │
│ │                     │   │   │   │   'hate': {'filtered': False, 'severity': 'safe'},         │ │
│ │                     │   │   │   │   'self_harm': {'filtered': False, 'severity': 'safe'},    │ │
│ │                     │   │   │   │   'sexual': {'filtered': False, 'severity': 'safe'},       │ │
│ │                     │   │   │   │   'violence': {'filtered': False, 'severity': 'safe'}      │ │
│ │                     │   │   │   },                                                           │ │
│ │                     │   │   │   'prompt_index': 0                                            │ │
│ │                     │   │   }                                                                │ │
│ │                     │   ]                                                                    │ │
│ │                     )                                                                        │ │
│ │         func_call = {'name': None, 'arguments': ''}                                          │ │
│ │      is_code_role = False                                                                    │ │
│ │ is_dsc_shell_role = False                                                                    │ │
│ │     is_shell_role = False                                                                    │ │
│ │            kwargs = {                                                                        │ │
│ │                     │   'messages': [                                                        │ │
│ │                     │   │   {                                                                │ │
│ │                     │   │   │   'role': 'system',                                            │ │
│ │                     │   │   │   'content': 'You are ShellGPT\nYou are programming and system │ │
│ │                     administration assistant.\nYou ar'+287                                   │ │
│ │                     │   │   },                                                               │ │
│ │                     │   │   {'role': 'user', 'content': 'hi'}                                │ │
│ │                     │   ],                                                                   │ │
│ │                     │   'model': 'gpt-4',                                                    │ │
│ │                     │   'temperature': 0.0,                                                  │ │
│ │                     │   'top_p': 1.0,                                                        │ │
│ │                     │   'functions': None                                                    │ │
│ │                     }                                                                        │ │
│ │              self = <sgpt.handlers.default_handler.DefaultHandler object at                  │ │
│ │                     0x000001B004ABBFA0>                                                      │ │
│ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
IndexError: list index out of range

[TheR1D]

I recommend using LocalAI as the backend to pair ShellGPT with custom or open-source LLMs, more details in README.md. Currently, ShellGPT has been optimized and tested only with the OpenAI backend. Therefore, I'm closing this issue. Thank you for your interest and feedback!

[johnd0e]

It's a pity to hear that. This should be so easy to fix on shell_gpt end. The model I use is not self-hosted, it is OpenAI-compatible web service, so I do not see how LacalAI could solve the issue.

[TheR1D]

It's a pity to hear that. This should be so easy to fix on shell_gpt end. The model I use is not self-hosted, it is OpenAI-compatible web service, so I do not see how LacalAI could solve the issue.

I think the right way to resolve the issue in this case is to fix the response of your custom model/backend so it doesn't return empty choices. Or just change ShellGPT code on your end to ignore empty choices.

[johnd0e]

I have no control on that backend. And that response with empty choices is not an error, it also serves own purpose, bringing extra info in other fields. Honestly I have no idea where exactly those extra fields (like prompt_filter_results) are used, but as I see it - that is just minor extension over standard OpenAI output, and it would be great if such minor differences would not crash shell_gpt.

It would have been beneficial for shell_gpt to be robust in processing different data: in the end - that is all valid json, with correct structure.

[johnd0e]

You can check yourself: api base: https://one.caifree.com/v1 token: sk-oR2hYL4yYPeFKip96c6a8256C05d4d628bE7E526336718Ff