Empty result when using gpt-4-****-**** models

[k-martynov]

I've observed that I receive an empty result when utilizing either gpt-4-1106-preview or gpt-4-0125-preview. This issue occurs without any accompanying code, but rather in the context of an explanation.

I've configured custom actions similarly to the default settings, with the modification being the model selection:

{
  "complete_code": {
    "type": "chat",
    "opts": {
      "template": "Complete the following code written in {{lang}} by pasting the existing code and continuing it.\n\nExisting code:\n```{{filetype}}\n{{input}}\n```\n\n```{{filetype}}\n",
      "strategy": "replace",
      "params": {
        "model": "gpt-4-0125-preview",
        "stop": ["```"]
      }
    }
  },
  "grammar_correction": {
    "type": "chat",
    "opts": {
      "template": "Correct this to standard {{lang}}:\n\n{{input}}",
      "strategy": "replace",
      "params": {
        "model": "gpt-4-0125-preview"
      }
    },
    "args": {
      "lang": {
        "type": "string",
        "optional": "true",
        "default": "english"
      }
    }
  },
  "translate": {
    "type": "chat",
    "opts": {
      "template": "Translate this into {{lang}}:\n\n{{input}}",
      "strategy": "replace",
      "params": {
        "model": "gpt-4-0125-preview",
        "temperature": 0.3
      }
    },
    "args": {
      "lang": {
        "type": "string",
        "optional": "true",
        "default": "english"
      }
    }
  },
  "keywords": {
    "type": "chat",
    "opts": {
      "template": "Extract the main keywords from the following text.\n\n{{input}}",
      "strategy": "replace",
      "params": {
        "model": "gpt-4-0125-preview",
        "temperature": 0.5,
        "frequency_penalty": 0.8
      }
    }
  },
  "docstring": {
    "type": "chat",
    "opts": {
      "template": "# An elaborate, high quality docstring for the above function:\n# Writing a good docstring\n\nThis is an example of writing a really good docstring that follows a best practice for the given language. Attention is paid to detailing things like\n* parameter and return types (if applicable)\n* any errors that might be raised or returned, depending on the language\n\nI received the following code:\n\n```{{filetype}}\n{{input}}\n```\n\nThe code with a really good docstring added is below:\n\n```{{filetype}}",
      "strategy": "edit",
      "params": {
        "model": "gpt-4-0125-preview",
        "stop": ["```"]
      }
    }
  },
  "add_tests": {
    "type": "chat",
    "opts": {
      "template": "Implement tests for the following code.\n\nCode:\n```{{filetype}}\n{{input}}\n```\n\nTests:\n```{{filetype}}",
      "strategy": "append",
      "params": {
        "model": "gpt-4-0125-preview",
        "stop": ["```"]
      }
    }
  },
  "optimize_code": {
    "type": "chat",
    "opts": {
      "template": "Optimize the following code.\n\nCode:\n```{{filetype}}\n{{input}}\n```\n\nOptimized version:\n```{{filetype}}",
      "strategy": "edit",
      "params": {
        "model": "gpt-4-1106-preview",
        "stop": ["```"]
      }
    }
  },
  "refactor_code": {
    "type": "chat",
    "opts": {
      "template": "Refactor the following code as Senior Developer.\n\nCode:\n```{{filetype}}\n{{input}}\n```\n\nRefactored version:\n```{{filetype}}",
      "strategy": "edit",
      "params": {
        "model": "gpt-4-0125-preview",
        "stop": ["```"]
      }
    }
  },
  "summarize": {
    "type": "chat",
    "opts": {
      "template": "Summarize the following text.\n\nText:\n\"\"\"\n{{input}}\n\"\"\"\n\nSummary:",
      "strategy": "edit",
      "params": {
        "model": "gpt-4-0125-preview"
      }
    }
  },
  "fix_bugs": {
    "type": "chat",
    "opts": {
      "template": "Fix bugs in the below code\n\nCode:\n```{{filetype}}\n{{input}}\n```\n\nFixed code:\n```{{filetype}}",
      "strategy": "edit",
      "params": {
        "model": "gpt-4-0125-preview",
        "stop": ["```"]
      }
    }
  },
  "explain_code": {
    "type": "chat",
    "opts": {
      "title": " Explain Code",
      "template": "Explain the following code:\n\nCode:\n```{{filetype}}\n{{input}}\n```\n\nUse markdown format.\nHere's what the above code is doing:\n```",
      "strategy": "display",
      "params": {
        "model": "gpt-4-0125-preview",
        "stop": ["```"]
      }
    }
  },
  "roxygen_edit": {
    "type": "chat",
    "opts": {
      "template": "Insert a roxygen skeleton to document this R function:\n\n```{{filetype}}\n[insert]\n\n{{input}}\n\n```",
      "strategy": "prepend",
      "params": {
        "model": "gpt-4-0125-preview",
        "temperature": 0.5,
        "max_tokens": 1024
      }
    }
  },
  "code_readability_analysis": {
    "type": "chat",
    "opts": {
      "strategy": "quick_fix",
      "template": "{{input}}",
      "params": {
        "model": "gpt-4-0125-preview",
        "max_tokens": 2048,
        "messages": [
          {
            "role": "system",
            "content": "\nYou must identify any readability issues in the code snippet.\nSome readability issues to consider:\n- Unclear naming\n- Unclear purpose\n- Redundant or obvious comments\n- Lack of comments\n- Long or complex one liners\n- Too much nesting\n- Long variable names\n- Inconsistent naming and code style.\n- Code repetition\nYou may identify additional problems. The user submits a small section of code from a larger file.\nOnly list lines with readability issues, in the format <line_num>|<issue and proposed solution>\nYour commentary must fit on a single line\nDo not use the range of lines but just single line number\nIf there's no issues with code respond with only: <OK>\n"
          },
          {
            "role": "user",
            "content": "\n0 public class Logic {\n1      public static void main(String[] args) {\n2         Scanner sc = new Scanner(System.in);\n3         int n = sc.nextInt();\n4         int[] arr = new int[n];\n5         for (int i = 0; i < n; i++) {\n6             arr[i] = sc.nextInt();\n7         }\n8         int[] dp = new int[n];\n9         dp[0] = arr[0];\n10         dp[1] = Math.max(arr[0], arr[1]);\n11         for (int i = 2; i < n; i++) {\n12             dp[i] = Math.max(dp[i - 1], dp[i - 2] + arr[i]);\n13         }\n14         System.out.println(dp[n - 1]);\n15     }\n16 }"
          },
          {
            "role": "assistant",
            "content": "\n0: The class name 'Logic' is too generic. A more meaningful name could be 'DynamicProgramming'\n2: The variable name 'sc' is unclear. A more meaningful name could be 'scanner'.\n3: The variable name 'n' is unclear. A more meaningful name could be 'arraySize' or 'numElements'.\n4: The variable name 'arr' unclear. A more descriptive name could be 'inputArray' or 'elementValues'.\n8: The variable name 'dp' is unclear. A more informative name could be 'maxSum' or 'optimalSolution'.\n9: There are no comments explaining the meaning of the 'dp' array values and how they relate to the problem statement.\n11: There are no comments explaining the logic and purpose of the for loop"
          },
          {
            "role": "user",
            "content": "\n0    for (let i: number = 0; i < l; i++) {\n1       let notAddr: boolean = false;\n2       // non standard input\n3       if (items[i].scriptSig && !items[i].addr) {\n4         items[i].addr = 'Unparsed address [' + u++ + ']';\n5         items[i].notAddr = true;\n6         notAddr = true;\n7       }\n8 \n9       // non standard output\n10       if (items[i].scriptPubKey && !items[i].scriptPubKey.addresses) {\n11         items[i].scriptPubKey.addresses = ['Unparsed address [' + u++ + ']'];"
          },
          {
            "role": "assistant",
            "content": "\n0: The variable name 'i' and 'l' are unclear and easily confused with other characters like '1'. More meaningful names could be 'index' and 'length' respectively.\n1: The variable name 'notAddr' is unclear and a double negative. An alternative could be 'hasUnparsedAddress'.\n3: The comment \"non standard input\" is not very informative. It could be more descriptive, e.g., \"Check for non standard input address\"\n9: The comment \"non standard output\" is not very informative. It could be more descriptive, e.g., \"Check for non standard output address\"\n10: The variable name 'items' might be more informative if changed to 'transactions' or 'txItems'.\n11: The array element 'Unparsed address [' + u++ + ']' could use a more descriptive comment, e.g., \"Assign a unique identifier to non standard output addresses\"\n11: The variable name 'u' is unclear. A more meaningful name could be 'unparsedAddressCount' or 'unparsedAddressId'."
          },
          {
            "role": "user",
            "content": "\n0 function BaseAction:init(opts)\n1   self.opts = opts\n2 end"
          },
          {
            "role": "assistant",
            "content": "<OK>"
          }
        ]
      }
    }
  }
}

Here is my config:

  {
    "jackMort/ChatGPT.nvim",
    event = "VeryLazy",
    config = function()
      local chatgpt = require("chatgpt")
      local wk = require("which-key")

      chatgpt.setup({
        actions_paths = { "~/.config/nvim/helpers/actions.json" },
        open_ai_params = {
          model = "gpt-4-0125-preview",
        },
        openai_edit_params = {
          model = "gpt-4-0125-preview",
        },
      })
}

Here how it looks for example optimize_code action:

Same optimize_code action with gpt-4 model:

[k-martynov]

Hi @jackMort,

I hope you're doing well. I find myself in need of some assistance and was wondering if you could possibly lend me a hand with an issue I'm facing. Your expertise would be greatly appreciated

[BetaGo]

This problem should be solved after this PR is merged. #396