Issues connecting with Mistral through LM Studio

[vanstanian]

Hello,

I've been investigating a bit this pluggin, and I've found that it's virtually possible to use it with any other model appart from OpenAI GPTs if such model runs on a server that conveys with OpenAI's REST intercomunication protocols. As far as I can tell, I tried my model throught Postman with the generated request of this plugin

{

  "model" : "TheBloke/deepseek-coder-6.7B-instruct-GGUF/deepseek-coder-6.7b-instruct.Q4_K_S.gguf",

  "functions" : [ {

    "name" : "webSearch",

    "description" : "Performs a search using a Duck Duck Go search engine and returns the search result json.",

    "parameters" : {

      "type" : "object",

      "properties" : {

        "query" : {

          "type" : "string",

          "description" : "A search query"

        }

      },

      "required" : [ "query" ]

    }

  }, {

    "name" : "getJavaDoc",

    "description" : "Get the JavaDoc for the given compilation unit.  For example,a class B defined as a member type of a class A in package x.y should have athe fully qualified name \"x.y.A.B\".Note that in order to be found, a type name (or its top level enclosingtype name) must match its corresponding compilation unit name.",

    "parameters" : {

      "type" : "object",

      "properties" : {

        "fullyQualifiedName" : {

          "type" : "string",

          "description" : "A fully qualified name of the compilation unit"

        }

      },

      "required" : [ "fullyQualifiedName" ]

    }

  }, {

    "name" : "getCurrentWeather",

    "description" : "Get the current weather in a given location",

    "parameters" : {

      "type" : "object",

      "properties" : {

        "location" : {

          "type" : "string",

          "description" : "The city and state or country, e.g. San Francisco, CA"

        },

        "unit" : {

          "type" : "string",

          "description" : "The temperature unit, e.g. metric or imperial. Default value: metric"

        }

      },

      "required" : [ "location", "unit" ]

    }

  }, {

    "name" : "readWebPage",

    "description" : "Reads the content of the given web site and returns its content as a markdown text.",

    "parameters" : {

      "type" : "object",

      "properties" : {

        "url" : {

          "type" : "string",

          "description" : "A web site URL"

        }

      },

      "required" : [ "url" ]

    }

  }, {

    "name" : "getSource",

    "description" : "Get the source for the given class.",

    "parameters" : {

      "type" : "object",

      "properties" : {

        "fullyQualifiedClassName" : {

          "type" : "string",

          "description" : "A fully qualified class name of the Java class"

        }

      },

      "required" : [ "fullyQualifiedClassName" ]

    }

  } ],

  "messages" : [ {

    "role" : "system",

    "content" : "You are an AI assistant expert in software engineering, integrated as a plugin in an IDE. Your primary role is to assist the user in writing and analyzing source code. Adhere to these guidelines to ensure effective assistance:\r\n\r\n1. Focus on Coding: Provide code solutions or suggestions directly. Avoid elaborating on your responses unless the user specifically asks for explanations.\r\n2. Markdown Formatting: When presenting code examples or snippets, always use Markdown code blocks. This ensures clarity and readability within the IDE environment.\r\n3. Concise Explanations: If the user requests an explanation, keep your response accurate and concise. Avoid lengthy or unnecessary elaboration.\r\n4. Step-by-Step Approach: Tackle problems in a systematic, step-by-step manner. This method helps in breaking down complex tasks and ensures thoroughness in solving coding issues.\r\n5. Responsive Interaction: Engage with the user's queries responsively, adapting your assistance based on the user's input and feedback. Aim to collaboratively find the most accurate and efficient solution.\r\n\r\nRemember, your primary goal is to aid in code development and problem-solving, prioritizing direct and efficient assistance."

  }, {

    "role" : "user",

    "content" : "Hi!"

  }],

  "temperature" : 1.0,

  "stream" : true
}

(I've got to quit the "assist" message, because with it failed... but for another reason, because Mistral seems to need all the messages fullfilled...)

and everything seems fine, but when I tried this plugin with this config

I've got this

!ENTRY org.eclipse.e4.ui.workbench 4 0 2024-04-02 18:20:08.134
!MESSAGE request timed out
!STACK 0
java.net.http.HttpTimeoutException: request timed out
    at java.net.http/jdk.internal.net.http.HttpClientImpl.send(HttpClientImpl.java:949)
    at java.net.http/jdk.internal.net.http.HttpClientFacade.send(HttpClientFacade.java:133)
    at com.github.gradusnikov.eclipse.assistai.services.OpenAIStreamJavaHttpClient.lambda$4(OpenAIStreamJavaHttpClient.java:197)
    at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1804)
    at java.base/java.util.concurrent.CompletableFuture$AsyncRun.exec(CompletableFuture.java:1796)
    at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387)
    at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312)
    at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843)
    at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808)
    at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188)

I don't know if I'm getting trouble with some configuration I missed to allow Eclipse to do request or something, but I don't know how to circumvent this bug.

I hope someone could bring me some help and I will be gratefull if someone can respond my answer.

Regards.

[gradusnikov]

Hi @vanstanian

The most probable cause the lack of HTTP 2.0 protocol support of your local server. I had the same issue with LM Studio. I have patched the code so that the HttpClient uses the HTTP 1.1 protocol version.

Thank you!

/w.

[vanstanian]

It works nice! This will be a great step in your project. Thank you!