Confused with setting up gpt-4

[Kabouik]

I needed gpt-4 for more context tokens, and realized that I needed add some credits to my OpenAI balance to get access to it from the API. I added some, and then immediatelyu after renewing my API key, I did notice indeed that gptel would suddenly succeed querying ChatGPT despite me setting the model to gpt-4, while it would fail before.

However, the AI replies that it is based on GPT-3 model, and its limit is 4096 tokens. This contradicts what is written here, stating that gpt-4 has a limit of 8192 tokens. I'm confused. I know this may not be a gptel issue, but if entering gpt-4 when in the gptel configuration prompt buffer is not the right way to do it, what am I missing?

I'm sorry if I did somethign wrong, I have no experience with gptel yet.

[karthink]

How are you setting the model to gpt-4?

[JT-McLeod]

I am in the similar position. I have set up an platform.openai.com/account and created an API keys Secret key. This gets to the GPT-3, but not other models such as GPT-3.5-turbo or others listed under "Rate limits" tab. (NB I note that GPT-4 is missing from this list. I did expect to see it there.) Thanks

[JT-McLeod]

After setting up a pre-payment of $20, I can now see the gpt-4 option in the "Rate Limits" tab available through the ..openai.com/account. Still not able to chat with the gpt-4 model through gptel.

[JT-McLeod]

I now have "gptel-model" set to "gpt-4" and this showing in the ChatGPT buffer, but in answer to the question "Which GPT model are you?", ChatGPT says it is "GPT-3". Weird! btw Billing is now reporting that I am using GPT-4.

[karthink]

I now have "gptel-model" set to "gpt-4"

Just checking: how did you set gptel-model to gpt-4?

[JT-McLeod]

C-u C-c Enter, m and select it from the options.

[Kabouik]

How are you setting the model to gpt-4?

I set it up with M-x gptel-menu m "gpt-4". But if the model was not right, the query would fail, and now it works after I added some credit to my balance; so it must be right, but somehow the model doesn't seem to be GPT-4. Is that possible that gptel checks if the API has access to the model before querying it, but mistakenly queries GPT-3.5 when the check passes?

I later set the model to "gpt-3.5-turbo-16k" and here, the token limit was adequately adjusted to 16k.

[JT-McLeod]

I tried to use the gpt-4-32k and got the response "ChatGPT error:" the model doesn't exist. Inference it is the API is getting the request I expect.

I then changed gptel m to gpt-4 put another $20 on credit (too ensure it wasn't the limiting factor) and asked ChatGPT "Are you gpt 4?" It's response was: "No, I am based on OpenAI's GPT-3 model. As of the time of this interaction, GPT-4 has not been released yet. Any updates to the model version will be posted by OpenAI in their official announcements." Meanwhile I appear to be further billed for gpt-4. So changing back to gpt-3 for now (to protect my credit), with the intent of in the weekend blundering around in the code and looking at what is actually being sent to the OpenAI API.

[karthink]

@Kabouik and @JT-McLeod: If I understand correctly, you've paid for gpt-4 access on openai's website and can access gpt-4 directly (through a browser), but not when using gptel.

According to OpenAI, you should have access to the GPT-4 API if you're piad more than $1 so far. I haven't yet so I can't access GPT-4 to test this bug. (I can't add credits to my account either since I don't see the "prepaid billing" option; my account is older than Aug 23.)

Could you try the following curl command and let me know what the result is?

curl  --location  --silent  --compressed  --disable  -XPOST  \
      -w'(abcdefgh . %{size_header})'  -m60  -D-  \
      -d'{"model":"gpt-4","messages":[{"role":"system","content":"You are a large language model living in Emacs and a helpful assistant. Respond concisely."},{"role":"user","content":"What model of gpt are you?"}],"stream":false,"temperature":1.0}' \
      -H'Content-Type: application/json' \
      -H'Authorization: Bearer $YOUR_API_KEY'  \
      https://api.openai.com/v1/chat/completions 

(Replace $YOUR_API_KEY with your key)

[jmberen]

I have the same issue and I've tried the curl command you suggested. This is the result:

HTTP/2 200 
date: Wed, 04 Oct 2023 23:31:53 GMT
content-type: application/json
access-control-allow-origin: *
cache-control: no-cache, must-revalidate
openai-model: gpt-4-0613
openai-organization: user-m4qjlmskyqztfbzavywlw85e
openai-processing-ms: 1328
openai-version: 2020-10-01
strict-transport-security: max-age=15724800; includeSubDomains
x-ratelimit-limit-requests: 200
x-ratelimit-limit-tokens: 10000
x-ratelimit-remaining-requests: 199
x-ratelimit-remaining-tokens: 9953
x-ratelimit-reset-requests: 300ms
x-ratelimit-reset-tokens: 282ms
x-request-id: 9d0a9127222c4f9fd7564b557d29e57a
cf-cache-status: DYNAMIC
server: cloudflare
cf-ray: 8111388aaa6d1946-BCN
content-encoding: gzip
alt-svc: h3=":443"; ma=86400

{
  "id": "chatcmpl-865dIVyGFGHXM717RTAuv3rkwN6zp",
  "object": "chat.completion",
  "created": 1696462312,
  "model": "gpt-4-0613",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "I'm based on GPT-3, OpenAI's latest and most powerful version of the Generative Pretrained Transformer series as of the time of this interaction."
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 38,
    "completion_tokens": 33,
    "total_tokens": 71
  }
}
(abcdefgh . 741)

[karthink]

@jmberen thanks.

It looks like it's actually gpt-4:

"model": "gpt-4-0613",

But the answer it provides is that it's GPT-3:

"content": "I'm based on GPT-3, OpenAI's latest and most powerful version of the Generative Pretrained Transformer series as of the time of this interaction."

I'm not sure what to make of it. :confused:

[jmberen]

Hi @karthink And surprisingly, it charges in on the bill as GPT-4 request...

[Kabouik]

@JT-McLeod and @jmberen already replied (thanks), so just commenting on this:

(I can't add credits to my account either since I don't see the "prepaid billing" option; my account is older than Aug 23.)

I had the same issue and had to disable my existing payment option to re-create it, and then the option appeared.

[PBorocz]

Based on @Kabouik's comment, I also went through the process of deleting my existing billing plan and setting a new one up (as well as creating a new key)...now my curl test does operate (it wouldn't before) but gives me the same odd output @jmberen got.

Interestingly, when I try to "ask" Chat what model I'm now using (through several various attempts), it answers that it doesn't know about GPT-4 (even though it's set through gptel-menu)...leading me to believe its still processing using a GPT-3 version.

Haven't looked through the code but would love to look through (or set?) detailed logging to see what's going back and forth to the openai.

Net net, great tool, thanks for bringing this to fruition @karthink !!!

[karthink]

leading me to believe its still processing using a GPT-3 version.

Based on the curl output, this bug is not related to gptel -- the curl command you tried is the one gptel uses under the hood, and ChatGPT's repsonse identifies the model as gpt-4-0613.

I'm leaving this issue open so we (or someone else looking into it) can figure out the inconsistency.

[jmberen]

Absolutely agree with @karthink, the bug seems an openAI API issue, not a gptel issue. By the way, my recommendation by the moment is to set the gptel-3.5-turbo model. Otherwise, you'll get GPT-3 responses but it will get billed as GPT-4 responses. At least we can save some money....

[pondersson]

It seems like GPT-4 might not be aware of its own existence: https://community.openai.com/t/gpt-4-through-api-says-its-gpt-3/286881

[JT-McLeod]

To ChatGPT-4: [image: User] Hi, which model are you?

I am based on the GPT-4 architecture by OpenAI. How can I assist you further?

On Sat, Oct 7, 2023 at 9:51 PM pondersson @.***> wrote:

It seems like GPT-4 might not be aware of its own existence: https://community.openai.com/t/gpt-4-through-api-says-its-gpt-3/286881/7

— Reply to this email directly, view it on GitHub https://github.com/karthink/gptel/issues/106#issuecomment-1751655014, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAM4NSCRBWNPG5S26IBHCPTX6EJX3AVCNFSM6AAAAAA5JVYHUWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONJRGY2TKMBRGQ . You are receiving this because you were mentioned.Message ID: @.***>

[pondersson]

@JT-McLeod Is that the website version of GPT-4? According to this thread it may have some extra background prompts (and maybe training?) that help it with this particular question: https://community.openai.com/t/gpt-4-via-the-api-not-nearly-as-powerful-as-gpt-4-on-the-website/330954

I'm not sure that asking the model for its version is the most reliable test. When I pass the same questions to GPT-3 and GPT-4 through gptel and compare the responses, the ones from GPT-4 tend to be of higher quality. But I could of course be mistaken and just a victim of confirmation bias :shrug:

[JT-McLeod]

I do as @jmberen describes, run gptel with gpt-3.5-turbo. My experience is with some code, gptel doesn't have current information. For example code suggested by gptel for the Bokeh rangetool errors as it is using an earlier version. Take the code suggestion and pass it through Chat-gpt with GPT-4 and the modified code runs. Conclusion, gptel is not accessing the newer information.

I am still impressed with and start with gptel as it better fits my workflow and thinking style :).

I have in mind to see what knowledge base JuypterAI taps and how that compares to gptel. I want to explore if they use the same API as gptel. If not then can gptel be modified to do so.

[karthink]

run gptel with gpt-3.5-turbo.

Conclusion, gptel is not accessing the newer information.

What happens if you run gptel with gpt-4 instead?

[JT-McLeod]

My initial expectation was I would get an error as I mentioned near the top of this thread. Having just tried it again (ie C-u C-c Enter, m gpt-4) and repeated the coding request I was working on, I now find I get a sensible and more refined response ! Ie it now doesn't error out. Gpt-4 seems to be working in gptel.

More exploration is now possible with gptel using "gpt-4".

Exciting times - thank you!

[leungbk]

My initial expectation was I would get an error as I mentioned near the top of this thread. Having just tried it again (ie C-u C-c Enter, m gpt-4) and repeated the coding request I was working on, I now find I get a sensible and more refined response ! Ie it now doesn't error out. Gpt-4 seems to be working in gptel.

This used to work for me in prior checkouts, but does not on the most recent checkout of gptel (d7a89d7575e5d0f5148d7498536e470fda628f1e)

[karthink]

What error do you get?

On Fri, Nov 3, 2023, 2:09 AM Brian Leung @.***> wrote:

My initial expectation was I would get an error as I mentioned near the top of this thread. Having just tried it again (ie C-u C-c Enter, m gpt-4) and repeated the coding request I was working on, I now find I get a sensible and more refined response ! Ie it now doesn't error out. Gpt-4 seems to be working in gptel.

This used to work for me in prior checkouts, but does not on the most recent checkout of gptel (d7a89d7 https://github.com/karthink/gptel/commit/d7a89d7575e5d0f5148d7498536e470fda628f1e )

— Reply to this email directly, view it on GitHub https://github.com/karthink/gptel/issues/106#issuecomment-1792087056, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACBVOLDUQY6J6RFIH355GBDYCSYD7AVCNFSM6AAAAAA5JVYHUWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTOOJSGA4DOMBVGY . You are receiving this because you were mentioned.Message ID: @.***>

[karthink]

@leungbk If you are still using gptel + gpt4, could you confirm if it does not work for you?

[leungbk]

karthink @.***> writes:

@leungbk If you are still using gptel + gpt4, could you confirm if it does not work for you?

Sorry for the delay. I haven't been using GPT-4 lately, but I remember concluding that my previous report was a false alarm.

However, I think the issue mentioned in this thread involving an observed difference between C-u C-c RET and C-c RET is a real one. I can't provide any further testing for this since I'm not on GPT-4 right now.

[karthink]

However, I think the issue mentioned in this thread involving an observed difference between C-u C-c RET and C-c RET is a real one.

Now that I can access gpt-4, I can confirm that setting the model using C-u gptel-send works as expected. The chat still self-identifies as gpt-3, but the model being used is gpt-4 as the billing and the http headers attest. Assuming there's nothing else to be done from gptel's side, I'm closing this issue.

[schmudde]

Thanks for all the work on this. One quick followup confirming GPT4 (but also unexpected behavior):

GPTel header confirms I'm using the right interface: [Send: buffer] [gpt-4-turbo-preview]. But I can't read a URL from GPTel:

can you read this url? https://plaid.com/

I can't directly access or read content from URLs, including the one you've provided for Plaid's documentation on webhook verification. However, I can still offer information and guidance based on my training data up until my last update in April 2023. ...

However, we all know that the web interface can. Just to make sure, here's the answer of the last training data from my GPT 4 web interface:

what was your last training data update?

My last training data update was in April 2023.