[Bug]: V1 - Cloudflare triggers even with PUID

[acheong08]

Is there an existing issue for this?

• [X] I have searched the existing issues and checked the recent builds/commits

What happened?

OpenAI has disabled privileges given to plus users and made cloudflare captchas more common

Version where the problem happens

All

What is your operating system ?

Linux

Console logs

Error code: 1020

[acheong08]

I will migrate to alternative method. Please be patient

[peanut996]

I will migrate to alternative method. Please be patient

what is alternative method？

[stoickk]

403 and Error reference number: 1020

[acheong08]

I will migrate to alternative method. Please be patient

what is alternative method？

Using cf_clearance. Unfortunately, Cloudflare also added more fingerprints today and cf_clearance doesn't work by itself anymore either. V1 will be disabled until further notice

[Germey]

I have found a way to bypass the new cloudfare's policy (for today's change), below are some tips I found, hope it can help you:

1. Cloudfare checks the fingerprint of browser, and the fingerprint is associated with SSL ciphers, User-Agent, IP, and Browser's version (I tested Browser version is checked).
2. https://github.com/lwthiker/curl-impersonate provides a simulation of Browser's fingerprint for curl
3. https://github.com/yifeikong/curl_cffi provides a Python wrapper of curl-impersonate

Below are the way which I bypass Cloudfare:

1. Because curl-impersonate and curl_cffi only provides lower simulation for Browser's fingerprint, like 104, 110. So the easier way is to use a downgraded-version of Browser. For example, I downloaded a Chromium 104 version, which is used by playwright.

2. Use Playwright to bypass the Cloudfare firstly, and get all the cookies, like cf_clearance, __Host-next-auth.csrf-token, etc. When launching Playwirght, change the execute_path for launch function, like:

with sync_playwright() as p:
        browser = p.chromium.launch(
            headless=False, executable_path='/Users/germey/Library/Caches/ms-playwright/chromium-1015/chrome-mac/Chromium.app/Contents/MacOS/chromium')
        page = browser.new_page()

3. After you get the cookies, you can also login with it, and just test one request. For example, conversation list, just copy curl and convert it to requests code from https://curlconverter.com/.

4. If you only run the generated code, it won't work, because Cloudfare checks requests's fingerprint, and it won't pass, you will get 403.

5. Then you need to change requests to https://github.com/yifeikong/curl_cffi, to from curl_cffi import requests, and then add impersonate="chrome104" (Note this need to be aligned with your Chrome version) to get function.

6. Then you will find it got bypassed!

[Germey]

If you want to simulate the latest Browser's fingerprint, for example, the latest version 112 for Chrome, below are some tips I found:

1. Add a new script, like for 112 in https://github.com/lwthiker/curl-impersonate, and add related configs, like browsers.json

2. Follow INSTALL.md to compile it
3. Put the output into the reference destination to https://github.com/yifeikong/curl_cffi, and modifiy some configs, to let you will be able to specify impersonate="chrome112".

Hope this can help.

[Germey]

I think if above tips work for you, you can just combime it into your https://github.com/acheong08/ChatGPT-Proxy-V4 reverse proxy. Maybe need to refactor by Python or you make a Go wrapper based on curl-impersonate. Then it may get worked again!

[acheong08]

I have implemented tls spoofing quite a while ago on https://github.com/acheong08/ChatGPT-Proxy-V4. My current issue is that my cf_clearance is unreliable for an unknown reason. It works intermittently

https://github.com/acheong08/ChatGPT-Proxy-V4/blob/cdcaecd0105d94c0d90e4bde0c8262a8a6e17e79/main.go#L14-L22

[Germey]

Yeah, in Go lang seem it also supports that, looks good!

Not too sure about how long the cf_clearance related cookies will work because I bypassed just now, haha.

I think it is expected as the cf_clearance cookeis does not work with a long duration, because you will find you will see cf verifacation after a while, even you are in the chat.openai.com website and did not use it for a while. Maybe in PLUS accout it will be better. I think the solution is keep using the browser to refresh the cf cookies every several minutes.

I will also try to use the new bypass way to build a proxy, to see how it will work and what else need to do for building it. Keep in touch.

[acheong08]

I just checked with curl impersonate and it failed. There seems to be a problem on my end as someone running the exact same code got it working while another faced the same issue

[acheong08]

The expiration seems to be set at around 30 minutes on a standard account

[Germey]

I just checked with curl impersonate and it failed. There seems to be a problem on my end as someone running the exact same code got it working while another faced the same issue

I see one point, cf checkes the Browser version.

One thing can prove it: I tried to use "chrome104" to get the cf cookies using Playwright, and then I used the cf cookies in https://github.com/yifeikong/curl_cffi and tried using Chrome 110 by setting impersonate="chrome110, it did not work, the status code is 403, and then I tried to use impersonate="chrome104 it will work.

In your tls client, you use Chrome 109 to simulate the fingprint https://github.com/acheong08/ChatGPT-Proxy-V4/blob/dee64665b1084c3054fce09d406ba602f34a352c/main.go#L18, but actually I see you used the Chrome 112 to get the cf cookies, https://github.com/acheong08/ChatGPT-Proxy-V4/blob/dee64665b1084c3054fce09d406ba602f34a352c/main.go#L109, it does not match. Different version Browser will get different fingerprint, and the fingerprint is somehow related to cf cookies, so CF will not pass it and you will get failed. You can try to use the same version of Browser.

Hope can help.

[Germey]

The expiration seems to be set at around 30 minutes on a standard account

Yes, this is aliged with my estimated time.

[acheong08]

Different version Browser will get different fingerprint, so CF will not pass it and you will get failed. You can try to use the same version of Browser.

Ah that makes sense. It has to be the same fingerprint

[acheong08]

I suppose I'll need to find a way to downgrade my browser versions since newer fingerprints haven't been supported yet

Edit: Just needed to upgrade lib version

[Germey]

I suppose I'll need to find a way to downgrade my browser versions since newer fingerprints haven't been supported yet

Yeah, I found two ways for the solution:

• One is for downgrade the Automation Browser's version, to match with the fingerprint lib's version. For Playwright, I just specified a execute_path for launch function, see details in https://github.com/acheong08/ChatGPT/issues/1336#issuecomment-1510203468
• The other way is manually build the fingerprint lib to match the latest Automation Browser's version, see details in https://github.com/acheong08/ChatGPT/issues/1336#issuecomment-1510211607

I choosed the first way because it is relatively easier.

[acheong08]

fingerprints above chrome/firefox 110 seems to be randomized and doesn't work. the first method is the only way I think

[Germey]

I have another idea for bypass cf, it is all through Browser, all requests are delegated to Browser to handle. For example, based on Playwright, we can login and get into the chat page in chat.openai.com. Then, we can use https://playwright.dev/docs/network#replaying-from-har to simulate a XHR request, and use https://playwright.dev/docs/network#modify-requests to intercept the request object and modifiy the access_token, request body. And use https://playwright.dev/docs/api/class-page#page-event-response to capture the response and then forward out.

Just an idea and I did not tried yet, maybe it can be another optional solution and you can have a try also.

[peanut996]

I have another idea for bypass cf, it is all through Browser, all requests are delegated to Browser to handle. For example, based on Playwright, we can login and get into the chat page in chat.openai.com. Then, we can use https://playwright.dev/docs/network#replaying-from-har to simulate a XHR request, and use https://playwright.dev/docs/network#modify-requests to intercept the request object and modifiy the access_token, request body.

Just an idea and I did not tried yet, maybe it can be another optional solution.

Is this way need to start a headless browser?

[acheong08]

Is this way need to start a headless browser?

Server side. I could start a browser there and forward all the requests through it

[acheong08]

It would be quite similar to what I had in the past: https://github.com/acheong08/ChatGPT-API-server https://github.com/acheong08/ChatGPT-API-agent

Requires a lot of changes from the original code

IBDP examinations are coming up so development of anything will be slow. I hope someone in the community can help out a bit until my exams are over

[Germey]

I have another idea for bypass cf, it is all through Browser, all requests are delegated to Browser to handle. For example, based on Playwright, we can login and get into the chat page in chat.openai.com. Then, we can use https://playwright.dev/docs/network#replaying-from-har to simulate a XHR request, and use https://playwright.dev/docs/network#modify-requests to intercept the request object and modifiy the access_token, request body. Just an idea and I did not tried yet, maybe it can be another optional solution.

Is this way need to start a headless browser?

I remember Headless browser will be detected by CF, and you even won't get passed of CF. You can use a headful browser with some stealth modification and use xvfb to simulate a virual screen in Docker. See https://playwright.dev/docs/ci#running-headed

[acheong08]

https://github.com/ultrafunkamsterdam/undetected-chromedriver

[Germey]

https://github.com/ultrafunkamsterdam/undetected-chromedriver

Yes! I also used this to bypass cloudfare, and used xvfb to let it able to run in Linux Docker.

[Germey]

It would be quite similar to what I had in the past: https://github.com/acheong08/ChatGPT-API-server https://github.com/acheong08/ChatGPT-API-agent

Requires a lot of changes from the original code

IBDP examinations are coming up so development of anything will be slow. I hope someone in the community can help out a bit until my exams are over

Aha, I also use my part-time to solve these problems, but I will also have a try using my way recently, will let you know if I succeed.

[Germey]

@peanut996 Do you have some ideas or dev plans for it?

It would be quite similar to what I had in the past: https://github.com/acheong08/ChatGPT-API-server https://github.com/acheong08/ChatGPT-API-agent

Requires a lot of changes from the original code

IBDP examinations are coming up so development of anything will be slow. I hope someone in the community can help out a bit until my exams are over

[peanut996]

@peanut996 Do you have some ideas or dev plans for it?

It would be quite similar to what I had in the past: https://github.com/acheong08/ChatGPT-API-server https://github.com/acheong08/ChatGPT-API-agent Requires a lot of changes from the original code IBDP examinations are coming up so development of anything will be slow. I hope someone in the community can help out a bit until my exams are over

No plan yet, I also can only spend spare time on it. 🤣

[Germey]

You can temporarily switch to https://chat.gateway.do

For v1, set env: CHATGPT_BASE_URL=https://chat.gateway.do/api/

Oh, you used Cloudflare itself to bypass itself, right?

[acheong08]

You can temporarily switch to chat.gateway.do

For v1, set env: CHATGPT_BASE_URL=https://chat.gateway.do/api/

May I set that as default while I try to get my implementation working?

[acheong08]

Thank you.

[Haibersut]

If you just want to use it temporarily before the service is fixed, you can also use these free accounts.

There are no risks involved :)

It doesn't seem to be compatible with the way this project requests.

Below is a screenshot of running your project on my own server.

[acheong08]

@Haibersut I just tested and it works.

[acheong08]

AsyncChatbot has issues though but that is because I haven't done well supporting it myself

[twelvehouse]

revChatGPT.typings.Error: OpenAI: {"detail":"forbidden"} (code: 403) Is this error because Im using AsyncChatBot?

[acheong08]

Have you upgraded to https://github.com/acheong08/ChatGPT/releases/tag/4.2.3?

[twelvehouse]

Yes. But python version is 3.9

[acheong08]

Send me some test code. I'll debug on my end later

[peanut996]

You can temporarily switch to chat.gateway.do For v1, set env: CHATGPT_BASE_URL=https://chat.gateway.do/api/

May I set that as default while I try to get my implementation working?

sure. you can switch at any time.

所以 你是咋绕过cf 的呢

[twelvehouse]

Send me some test code. I'll debug on my end later

Im so sorry... It was just the access token had expired

[Haibersut]

@Haibersut I just tested and it works.

I upgraded to the latest version and changed the endpoint to be my own, but the error still occurred. I also tried the sample code you provided, and the error also occurred.

Maybe it was because I built that project myself?

[peanut996]

@Haibersut I just tested and it works.

I upgraded to the latest version and changed the endpoint to be my own, but the error still occurred. I also tried the sample code you provided, and the error also occurred.

Maybe it was because I built that project myself?

your endpoint is outdated, please set env CHATGPT_BASE_URL=https://chat.gateway.do/api/

[Haibersut]

@Haibersut I just tested and it works.

I upgraded to the latest version and changed the endpoint to be my own, but the error still occurred. I also tried the sample code you provided, and the error also occurred. Maybe it was because I built that project myself?

your endpoint is outdated, please set env CHATGPT_BASE_URL=https://chat.gateway.do/api/

我是直接使用pengzhile这个项目的，理论上不应该存在这个问题

[peanut996]

@Haibersut I just tested and it works.

I upgraded to the latest version and changed the endpoint to be my own, but the error still occurred. I also tried the sample code you provided, and the error also occurred. Maybe it was because I built that project myself?

your endpoint is outdated, please set env CHATGPT_BASE_URL=https://chat.gateway.do/api/

我是直接使用pengzhile这个项目的，理论上不应该存在这个问题

你也说了是理论。。 以前遇到 405就是endpoint 不对的问题。。

[Haibersut]

@Haibersut I just tested and it works.

I upgraded to the latest version and changed the endpoint to be my own, but the error still occurred. I also tried the sample code you provided, and the error also occurred. Maybe it was because I built that project myself?

your endpoint is outdated, please set env CHATGPT_BASE_URL=https://chat.gateway.do/api/

我是直接使用pengzhile这个项目的，理论上不应该存在这个问题

你也说了是理论。。 以前遇到 405就是endpoint 不对的问题。。

pengzhile更新了自己的项目吗，因为ta提供的https://chat.gateway.do/api/也应该是由同一个项目搭建的，我使用这个项目搭建的后端是显示不允许这种请求方式，不是类似403或者502之类的，我自己在浏览器是可以正常打开并访问的

我加上了图片：

[peanut996]

你之前能用的原因是因为作者本身就改了endpoint 吧 https://github.com/pengzhile/pandora/blob/9888dcefc95cba9571a9a0170716f2efe87160e5/src/pandora/launcher.py#L108-L112， 你可以尝试把debug 日志打开 看下 他最终访问的域名是什么

[Germey]

You can temporarily switch to chat.gateway.do For v1, set env: CHATGPT_BASE_URL=https://chat.gateway.do/api/

May I set that as default while I try to get my implementation working?

sure. you can switch at any time.

所以 你是咋绕过cf 的呢

Use Cloudflare Workers to bypass itself.

[Haibersut]

你之前能用的原因是因为作者本身就改了endpoint 吧 https://github.com/pengzhile/pandora/blob/9888dcefc95cba9571a9a0170716f2efe87160e5/src/pandora/launcher.py#L108-L112， 你可以尝试把debug 日志打开 看下 他最终访问的域名是什么

这个debug模式下日记不完整

有没有可能是因为请求方式的问题

[peanut996]

You can temporarily switch to chat.gateway.do For v1, set env: CHATGPT_BASE_URL=https://chat.gateway.do/api/

May I set that as default while I try to get my implementation working?

sure. you can switch at any time.

所以 你是咋绕过cf 的呢

Use Cloudflare Workers to bypass itself.

那这种方式还需要设置复杂的puid 以及伪造指纹吗， 还是说直接转发就行。。

[Germey]

You can temporarily switch to chat.gateway.do For v1, set env: CHATGPT_BASE_URL=https://chat.gateway.do/api/

May I set that as default while I try to get my implementation working?

sure. you can switch at any time.

所以 你是咋绕过cf 的呢

Use Cloudflare Workers to bypass itself.

那这种方式还需要设置复杂的puid 以及伪造指纹吗， 还是说直接转发就行。。

看起来不需要，直接转发就行。不完全确定，我还没试，得再研究研究。

[peanut996]

You can temporarily switch to chat.gateway.do For v1, set env: CHATGPT_BASE_URL=https://chat.gateway.do/api/

May I set that as default while I try to get my implementation working?

sure. you can switch at any time.

所以 你是咋绕过cf 的呢

Use Cloudflare Workers to bypass itself.

那这种方式还需要设置复杂的puid 以及伪造指纹吗， 还是说直接转发就行。。

看起来不需要，直接转发就行。不完全确定，我还没试，得再研究研究。

看起来不太行