Closed zjrwtx closed 1 year ago
same
same, found cf_clearance token still not work
same
did u guys set the correct user_agent from your browser ?
Same error for me.
did u guys set the correct user_agent from your browser ?
Yeah, actually i set my agent from whatismybrowser.com
@zjrwtx we need to set user_agent only in the file streamlit_app.py ?
and is there a need to set any value in cf_clearance in the same file?
did u guys set the correct user_agent from your browser ?
Can you please tell me where exactly i can set the user_agent from my browser?
and where if im using phind should i insert this code? `# if using CompletionModel.load set these ora.user_id = '...' ora.session_token = '...'
model = ora.CompletionModel.load(chatbot_id, 'gpt-4')
and which cf_clearance token to use?
I too get the same error:
Following are my settings in streamlit_app.py
:
phind.user_agent = '<set using https://whatmyuseragent.com>'
phind.cf_clearance = ''
same issue, running on localhost
Main things are: make sure your using the CORRECT useragent, 2 make sure your getting the clearance token FROM THE API SITE aka if its from phind go to the phind API and get the clearance and lastly IF ITS ON A REMOTE SERVER like a vps the clearance WILL NOT WORK because the IP has to match, make sure to use a proxy server hosted on that VPS then get the clerance with that proxy
Bare with me please, i got my useragent and put it in phind.user_agent '" and im on phind.com, how am i supposed to get my clearence token to fill in the phind.cf_clearence = i suppose ?
Main things are: make sure your using the CORRECT useragent, 2 make sure your getting the clearance token FROM THE API SITE aka if its from phind go to the phind API and get the clearance and lastly IF ITS ON A REMOTE SERVER like a vps the clearance WILL NOT WORK because the IP has to match, make sure to use a proxy server hosted on that VPS then get the clerance with that proxy
so, i got my useragent from whatismybrowser, and got cf_clearance from phind.com (cookies). Still same error. Am I wrong somewhere ?
Bare with me please, i got my useragent and put it in phind.user_agent '" and im on phind.com, how am i supposed to get my clearence token to fill in the phind.cf_clearence = i suppose ?
You get it using the chrome dev tools and going to cookies
Thank you for showing me this, I got my cf_clearance and i got the user id from whatmyuseragent.com and filled them in the right places in the streamlit_app.py running it again i get this new error : An error occured, please make sure you are using a cf_clearance token and correct useragent | 'rawBingResults'
@maxolie00 , I got it working jut after couple of refreshes Thanks @localuser-isback for your help :)
@maxolie00 , I got it working jut after couple of refreshes Thanks @localuser-isback for your help :)
Same, now working after refreshes too, thanks all
Hi, phind in their panick are sometimes disabling and re-enabling cloudflare, which may require you to not set it and to set it, please refer to the comment of @localuser-isback
"make sure your using the CORRECT useragent, 2 make sure your getting the clearance token FROM THE API SITE aka if its from phind go to the phind API and get the clearance and lastly IF ITS ON A REMOTE SERVER like a vps the clearance WILL NOT WORK because the IP has to match, make sure to use a proxy server hosted on that VPS then get the clerance with that proxy"
you can also just use chatgpt and ask: "how do I get the user agent of my browser" and "how do I see the cookies a site is using"
Bare with me please, i got my useragent and put it in phind.user_agent '" and im on phind.com, how am i supposed to get my clearence token to fill in the phind.cf_clearence = i suppose ?
You get it using the chrome dev tools and going to cookies
Thanks, I managed to get this running. My question is, do i need to get the cf_clearance token everytime i start the streamlit app?
Why is there only “https://www.phind.com/” in my cookies ,and I don't see the cf_clearance
can i use the chai.openai.com cf_clearance? I can see two cf_clearance from the cookies
maybe the project need a clearer tutorial /(ㄒoㄒ)/~~
EDIT 2: https://github.com/xtekky/gpt4free/issues/153 - The use of phind.com is now forbidden
EDIT 1: This is just a band-aid solution to make it work. Also, please keep in mind that the cf_clearance
expires and you need to refresh the page to get a new one.
Here:
Step 1: Go to phind.com and get your cf_clearance via inspect element -> application -> storage -> cookies (see the images of others above) -- They expire.
Step 2: Get your user-agent
here
Step 3: Edit your phind folder's __init__.py
(basically copy this and follow through. I've removed some of the lines, you may compare them later with the original code.
cf_clearance
on lines: 79
, 139
, and 203
user-agent
on lines: 88
, 146
, and 212
from urllib.parse import quote
from time import time
from datetime import datetime
from queue import Queue, Empty
from threading import Thread
from re import findall
from curl_cffi.requests import post
cf_clearance = '' # No need to put anything here (gets wiped) user_agent = '' # No need to put anything here (gets wiped)
class PhindResponse:
class Completion:
class Choices:
def __init__(self, choice: dict) -> None:
self.text = choice['text']
self.content = self.text.encode()
self.index = choice['index']
self.logprobs = choice['logprobs']
self.finish_reason = choice['finish_reason']
def __repr__(self) -> str:
return f'''<__main__.APIResponse.Completion.Choices(\n text = {self.text.encode()},\n index = {self.index},\n logprobs = {self.logprobs},\n finish_reason = {self.finish_reason})object at 0x1337>'''
def __init__(self, choices: dict) -> None:
self.choices = [self.Choices(choice) for choice in choices]
class Usage:
def __init__(self, usage_dict: dict) -> None:
self.prompt_tokens = usage_dict['prompt_tokens']
self.completion_tokens = usage_dict['completion_tokens']
self.total_tokens = usage_dict['total_tokens']
def __repr__(self):
return f'''<__main__.APIResponse.Usage(\n prompt_tokens = {self.prompt_tokens},\n completion_tokens = {self.completion_tokens},\n total_tokens = {self.total_tokens})object at 0x1337>'''
def __init__(self, response_dict: dict) -> None:
self.response_dict = response_dict
self.id = response_dict['id']
self.object = response_dict['object']
self.created = response_dict['created']
self.model = response_dict['model']
self.completion = self.Completion(response_dict['choices'])
self.usage = self.Usage(response_dict['usage'])
def json(self) -> dict:
return self.response_dict
class Search: def create(prompt: str, actualSearch: bool = True, language: str = 'en') -> dict: # None = no search
if not actualSearch:
return {
'_type': 'SearchResponse',
'queryContext': {
'originalQuery': prompt
},
'webPages': {
'webSearchUrl': f'https://www.bing.com/search?q={quote(prompt)}',
'totalEstimatedMatches': 0,
'value': []
},
'rankingResponse': {
'mainline': {
'items': []
}
}
}
headers = {
'authority': 'www.phind.com',
'accept': '*/*',
'accept-language': 'en,fr-FR;q=0.9,fr;q=0.8,es-ES;q=0.7,es;q=0.6,en-US;q=0.5,am;q=0.4,de;q=0.3',
'cookie': f'cf_clearance=PASTEHERE',
'origin': 'https://www.phind.com',
'referer': 'https://www.phind.com/search?q=hi&c=&source=searchbox&init=true',
'sec-ch-ua': '"Chromium";v="112", "Google Chrome";v="112", "Not:A-Brand";v="99"',
'sec-ch-ua-mobile': '?0',
'sec-ch-ua-platform': '"macOS"',
'sec-fetch-dest': 'empty',
'sec-fetch-mode': 'cors',
'sec-fetch-site': 'same-origin',
'user-agent': '' # PASTE HERE
}
return post('https://www.phind.com/api/bing/search', headers = headers, json = {
'q': prompt,
'userRankList': {},
'browserLanguage': language}).json()['rawBingResults']
class Completion: def create( model = 'gpt-4', prompt: str = '', results: dict = None, creative: bool = False, detailed: bool = False, codeContext: str = '', language: str = 'en') -> PhindResponse:
if results is None:
results = Search.create(prompt, actualSearch = True)
if len(codeContext) > 2999:
raise ValueError('codeContext must be less than 3000 characters')
models = {
'gpt-4' : 'expert',
'gpt-3.5-turbo' : 'intermediate',
'gpt-3.5': 'intermediate',
}
json_data = {
'question' : prompt,
'bingResults' : results, #response.json()['rawBingResults'],
'codeContext' : codeContext,
'options': {
'skill' : models[model],
'date' : datetime.now().strftime("%d/%m/%Y"),
'language': language,
'detailed': detailed,
'creative': creative
}
}
headers = {
'authority': 'www.phind.com',
'accept': '*/*',
'accept-language': 'en,fr-FR;q=0.9,fr;q=0.8,es-ES;q=0.7,es;q=0.6,en-US;q=0.5,am;q=0.4,de;q=0.3',
'content-type': 'application/json',
'cookie': f'cf_clearance=PASTEHERE',
'origin': 'https://www.phind.com',
'referer': 'https://www.phind.com/search?q=hi&c=&source=searchbox&init=true',
'sec-ch-ua': '"Chromium";v="112", "Google Chrome";v="112", "Not:A-Brand";v="99"',
'sec-ch-ua-mobile': '?0',
'sec-ch-ua-platform': '"macOS"',
'sec-fetch-dest': 'empty',
'sec-fetch-mode': 'cors',
'sec-fetch-site': 'same-origin',
'user-agent': '' # PASTE HERE
}
completion = ''
response = post('https://www.phind.com/api/infer/answer', headers = headers, json = json_data, timeout=99999, impersonate='chrome110')
for line in response.text.split('\r\n\r\n'):
completion += (line.replace('data: ', ''))
return PhindResponse({
'id' : f'cmpl-1337-{int(time())}',
'object' : 'text_completion',
'created': int(time()),
'model' : models[model],
'choices': [{
'text' : completion,
'index' : 0,
'logprobs' : None,
'finish_reason' : 'stop'
}],
'usage': {
'prompt_tokens' : len(prompt),
'completion_tokens' : len(completion),
'total_tokens' : len(prompt) + len(completion)
}
})
class StreamingCompletion: message_queue = Queue() stream_completed = False
def request(model, prompt, results, creative, detailed, codeContext, language) -> None:
models = {
'gpt-4' : 'expert',
'gpt-3.5-turbo' : 'intermediate',
'gpt-3.5': 'intermediate',
}
json_data = {
'question' : prompt,
'bingResults' : results,
'codeContext' : codeContext,
'options': {
'skill' : models[model],
'date' : datetime.now().strftime("%d/%m/%Y"),
'language': language,
'detailed': detailed,
'creative': creative
}
}
headers = {
'authority': 'www.phind.com',
'accept': '*/*',
'accept-language': 'en,fr-FR;q=0.9,fr;q=0.8,es-ES;q=0.7,es;q=0.6,en-US;q=0.5,am;q=0.4,de;q=0.3',
'content-type': 'application/json',
'cookie': f'cf_clearance=PASTEHERE',
'origin': 'https://www.phind.com',
'referer': 'https://www.phind.com/search?q=hi&c=&source=searchbox&init=true',
'sec-ch-ua': '"Chromium";v="112", "Google Chrome";v="112", "Not:A-Brand";v="99"',
'sec-ch-ua-mobile': '?0',
'sec-ch-ua-platform': '"macOS"',
'sec-fetch-dest': 'empty',
'sec-fetch-mode': 'cors',
'sec-fetch-site': 'same-origin',
'user-agent': '' # PASTE HERE
}
response = post('https://www.phind.com/api/infer/answer',
headers = headers, json = json_data, timeout=99999, impersonate='chrome110', content_callback=StreamingCompletion.handle_stream_response)
StreamingCompletion.stream_completed = True
@staticmethod
def create(
model : str = 'gpt-4',
prompt : str = '',
results : dict = None,
creative : bool = False,
detailed : bool = False,
codeContext : str = '',
language : str = 'en'):
if results is None:
results = Search.create(prompt, actualSearch = True)
if len(codeContext) > 2999:
raise ValueError('codeContext must be less than 3000 characters')
Thread(target = StreamingCompletion.request, args = [
model, prompt, results, creative, detailed, codeContext, language]).start()
while StreamingCompletion.stream_completed != True or not StreamingCompletion.message_queue.empty():
try:
chunk = StreamingCompletion.message_queue.get(timeout=0)
if chunk == b'data: \r\ndata: \r\ndata: \r\n\r\n':
chunk = b'data: \n\n\r\n\r\n'
chunk = chunk.decode()
chunk = chunk.replace('data: \r\n\r\ndata: ', 'data: \n')
chunk = chunk.replace('\r\ndata: \r\ndata: \r\n\r\n', '\n\n\r\n\r\n')
chunk = chunk.replace('data: ', '').replace('\r\n\r\n', '')
yield PhindResponse({
'id' : f'cmpl-1337-{int(time())}',
'object' : 'text_completion',
'created': int(time()),
'model' : model,
'choices': [{
'text' : chunk,
'index' : 0,
'logprobs' : None,
'finish_reason' : 'stop'
}],
'usage': {
'prompt_tokens' : len(prompt),
'completion_tokens' : len(chunk),
'total_tokens' : len(prompt) + len(chunk)
}
})
except Empty:
pass
@staticmethod
def handle_stream_response(response):
StreamingCompletion.message_queue.put(response)
![image](https://user-images.githubusercontent.com/29201095/234626169-0ff36e98-5e2b-4144-a48a-4094e53d866e.png)
P.S. Improvements here is probably setting the keys via streamlit's implementation of `session_state` kinda looks like the keys get wiped every call for some reason
@arianyambao is it working normally for you as of this moment? im still getting the error
@arianyambao im getting this error
i examined the response, it returns an html page, looks like its not passing the cloudflare captcha
Im also getting the same error, "An error occured, please make sure you are using a cf_clearance token and correct useragent | Expecting value: line 1 column 1 (char 0)"
I've made sure my useragent and cf_clearance are both filled in correctly as well
;-;
My solution was to set phind.cf_clearance
and phind.user_agent
on lines 4 and 5 in streamlit_app.py
@jarvisar Thank you, and you're right. But after trying several times, the similar errors still occur and you must gain cf_clearance and copy it again. It doesn't seem very convenient.
@jarvisar Thank you, and you're right. But after trying several times, the similar errors still occur and you must gain cf_clearance and copy it. It doesn't seem very convenient.
Yes, the clearance, similar to cookies expires and gets renewed...
@AbdelrhmanNile have you tried pasting a new cf_clearance? It might be because the one you use expired... Expires after some time cc: @suckurmom @BastienYTB
Since I also got the same error after my token expired. (It's really not efficient)
The local version works, but slower than the real chatgpt
编辑:这只是使其工作的创可贴解决方案。另外,请记住,过期,您需要刷新页面才能获得新页面。在这里:第 1 步:转到 phind.com 并通过检查元素获取您的cf_clearance ->应用程序 ->存储 -> cookie(请参阅上面其他人的图像)-- 它们过期。第 2 步:在这里获取您的 第 3 步:编辑您的 phind 文件夹(基本上复制此文件夹并继续执行。我已经删除了一些行,您可以稍后将它们与原始代码进行比较。
cf_clearance``user-agent``__init__.py
- 手动粘贴您的行:、 和
cf_clearance``79``139``203
- 手动粘贴您的行:、 和
user-agent``88``146``212
from urllib.parse import quote from time import time from datetime import datetime from queue import Queue, Empty from threading import Thread from re import findall from curl_cffi.requests import post cf_clearance = '' # No need to put anything here (gets wiped) user_agent = '' # No need to put anything here (gets wiped) class PhindResponse: class Completion: class Choices: def __init__(self, choice: dict) -> None: self.text = choice['text'] self.content = self.text.encode() self.index = choice['index'] self.logprobs = choice['logprobs'] self.finish_reason = choice['finish_reason'] def __repr__(self) -> str: return f'''<__main__.APIResponse.Completion.Choices(\n text = {self.text.encode()},\n index = {self.index},\n logprobs = {self.logprobs},\n finish_reason = {self.finish_reason})object at 0x1337>''' def __init__(self, choices: dict) -> None: self.choices = [self.Choices(choice) for choice in choices] class Usage: def __init__(self, usage_dict: dict) -> None: self.prompt_tokens = usage_dict['prompt_tokens'] self.completion_tokens = usage_dict['completion_tokens'] self.total_tokens = usage_dict['total_tokens'] def __repr__(self): return f'''<__main__.APIResponse.Usage(\n prompt_tokens = {self.prompt_tokens},\n completion_tokens = {self.completion_tokens},\n total_tokens = {self.total_tokens})object at 0x1337>''' def __init__(self, response_dict: dict) -> None: self.response_dict = response_dict self.id = response_dict['id'] self.object = response_dict['object'] self.created = response_dict['created'] self.model = response_dict['model'] self.completion = self.Completion(response_dict['choices']) self.usage = self.Usage(response_dict['usage']) def json(self) -> dict: return self.response_dict class Search: def create(prompt: str, actualSearch: bool = True, language: str = 'en') -> dict: # None = no search if not actualSearch: return { '_type': 'SearchResponse', 'queryContext': { 'originalQuery': prompt }, 'webPages': { 'webSearchUrl': f'https://www.bing.com/search?q={quote(prompt)}', 'totalEstimatedMatches': 0, 'value': [] }, 'rankingResponse': { 'mainline': { 'items': [] } } } headers = { 'authority': 'www.phind.com', 'accept': '*/*', 'accept-language': 'en,fr-FR;q=0.9,fr;q=0.8,es-ES;q=0.7,es;q=0.6,en-US;q=0.5,am;q=0.4,de;q=0.3', 'cookie': f'cf_clearance=PASTEHERE', 'origin': 'https://www.phind.com', 'referer': 'https://www.phind.com/search?q=hi&c=&source=searchbox&init=true', 'sec-ch-ua': '"Chromium";v="112", "Google Chrome";v="112", "Not:A-Brand";v="99"', 'sec-ch-ua-mobile': '?0', 'sec-ch-ua-platform': '"macOS"', 'sec-fetch-dest': 'empty', 'sec-fetch-mode': 'cors', 'sec-fetch-site': 'same-origin', 'user-agent': '' # PASTE HERE } return post('https://www.phind.com/api/bing/search', headers = headers, json = { 'q': prompt, 'userRankList': {}, 'browserLanguage': language}).json()['rawBingResults'] class Completion: def create( model = 'gpt-4', prompt: str = '', results: dict = None, creative: bool = False, detailed: bool = False, codeContext: str = '', language: str = 'en') -> PhindResponse: if results is None: results = Search.create(prompt, actualSearch = True) if len(codeContext) > 2999: raise ValueError('codeContext must be less than 3000 characters') models = { 'gpt-4' : 'expert', 'gpt-3.5-turbo' : 'intermediate', 'gpt-3.5': 'intermediate', } json_data = { 'question' : prompt, 'bingResults' : results, #response.json()['rawBingResults'], 'codeContext' : codeContext, 'options': { 'skill' : models[model], 'date' : datetime.now().strftime("%d/%m/%Y"), 'language': language, 'detailed': detailed, 'creative': creative } } headers = { 'authority': 'www.phind.com', 'accept': '*/*', 'accept-language': 'en,fr-FR;q=0.9,fr;q=0.8,es-ES;q=0.7,es;q=0.6,en-US;q=0.5,am;q=0.4,de;q=0.3', 'content-type': 'application/json', 'cookie': f'cf_clearance=PASTEHERE', 'origin': 'https://www.phind.com', 'referer': 'https://www.phind.com/search?q=hi&c=&source=searchbox&init=true', 'sec-ch-ua': '"Chromium";v="112", "Google Chrome";v="112", "Not:A-Brand";v="99"', 'sec-ch-ua-mobile': '?0', 'sec-ch-ua-platform': '"macOS"', 'sec-fetch-dest': 'empty', 'sec-fetch-mode': 'cors', 'sec-fetch-site': 'same-origin', 'user-agent': '' # PASTE HERE } completion = '' response = post('https://www.phind.com/api/infer/answer', headers = headers, json = json_data, timeout=99999, impersonate='chrome110') for line in response.text.split('\r\n\r\n'): completion += (line.replace('data: ', '')) return PhindResponse({ 'id' : f'cmpl-1337-{int(time())}', 'object' : 'text_completion', 'created': int(time()), 'model' : models[model], 'choices': [{ 'text' : completion, 'index' : 0, 'logprobs' : None, 'finish_reason' : 'stop' }], 'usage': { 'prompt_tokens' : len(prompt), 'completion_tokens' : len(completion), 'total_tokens' : len(prompt) + len(completion) } }) class StreamingCompletion: message_queue = Queue() stream_completed = False def request(model, prompt, results, creative, detailed, codeContext, language) -> None: models = { 'gpt-4' : 'expert', 'gpt-3.5-turbo' : 'intermediate', 'gpt-3.5': 'intermediate', } json_data = { 'question' : prompt, 'bingResults' : results, 'codeContext' : codeContext, 'options': { 'skill' : models[model], 'date' : datetime.now().strftime("%d/%m/%Y"), 'language': language, 'detailed': detailed, 'creative': creative } } headers = { 'authority': 'www.phind.com', 'accept': '*/*', 'accept-language': 'en,fr-FR;q=0.9,fr;q=0.8,es-ES;q=0.7,es;q=0.6,en-US;q=0.5,am;q=0.4,de;q=0.3', 'content-type': 'application/json', 'cookie': f'cf_clearance=PASTEHERE', 'origin': 'https://www.phind.com', 'referer': 'https://www.phind.com/search?q=hi&c=&source=searchbox&init=true', 'sec-ch-ua': '"Chromium";v="112", "Google Chrome";v="112", "Not:A-Brand";v="99"', 'sec-ch-ua-mobile': '?0', 'sec-ch-ua-platform': '"macOS"', 'sec-fetch-dest': 'empty', 'sec-fetch-mode': 'cors', 'sec-fetch-site': 'same-origin', 'user-agent': '' # PASTE HERE } response = post('https://www.phind.com/api/infer/answer', headers = headers, json = json_data, timeout=99999, impersonate='chrome110', content_callback=StreamingCompletion.handle_stream_response) StreamingCompletion.stream_completed = True @staticmethod def create( model : str = 'gpt-4', prompt : str = '', results : dict = None, creative : bool = False, detailed : bool = False, codeContext : str = '', language : str = 'en'): if results is None: results = Search.create(prompt, actualSearch = True) if len(codeContext) > 2999: raise ValueError('codeContext must be less than 3000 characters') Thread(target = StreamingCompletion.request, args = [ model, prompt, results, creative, detailed, codeContext, language]).start() while StreamingCompletion.stream_completed != True or not StreamingCompletion.message_queue.empty(): try: chunk = StreamingCompletion.message_queue.get(timeout=0) if chunk == b'data: \r\ndata: \r\ndata: \r\n\r\n': chunk = b'data: \n\n\r\n\r\n' chunk = chunk.decode() chunk = chunk.replace('data: \r\n\r\ndata: ', 'data: \n') chunk = chunk.replace('\r\ndata: \r\ndata: \r\n\r\n', '\n\n\r\n\r\n') chunk = chunk.replace('data: ', '').replace('\r\n\r\n', '') yield PhindResponse({ 'id' : f'cmpl-1337-{int(time())}', 'object' : 'text_completion', 'created': int(time()), 'model' : model, 'choices': [{ 'text' : chunk, 'index' : 0, 'logprobs' : None, 'finish_reason' : 'stop' }], 'usage': { 'prompt_tokens' : len(prompt), 'completion_tokens' : len(chunk), 'total_tokens' : len(prompt) + len(chunk) } }) except Empty: pass @staticmethod def handle_stream_response(response): StreamingCompletion.message_queue.put(response)
附言这里的改进可能是通过 streamlit 的实现设置键,看起来由于某种原因每次调用都会擦除键
session_state
sstillll not work
Check this issue #153 , phind.com is forbidden now
After updating my cookies it seems to work now, thanks
This problem is so serious that the project cannot be deployed. Most of us have encountered this problem. Please come out and give a correct answer