error while use headless=True

[hereTac]

It is all good while using headless=False, got error when use headless=true, started it with xvfb-run python3 test_s.py.

Here is the whole source code .

import time

from playwright.sync_api import sync_playwright
import sys
sys.path.append("..")
import cf_clearance
#from cf_clearance import sync_stealth, sync_cf_retry
import requests

# not use cf_clearance, cf challenge is fail
proxies = {
   "all": "http://192.168.181.104:7890"
}
res = requests.get('https://steamdb.info/charts/', proxies=proxies)
if '<title>Just a moment...</title>' in res.text:
   print("cf challenge fail")
# get cf_clearance
with sync_playwright() as p:
   browser = p.chromium.launch(headless=True, proxy={"server": "http://192.168.181.104:7890"})
   page = browser.new_page()
   cf_clearance.sync_stealth(page, pure=True)
   page.goto('https://steamdb.info/charts/')
   res = cf_clearance.sync_cf_retry(page)
   time.sleep(6)
   page.screenshot(path="example.png")
   if res:
       cookies = page.context.cookies()
       for cookie in cookies:
           if cookie.get('name') == 'cf_clearance':
               cf_clearance_value = cookie.get('value')
               print(cf_clearance_value)
       ua = page.evaluate('() => {return navigator.userAgent}')
       print(ua)
   else:
       print("cf challenge fail")
   time.sleep(10)
   browser.close()
# use cf_clearance, must be same IP and UA
headers = {"user-agent": ua}
cookies = {"cf_clearance": cf_clearance_value}
res = requests.get('https://steamdb.info/charts/', proxies=proxies, headers=headers, cookies=cookies)
if '<title>Just a moment...</title>' not in res.text:
   print("cf challenge success")

[vvanglro]

Please use interface mode, You must add headless=False. If you use it on linux or docker, use XVFB.

[hereTac]

I apologize for any confusion. Let me explain again. I used headless=True and XVFB on Linux, then start it by running xvfb-run python3 test_s.py, cloudflare response the access denied page. However, when I use headless=False and the interface mode on Ubuntu Desktop, that works fine and it give me the correct target page. @vvanglro

[vvanglro]

You can only pass the challenge with headless=false, if you are using a server with an interface, you don't need to use XVFB.

[hereTac]

In my case, my purpose is to run the script on Linux without an interface, so must be XVFB, is it right? But, unfortunated, is can not pass the challenge. (Just tested on Ubuntu Desktop and it is good with an interface.

[vvanglro]

try new code.

pip install git+https://github.com/vvanglro/cf-clearance.git@main

[hereTac]

Try it already yesterday. I build the test code in git clone https://github.com/vvanglro/cf-clearance.

[vvanglro]

Works fine, doesn't reproduce your problem.