get_daily() 雪球源数据再次错误

[Chenyang-Lu]

继上次修复后(https://github.com/refraction-ray/xalpha/issues/185 )，2024-9-4再次出现新的cookies相关错误 get_daily(SHxxxxxx),

Traceback (most recent call last): File "C:\Users\cyluc\anaconda3\envs\lof-nav-est\lib\site-packages\xalpha\universal.py", line 2007, in wrapper df0 = getattr(thismodule, "cached_dict")[key] KeyError: 'SH688017'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "get_fund_stock_hist.py", line 71, in daily = xa.get_daily(portfolio_code_list_prefix[i], prev=prev_days+10) File "C:\Users\cyluc\anaconda3\envs\lof-nav-est\lib\site-packages\xalpha\universal.py", line 2065, in wrapper df0 = f(*args, kws) File "C:\Users\cyluc\anaconda3\envs\lof-nav-est\lib\site-packages\xalpha\universal.py", line 1138, in _get_daily df = get_historicalfromxq(code, count, type=type_) File "C:\Users\cyluc\anaconda3\envs\lof-nav-est\lib\site-packages\xalpha\universal.py", line 165, in get_historical_fromxq cookies=get_token(), File "C:\Users\cyluc\anaconda3\envs\lof-nav-est\lib\site-packages\xalpha\universal.py", line 129, in newfunc return f_ttl(ttl_hash, *args, *kwargs) File "C:\Users\cyluc\anaconda3\envs\lof-nav-est\lib\site-packages\xalpha\universal.py", line 121, in time_aware return func(args, kwargs) File "C:\Users\cyluc\anaconda3\envs\lof-nav-est\lib\site-packages\xalpha\universal.py", line 145, in get_token return {"xq_a_token": r.cookies["xq_a_token"],"u": r.cookies["u"]} File "C:\Users\cyluc\anaconda3\envs\lof-nav-est\lib\site-packages\requests\cookies.py", line 334, in getitem return self._find_no_duplicates(name) File "C:\Users\cyluc\anaconda3\envs\lof-nav-est\lib\site-packages\requests\cookies.py", line 413, in _find_no_duplicates raise KeyError(f"name={name!r}, domain={domain!r}, path={path!r}") KeyError: "name='xq_a_token', domain=None, path=None"

[A-Guoo]

+1 我也是今天出现的

[cqzhao]

我是昨天出现的。看看是不是能切换到别的数据源？东方财富或者同花顺会不会好一点

[cqzhao]

还是get_token出的问题，雪球现在用js动态返回cookie了。先用selinium应应急，等ray出方案吧

[Chenyang-Lu]

@cqzhao 请问selinium应急就是开个浏览器直接驱动浏览器读数据吗？我在这方面比较外行，能否给个代码片段参考一下？谢谢！

[tmmpcn]

get_token 改下
先直接获取token

`from selenium import webdriver from selenium.webdriver.chrome.service import Service from selenium.webdriver.chrome.options import Options from webdriver_manager.chrome import ChromeDriverManager import time import json def get_xueqiu_specific_cookies():

设置 Chrome 选项

chrome_options = Options()
chrome_options.add_argument("--headless")  # 无头模式，不显示浏览器窗口
chrome_options.add_argument("--no-sandbox")
chrome_options.add_argument("--disable-dev-shm-usage")

# 初始化 WebDriver
service = Service(ChromeDriverManager().install())
driver = webdriver.Chrome(service=service, options=chrome_options)

try:
    # 访问雪球网
    driver.get("https://xueqiu.com")

    # 等待一段时间，让 JavaScript 有时间执行
    time.sleep(5)

    # 获取所有 cookies
    cookies = driver.get_cookies()

    # 只提取 'xq_a_token' 和 'u'
    specific_cookies = {}
    for cookie in cookies:
        if cookie['name'] in ['xq_a_token', 'u']:
            specific_cookies[cookie['name']] = cookie['value']

    return specific_cookies

finally:
    # 确保关闭浏览器
    driver.quit()

使用示例

if name == "main": result = get_xueqiu_specific_cookies()

写入 JSON 文件

xuqiu_cookie_file = 'xalpha_xuqiu_cookie.data'
with open(xuqiu_cookie_file, 'w') as f:
    json.dump(result, f)`

[cqzhao]

get_token 改下 先直接获取token

`from selenium import webdriver from selenium.webdriver.chrome.service import Service from selenium.webdriver.chrome.options import Options from webdriver_manager.chrome import ChromeDriverManager import time import json def get_xueqiu_specific_cookies(): # 设置 Chrome 选项 chrome_options = Options() chrome_options.add_argument("--headless") # 无头模式，不显示浏览器窗口 chrome_options.add_argument("--no-sandbox") chrome_options.add_argument("--disable-dev-shm-usage")

# 初始化 WebDriver
service = Service(ChromeDriverManager().install())
driver = webdriver.Chrome(service=service, options=chrome_options)

try:
    # 访问雪球网
    driver.get("https://xueqiu.com")

    # 等待一段时间，让 JavaScript 有时间执行
    time.sleep(5)

    # 获取所有 cookies
    cookies = driver.get_cookies()

    # 只提取 'xq_a_token' 和 'u'
    specific_cookies = {}
    for cookie in cookies:
        if cookie['name'] in ['xq_a_token', 'u']:
            specific_cookies[cookie['name']] = cookie['value']

    return specific_cookies

finally:
    # 确保关闭浏览器
    driver.quit()

使用示例

if name == "main": result = get_xueqiu_specific_cookies() # 写入 JSON 文件 xuqiu_cookie_file = 'xalpha_xuqiu_cookie.data' with open(xuqiu_cookie_file, 'w') as f: json.dump(result, f)`

就是这个思路。那个cookie存下来之后下次可以直接读了，不知道多久失效。

[tmmpcn]

失效了再运行一下就行了。。我看了下 大概一周过期

[refraction-ray]

我现在测试xueqiu是好的 xa.get_daily("SH600000"), 是恢复了还是不同网络环境网站策略不一样了？

[Chenyang-Lu]

是的，恢复了，啥动没动就好了

[refraction-ray]

又出现了问题，最新 commit 暂时修复，方案是外部配置 token，token 的获取可以是从浏览器复制或者像上面回复一样程序自动化。

xa.universal.set_token({"xq_a_token": "uuid", "u": "int"})
xa.get_daily("SH600000")

[Chenyang-Lu]

非常感谢！