search

puppylpg / oddish

Crawl csgo skin info from `buff.163.com` and steam, then find the most suitable one to buy from the former and to sell to the latter.
https://puppylpg.github.io/2019/12/07/python-crawler-buff-optimaze/
GNU General Public License v3.0
323 stars 81 forks source link

[WinError 10061] 由于目标计算机积极拒绝，无法连接 #34

Closed xydmhy closed 3 years ago

xydmhy commented 4 years ago

问题描述

爬前面一些数据还好 后面就突然开始报错 cmd窗口有时候会很长时间不会更新

复现步骤

报错信息

2020-10-26 13:41:44,976 [INFO ] GET steam history price 8/1653 for (P2000（StatTrak™） | 火灵 (久经沙场)): https://steamcommunity.com/market/pricehistory/?country=CN&currency=1&appid=730&market_hash_name=StatTrak%E2%84%A2%20P2000%20%7C%20Fire%20Elemental%20%28Field-Tested%29 2020-10-26 13:41:44,977 [INFO ] sleep 2s at 2020-10-26 13:41:44.977977 2020-10-26 13:41:49,028 [ERROR] unknown error for https://steamcommunity.com/market/pricehistory/?country=CN&currency=1&appid=730&market_hash_name=StatTrak%E2%84%A2%20P2000%20%7C%20Fire%20Elemental%20%28Field-Tested%29. Try again. Error string: SOCKSHTTPSConnectionPool(host='steamcommunity.com', port=443): Max retries exceeded with url: /market/pricehistory/?country=CN&currency=1&appid=730&market_hash_name=StatTrak%E2%84%A2%20P2000%20%7C%20Fire%20Elemental%20%28Field-Tested%29 (Caused by NewConnectionError('<urllib3.contrib.socks.SOCKSHTTPSConnection object at 0x00000265C1886880>: Failed to establish a new connection: [WinError 10061] 由于目标计算机积极拒绝，无法连接。')) 2020-10-26 13:41:49,030 [ERROR] Traceback (most recent call last): File "E:\Atuna\lib\site-packages\socks.py", line 787, in connect super(socksocket, self).connect(proxy_addr) ConnectionRefusedError: [WinError 10061] 由于目标计算机积极拒绝，无法连接。

相关截屏（最好贴一下）

软件信息

请完善以下信息：

puppylpg commented 4 years ago

额， #33 貌似是同样的问题，我晚上回去看看，可以先下载一个3.6.1版的用着

puppylpg commented 4 years ago

另外看看是不是代理配置的有问题

xydmhy commented 4 years ago

另外看看是不是代理配置的有问题

steam开了梯子才上了市场 后来梯子好像断了 代理没改动过 要改成什么吗

puppylpg commented 4 years ago

另外看看是不是代理配置的有问题

steam开了梯子才上了市场 后来梯子好像断了 代理没改动过 要改成什么吗

先保证代理没问题吧，代理有问题是没法从steam上爬数据的

xydmhy commented 4 years ago

另外看看是不是代理配置的有问题

steam开了梯子才上了市场 后来梯子好像断了 代理没改动过 要改成什么吗

先保证代理没问题吧，代理有问题是没法从steam上爬数据的

怎么查看本地端口呀 用的sstp

ccinv commented 4 years ago

@xydmhy 使用sstp时直接把代理留空，vpn会自动转发

xydmhy commented 4 years ago

@xydmhy使用sstp时直接把代理留空，vpn会自动转发

代理可以了 挂一会又出了新的错误 raise TypeError(f'the JSON object must be str, bytes or bytearray, ' TypeError: the JSON object must be str, bytes or bytearray, not dict

ccinv commented 4 years ago

@xydmhy使用sstp时直接把代理留空，vpn会自动转发

代理可以了 挂一会又出了新的错误 raise TypeError(f'the JSON object must be str, bytes or bytearray, ' TypeError: the JSON object must be str, bytes or bytearray, not dict

pull一下最新的commit，应该解决了这个问题

wbsddgg commented 4 years ago 
raise ConnectionError(e, request=request)

requests.exceptions.ConnectionError: SOCKSHTTPSConnectionPool(host='steamcommunity.com', port=443): Max retries exceeded with url: /market/pricehistory/?appid=730&market_hash_name=Glock-18%20%7C%20Synth%20Leaf%20%28Minimal%20Wear%29 (Caused by NewConnectionError('<urllib3.contrib.socks.SOCKSHTTPSConnection object at 0x000002C78F6B7B20>: Failed to establish a new connection: [WinError 10061] 由于目标计算机积极拒绝，无法连接。'))

2020-11-03 21:57:06,538 [ERROR] Timeout for https://steamcommunity.com/market/pricehistory/?appid=730&market_hash_name=Glock-18%20%7C%20Synth%20Leaf%20%28Minimal%20Wear%29 beyond the maximum(4) retry times. SKIP! 2020-11-03 21:57:06,538 [ERROR] Traceback (most recent call last): File "C:\Users\Charles\oddish-master\src\crawl\history_price_crawler.py", line 42, in crawl_history_price crawl_item_history_price(index, item, total_price_number) File "C:\Users\Charles\oddish-master\src\crawl\history_price_crawler.py", line 14, in crawl_item_history_price steam_history_prices = get_json_dict(steam_price_url, steam_cookies, True) File "C:\Users\Charles\oddish-master\src\util\requester.py", line 63, in get_json_dict store(url,json_data) File "C:\Users\Charles\oddish-master\src\util\cache.py", line 46, in store f.write(data) TypeError: write() argument must be str, not None

2020-11-03 21:57:06,538 [INFO ] GET steam history price 39/713 for (格洛克 18 型 | 暮光星系 (崭新出厂)): https://steamcommunity.com/market/pricehistory/?appid=730&market_hash_name=Glock-18%20%7C%20Twilight%20Galaxy%20%28Factory%20New%29 2020-11-03 21:57:06,538 [INFO ] sleep 4s at 2020-11-03 21:57:06.538109

奇怪了 一开始能爬下来不少，然后就总是被拒绝，但偶尔还能发现一个

ccinv commented 4 years ago 
raise ConnectionError(e, request=request)

requests.exceptions.ConnectionError: SOCKSHTTPSConnectionPool(host='steamcommunity.com', port=443): Max retries exceeded with url: /market/pricehistory/?appid=730&market_hash_name=Glock-18%20%7C%20Synth%20Leaf%20%28Minimal%20Wear%29 (Caused by NewConnectionError('<urllib3.contrib.socks.SOCKSHTTPSConnection object at 0x000002C78F6B7B20>: Failed to establish a new connection: [WinError 10061] 由于目标计算机积极拒绝，无法连接。'))

2020-11-03 21:57:06,538 [ERROR] Timeout for https://steamcommunity.com/market/pricehistory/?appid=730&market_hash_name=Glock-18%20%7C%20Synth%20Leaf%20%28Minimal%20Wear%29 beyond the maximum(4) retry times. SKIP! 2020-11-03 21:57:06,538 [ERROR] Traceback (most recent call last): File "C:\Users\Charles\oddish-master\src\crawl\history_price_crawler.py", line 42, in crawl_history_price crawl_item_history_price(index, item, total_price_number) File "C:\Users\Charles\oddish-master\src\crawl\history_price_crawler.py", line 14, in crawl_item_history_price steam_history_prices = get_json_dict(steam_price_url, steam_cookies, True) File "C:\Users\Charles\oddish-master\src\util\requester.py", line 63, in get_json_dict store(url,json_data) File "C:\Users\Charles\oddish-master\src\util\cache.py", line 46, in store f.write(data) TypeError: write() argument must be str, not None

2020-11-03 21:57:06,538 [INFO ] GET steam history price 39/713 for (格洛克 18 型 | 暮光星系 (崭新出厂)): https://steamcommunity.com/market/pricehistory/?appid=730&market_hash_name=Glock-18%20%7C%20Twilight%20Galaxy%20%28Factory%20New%29

2020-11-03 21:57:06,538 [INFO ] sleep 4s at 2020-11-03 21:57:06.538109 奇怪了 一开始能爬下来不少，然后就总是被拒绝，但偶尔还能发现一个

基本可以确认是代理稳定性问题

puppylpg commented 3 years ago

[WinError 10061] 由于目标计算机积极拒绝，无法连接

确认就是代理的问题，如果代理没开或者中途断了，就会出现这种错误，换个稳定点儿的代理吧