search

jhao104 / proxy_pool

Python ProxyPool for web spider
https://jhao104.github.io/proxy_pool/
MIT License
20.85k stars 5.05k forks source link

ConnectionRefusedError: [WinError 10061] 由于目标计算机积极拒绝，无法连接。 #779

Closed reoLantern closed 6 months ago

reoLantern commented 6 months ago

我克隆代码后，直接在项目目录运行docker-compose up，看起来是开始正确运行了。运行测试代码：

import requests

def get_proxy():
    return requests.get("http://127.0.0.1:5010/get/").json()

get_proxy()

报错：

ConnectionRefusedError: [WinError 10061] 由于目标计算机积极拒绝，无法连接。

以下是运行docker-compose up的部分记录：

PS C:\Users\***\Project\proxy_pool> docker-compose up
[+] Running 2/2
 ✔ Container proxy_redis  Recreated                                                                                                                                                                                             0.1s 
 ✔ Container proxy_pool   Recreated                                                                                                                                                                                             0.1s 
Attaching to proxy_pool, proxy_redis
proxy_redis  | 1:C 26 Dec 2023 10:15:56.573 # WARNING Memory overcommit must be enabled! Without it, a background save or replication may fail under low memory condition. Being disabled, it can also cause failures without low memory condition, see https://github.com/jemalloc/jemalloc/issues/1328. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
proxy_redis  | 1:C 26 Dec 2023 10:15:56.573 * oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
proxy_redis  | 1:C 26 Dec 2023 10:15:56.573 * Redis version=7.2.3, bits=64, commit=00000000, modified=0, pid=1, just started
proxy_redis  | 1:C 26 Dec 2023 10:15:56.573 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
proxy_redis  | 1:M 26 Dec 2023 10:15:56.573 * monotonic clock: POSIX clock_gettime
proxy_redis  | 1:M 26 Dec 2023 10:15:56.574 * Running mode=standalone, port=6379.
proxy_redis  | 1:M 26 Dec 2023 10:15:56.587 * Server initialized
proxy_redis  | 1:M 26 Dec 2023 10:15:56.587 * Loading RDB produced by version 7.2.3
proxy_redis  | 1:M 26 Dec 2023 10:15:56.587 * RDB age 193 seconds
proxy_redis  | 1:M 26 Dec 2023 10:15:56.587 * RDB memory usage when created 0.94 Mb
proxy_redis  | 1:M 26 Dec 2023 10:15:56.587 * Done loading RDB, keys loaded: 1, keys expired: 0.
proxy_redis  | 1:M 26 Dec 2023 10:15:56.587 * DB loaded from disk: 0.000 seconds
proxy_redis  | 1:M 26 Dec 2023 10:15:56.587 * Ready to accept connections tcp
: not found  | start.sh: line 2: 
proxy_pool   | 2023-12-26 18:15:57,074 launcher.py[line:45] INFO ProxyPool Version: 2.4.0
proxy_pool   | 2023-12-26 18:15:57,074 launcher.py[line:45] INFO ProxyPool Version: 2.4.0
proxy_pool   |
proxy_pool   | ****************************************************************
proxy_pool   | *** ______  ********************* ______ *********** _  ********
proxy_pool   | *** | ___ \_ ******************** | ___ \ ********* | | ********
proxy_pool   | *** | |_/ / \__ __   __  _ __   _ | |_/ /___ * ___  | | ********
proxy_pool   | *** |  __/|  _// _ \ \ \/ /| | | ||  __// _ \ / _ \ | | ********
proxy_pool   | *** | |   | | | (_) | >  < \ |_| || |  | (_) | (_) || |___  ****
proxy_pool   | *** \_|   |_|  \___/ /_/\_\ \__  |\_|   \___/ \___/ \_____/ ****
proxy_pool   | ****                       __ / /                          *****
proxy_pool   | ************************* /___ / *******************************
proxy_pool   | *************************       ********************************
proxy_pool   | ****************************************************************
proxy_pool   |
proxy_pool   |
proxy_pool   | ****************************************************************
proxy_pool   | *** ______  ********************* ______ *********** _  ********
proxy_pool   | *** | ___ \_ ******************** | ___ \ ********* | | ********
proxy_pool   | *** | |_/ / \__ __   __  _ __   _ | |_/ /___ * ___  | | ********
proxy_pool   | *** |  __/|  _// _ \ \ \/ /| | | ||  __// _ \ / _ \ | | ********
proxy_pool   | *** | |   | | | (_) | >  < \ |_| || |  | (_) | (_) || |___  ****
proxy_pool   | *** \_|   |_|  \___/ /_/\_\ \__  |\_|   \___/ \___/ \_____/ ****
proxy_pool   | ****                       __ / /                          *****
proxy_pool   | ************************* /___ / *******************************
proxy_pool   | *************************       ********************************
proxy_pool   | ****************************************************************
proxy_pool   |
proxy_pool   | 2023-12-26 18:15:57,074 launcher.py[line:50] INFO ProxyPool configure HOST: 0.0.0.0
proxy_pool   | 2023-12-26 18:15:57,074 launcher.py[line:50] INFO ProxyPool configure HOST: 0.0.0.0
proxy_pool   | 2023-12-26 18:15:57,074 launcher.py[line:51] INFO ProxyPool configure PORT: 5010
proxy_pool   | 2023-12-26 18:15:57,074 launcher.py[line:51] INFO ProxyPool configure PORT: 5010
proxy_pool   | 2023-12-26 18:15:57,075 launcher.py[line:52] INFO ProxyPool configure PROXY_FETCHER: ['freeProxy01', 'freeProxy02', 'freeProxy03', 'freeProxy04', 'freeProxy05', 'freeProxy06', 'freeProxy07', 'freeProxy08', 'freeProxy09', 'freeProxy10', 'freeProxy11']
proxy_pool   | 2023-12-26 18:15:57,075 launcher.py[line:52] INFO ProxyPool configure PROXY_FETCHER: ['freeProxy01', 'freeProxy02', 'freeProxy03', 'freeProxy04', 'freeProxy05', 'freeProxy06', 'freeProxy07', 'freeProxy08', 'freeProxy09', 'freeProxy10', 'freeProxy11']
proxy_pool   | 2023-12-26 18:15:57,111 launcher.py[line:58] INFO ============ DATABASE CONFIGURE ================
proxy_pool   | 2023-12-26 18:15:57,112 launcher.py[line:59] INFO DB_TYPE: REDIS
proxy_pool   | 2023-12-26 18:15:57,112 launcher.py[line:58] INFO ============ DATABASE CONFIGURE ================
proxy_pool   | 2023-12-26 18:15:57,112 launcher.py[line:60] INFO DB_HOST: proxy_redis
proxy_pool   | 2023-12-26 18:15:57,112 launcher.py[line:61] INFO DB_PORT: 6379
proxy_pool   | 2023-12-26 18:15:57,112 launcher.py[line:59] INFO DB_TYPE: REDIS
proxy_pool   | 2023-12-26 18:15:57,112 launcher.py[line:62] INFO DB_NAME: 0
proxy_pool   | 2023-12-26 18:15:57,112 launcher.py[line:60] INFO DB_HOST: proxy_redis
proxy_pool   | 2023-12-26 18:15:57,112 launcher.py[line:63] INFO DB_USER:
proxy_pool   | 2023-12-26 18:15:57,112 launcher.py[line:61] INFO DB_PORT: 6379
proxy_pool   | 2023-12-26 18:15:57,112 launcher.py[line:62] INFO DB_NAME: 0
proxy_pool   | 2023-12-26 18:15:57,112 launcher.py[line:64] INFO =================================================
proxy_pool   | 2023-12-26 18:15:57,112 launcher.py[line:63] INFO DB_USER:
proxy_pool   | 2023-12-26 18:15:57,113 launcher.py[line:64] INFO =================================================
proxy_pool   | [2023-12-26 18:15:57 +0800] [7] [INFO] Starting gunicorn 19.9.0
proxy_pool   | [2023-12-26 18:15:57 +0800] [7] [INFO] Listening at: http://0.0.0.0:5010 (7)
proxy_pool   | [2023-12-26 18:15:57 +0800] [7] [INFO] Using worker: sync
proxy_pool   | [2023-12-26 18:15:57 +0800] [19] [INFO] Booting worker with pid: 19
proxy_pool   | 2023-12-26 18:15:57,332 fetch.py[line:65] INFO ProxyFetch : start
proxy_pool   | 2023-12-26 18:15:57,333 fetch.py[line:68] INFO ProxyFetch - freeProxy01: start
proxy_pool   | 2023-12-26 18:15:57,333 fetch.py[line:68] INFO ProxyFetch - freeProxy02: start
proxy_pool   | 2023-12-26 18:15:57,333 fetch.py[line:68] INFO ProxyFetch - freeProxy03: start
proxy_pool   | 2023-12-26 18:15:57,333 fetch.py[line:68] INFO ProxyFetch - freeProxy04: start
proxy_pool   | 2023-12-26 18:15:57,333 fetch.py[line:68] INFO ProxyFetch - freeProxy05: start
proxy_pool   | 2023-12-26 18:15:57,334 fetch.py[line:68] INFO ProxyFetch - freeProxy06: start
proxy_pool   | 2023-12-26 18:15:57,334 fetch.py[line:68] INFO ProxyFetch - freeProxy07: start
proxy_pool   | 2023-12-26 18:15:57,334 fetch.py[line:68] INFO ProxyFetch - freeProxy08: start
proxy_pool   | 2023-12-26 18:15:57,334 fetch.py[line:68] INFO ProxyFetch - freeProxy09: start
proxy_pool   | 2023-12-26 18:15:57,334 fetch.py[line:68] INFO ProxyFetch - freeProxy10: start
proxy_pool   | 2023-12-26 18:15:57,334 fetch.py[line:68] INFO ProxyFetch - freeProxy11: start
proxy_pool   | 2023-12-26 18:15:57,335 fetch.py[line:36] INFO ProxyFetch - freeProxy01: start
proxy_pool   | 2023-12-26 18:15:57,336 fetch.py[line:36] INFO ProxyFetch - freeProxy02: start
proxy_pool   | 2023-12-26 18:15:57,336 fetch.py[line:36] INFO ProxyFetch - freeProxy03: start
proxy_pool   | 2023-12-26 18:15:57,340 fetch.py[line:36] INFO ProxyFetch - freeProxy04: start
proxy_pool   | 2023-12-26 18:15:57,342 fetch.py[line:36] INFO ProxyFetch - freeProxy05: start
proxy_pool   | 2023-12-26 18:15:57,345 fetch.py[line:36] INFO ProxyFetch - freeProxy06: start
proxy_pool   | 2023-12-26 18:15:57,345 fetch.py[line:36] INFO ProxyFetch - freeProxy07: start
proxy_pool   | 2023-12-26 18:15:57,347 fetch.py[line:36] INFO ProxyFetch - freeProxy08: start
proxy_pool   | 2023-12-26 18:15:57,349 fetch.py[line:36] INFO ProxyFetch - freeProxy09: start
proxy_pool   | 2023-12-26 18:15:57,350 fetch.py[line:36] INFO ProxyFetch - freeProxy10: start
proxy_pool   | 2023-12-26 18:15:57,352 fetch.py[line:36] INFO ProxyFetch - freeProxy11: start
reoLantern commented 6 months ago

使用docker的话，本机不需要安装redis了吧？相关配置（几乎没改）：

image

image

reoLantern commented 6 months ago

docker和网络的运行情况：

PS C:\Users\***\Project\proxy_pool> docker ps
CONTAINER ID   IMAGE                   COMMAND                   CREATED          STATUS          PORTS                    NAMES
287be47c0172   proxy_pool-proxy_pool   "sh start.sh"             20 minutes ago   Up 20 minutes   0.0.0.0:5010->5010/tcp   proxy_pool
a1adc60ac102   redis                   "docker-entrypoint.s…"   20 minutes ago   Up 20 minutes   6379/tcp                 proxy_redis
PS C:\Users\***\Project\proxy_pool> docker network ls
NETWORK ID     NAME                 DRIVER    SCOPE
63d76db92249   bridge               bridge    local
2f076407971b   host                 host      local
0896c8c3c7e7   none                 null      local
e309f2709a14   proxy_pool_default   bridge    local
reoLantern commented 6 months ago

在本机运行：telnet 127.0.0.1 5010，返回：

正在连接127.0.0.1...无法打开到主机的连接。 在端口 5010: 连接失败

但是进入proxy_pool容器：

/app # nc -vz 127.0.0.1 5010
127.0.0.1 (127.0.0.1:5010) open
reoLantern commented 6 months ago

通过wsl --update --web-download将wsl更新到最新版本后解决问题。原因可能是现在的Docker默认是基于wsl的，某个版本的wsl有问题。