search

BD777 / ipproxypool

Crawl and maintain IP proxy data, providing restful API for access | 免费IP代理池
MIT License
0 stars 0 forks source link

readme

ipproxypool

Version En | 中文

Collect and maintain IP proxy data, and provide a RESTful API for access.

Crawlers Status

Name Status Desc
89ip Detect 89ip
ip3366 Detect ip3366
kuaidaili Detect kuaidaili
ProxyListPlus Detect ProxyListPlus maybe github workflow's IP is blocked
zdaye Detect zdaye It seems that non-Chinese IP addresses will be blocked

Features

  1. Crawl IP proxies from the web and store them in a SQLite database.
  2. Periodically check if the proxies in the database are still valid and remove them if they are invalid.
  3. Provide RESTful APIs for access.

API

GET /

Query proxies.

param desc sample
type proxy type
1: transparent
2: annoymous
3: high annoymous		 3
protocol protocol type
1: HTTP
2: HTTPS
3: Both		 3
limit limit count 10

GET /count

Query total count of proxies.

POST /delete

Delete proxy from database.

param desc sample
ip ip to delete 10.10.10.10
port port 8080

Config

type Config struct {
    Mode string // "debug", "release"
    Host string // default "0.0.0.0"
    Port int    // default 9002

    CountThreshold int64 // if less than threshold, start crawler
    CrawlSleepSec  int64 // sleep seconds between crawls
    DetectSleepSec int64 // sleep seconds between detects proxy from db
}

Run

go run cmd/main.go