ZFC-Digital / cf-clearance-scraper

This library was created for testing and training purposes to retrieve the page source of websites, create Cloudflare Turnstile tokens and create Cloudflare WAF sessions.
MIT License
291 stars 55 forks source link

Issue with Proxy Usage on the New Version #44

Open ZuperpuZ opened 1 month ago

ZuperpuZ commented 1 month ago

Description

Hello,

First of all, thank you very much for your hard work and for sharing this project! It's incredibly useful.

I'm facing an issue with the latest version of the scraper when using a proxy.

Problem Description:

When I don't use a proxy, everything works perfectly. However, as soon as I try to use a proxy, I consistently get a timeout and a 500 error from Axios.

What’s strange is that with the previous version of the project, this same code worked fine with the proxy. Now, no matter what proxy I use, the issue persists. Context:

I’m using Ubuntu and running the project via Docker.

If you have any idea what might be causing this issue, or if recent changes might be related, I would greatly appreciate your help in resolving this.

Thanks again for your great work!

Best regards,

Full steps to reproduce the issue

const session = await axios.post(
  "http://localhost:3000/cf-clearance-scraper",
  {
    url: "https://example.com/my-url", // Masked URL
    mode: "waf-session",
    proxy: {
      host: proxy.ip,
      port: parseInt(proxy.port),
      username: proxy.auth.username,
      password: proxy.auth.password
    }
  },
  {
    headers: {
      "Content-Type": "application/json"
    }
  }
);

console.log(session);

Issue Type

Bug

Operating System

Linux

Do you use Docker?

Docker

a361234599 commented 1 month ago

1.过滤不必要的请求 2.使用更快的代理