search

code4craft / webmagic

A scalable web crawler framework for Java.
http://webmagic.io/
Apache License 2.0
11.42k stars 4.18k forks source link

downloader 设置 proxy 与 site 设置 proxy 有区别吗？ #1080

Open nesteiner opened 2 years ago

nesteiner commented 2 years ago

我发现可以对Site对象设置代理
image

也可以对downloader设置代理

        HttpClientDownloader downloader = new HttpClientDownloader();
        downloader.setProxyProvider(SimpleProxyProvider.from(
                new Proxy("127.0.0.1", 7890)
        ));

        Spider.create(this)
                .addUrl(starturl)
                .addPipeline(new XiannvPipeline())
                .setDownloader(downloader)
                .runAsync();

我的问题是，这两个方法等效吗，为什么要这样设置?