search

code4craft / webmagic

A scalable web crawler framework for Java.
http://webmagic.io/
Apache License 2.0
11.37k stars 4.18k forks source link

启动时，自定义参数放在Request后面接收不到 #1133

Open yangjinde opened 10 months ago

yangjinde commented 10 months ago

启动时，想将某个参数传到Processor中，但是发现根本获取不到，这个是什么原因？代码如下： `Request request = new Request("https:/www.xxxx.xxxx/xxx"); request.setRequestBody(HttpRequestBody.json("{'id':1}","utf-8"));

    Spider.create(douyinProcessor).thread(1)
            .addPipeline(douyinPipeline)
            .setDownloader(douyinDownloader)
            .addRequest(request)
            .runAsync();`

后面代码在Processor中获取，获取不到前面的id值 Request request = page.getRequest(); String paramJson = new String(request.getRequestBody().getBody());

获取不到前面传的body里边的内容呢，body是空的，用了request.putExtra方式也传不了。

sutra commented 10 months ago

在douyinDownloader里处理错了？

sparrow-ez commented 8 months ago

可以换种方式 Spider.create( new douyinProcessor(id) ).thread(1) ....然后在 douyinProcessor 构造方法中读取id