scrapinghub / scrapyrt

HTTP API for Scrapy spiders
BSD 3-Clause "New" or "Revised" License
824 stars 161 forks source link

Cancel request and yield item in modify_realtime_request? #144

Closed internalG closed 4 months ago

internalG commented 1 year ago

def modify_realtime_request(self, request): yield {'name':'test'}

{"status": "ok", "items": [], "items_dropped": [], "stats": {"elapsed_time_seconds": 0.004274, "finish_reason": "finished", "finish_time": "2022-10-27 08:50:15", "log_count/ERROR": 2, "log_count/INFO": 9, "memusage/max": 65720320, "memusage/startup": 65716224, "start_time": "2022-10-27 08:50:15"}, "spider_name": "rt"} AttributeError: 'generator' object has no attribute 'meta' AttributeError: 'generator' object has no attribute 'dont_filter'

Hope this is like scrapy for yield everywhere😁

pawelmhm commented 4 months ago

now you cannot yield items from modify_realtime_request, it needs to return request object.