Closed xinatcg closed 1 year ago
jpmckinney
commented
1 year ago Scrapyd essentially calls scrapy crawl on your project.
Generally, you should be able to achieve anything you need by using the spider_opened signal.
If you're doing something very unusual, you'll have to share some example code for me to understand.
xinatcg
commented
1 year ago Scrapyd essentially calls
scrapy crawlon your project.Generally, you should be able to achieve anything you need by using the spider_opened signal.
If you're doing something very unusual, you'll have to share some example code for me to understand.
Thanks very much, we will have a look!
May I ask if is there any way use scrpyd to run Python script with CrawlerProcess
for example, the use case is: we have some custom logic for python script before run spider.