search

pmyteh / RISJbot

A scrapy project to extract the text and metadata of articles from news websites
71 stars 30 forks source link

Spider not found #8

Open smyja opened 3 years ago

smyja commented 3 years ago

I keep getting key error spider not found:CNN when I run scrapy crawl cnn or for any news website. What directory am I supposed to run that in? The README is very vague.

pmyteh commented 3 years ago

If you're getting 'Spider not found' then scrapy itself is running. Have you installed scrapy in a venv, as recommended by their installation docs? Can you run any of the other spiders? The 'base' directory to look for spiders is found in your settings.py file, which should draw from RISJbot.spiders by default. And everything is case sensitive.