Adds more detail to our readme, rewords sections and reformats. In particular, it adds notes about how to run individual spiders and test them.
Why are we doing this?
I think this information is helpful for first-time users of this repo or those unfamiliar with Scrapy's API.
My motivation here is driven heavily by my own experience: this repo was my introduction to the city scrapers project and – even as someone with pipenv and Scrapy experience – I was somewhat frustrated there weren't better notes in the readme about how to get started. Since our goal is to make the City Scraper project as approachable as possible to folks who don't have much coding experience, I think it's important we make these instructions simple and clear.
Are there any smells or added technical debt to note?
The readmes for each city scraper repo take divergent approaches. Some, like Akron and Fresno, are barebones and simply link to our full documentation webpage. Others, like Indy and Philly, include quite detailed setup instructions.
I see a strong argument for the Akron/Fresno approach because it means we can have essentially a single source of truth for our documentation. However, to my mind, I quite like the idea of a hybrid approach: we link to our webpage for some details (like install instructions) but include notes on other stuff that isn't likely to change, like triggering a scraper and what the City Scrapers project is. I'm hopeful this makes the repo more accessible to people because they don't have to open up a whole new webpage.
I propose the structure of this readme is also adopted by our repos but I welcome feedback here. Again, there may be a strong argument for simply linking to our city-scrapers.org documentation page.
What's this PR do?
Adds more detail to our readme, rewords sections and reformats. In particular, it adds notes about how to run individual spiders and test them.
Why are we doing this?
I think this information is helpful for first-time users of this repo or those unfamiliar with Scrapy's API.
My motivation here is driven heavily by my own experience: this repo was my introduction to the city scrapers project and – even as someone with pipenv and Scrapy experience – I was somewhat frustrated there weren't better notes in the readme about how to get started. Since our goal is to make the City Scraper project as approachable as possible to folks who don't have much coding experience, I think it's important we make these instructions simple and clear.
Are there any smells or added technical debt to note?
The readmes for each city scraper repo take divergent approaches. Some, like Akron and Fresno, are barebones and simply link to our full documentation webpage. Others, like Indy and Philly, include quite detailed setup instructions.
I see a strong argument for the Akron/Fresno approach because it means we can have essentially a single source of truth for our documentation. However, to my mind, I quite like the idea of a hybrid approach: we link to our webpage for some details (like install instructions) but include notes on other stuff that isn't likely to change, like triggering a scraper and what the City Scrapers project is. I'm hopeful this makes the repo more accessible to people because they don't have to open up a whole new webpage.
I propose the structure of this readme is also adopted by our repos but I welcome feedback here. Again, there may be a strong argument for simply linking to our city-scrapers.org documentation page.