search

Police-Data-Accessibility-Project / scrapers

Code relating to scraping public police data.
https://pdap.io
GNU General Public License v3.0
157 stars 33 forks source link

Enforce geographic sorting and naming convention #224

Closed EvilDrPurple closed 10 months ago

EvilDrPurple commented 10 months ago

This should finish up the structure changes as part of #197

All scrapers are now sorted on the basis of scrapers_library/COUNTY/MUNICIPALITY/SCRAPER_NAME

I also implemented from_root into the scrapers as I went so that the directory changes should not break any scrapers that were not already broken.

And the scrapers folder has been renamed to scrapers_library

josh-chamberlain commented 10 months ago

this looks solid to me, passing to @mbodeantor !

mbodeantor commented 10 months ago

Thanks for this, looks like a lot of work. Can you move these lines below the final import statement to keep them all together with no blank lines? Sorry, I know that's probably not a small change.

p = from_root('CONTRIBUTING.md').parent
sys.path.insert(1, str(p))
EvilDrPurple commented 10 months ago

Thanks for this, looks like a lot of work. Can you move these lines below the final import statement to keep them all together with no blank lines? Sorry, I know that's probably not a small change.

p = from_root('CONTRIBUTING.md').parent
sys.path.insert(1, str(p))

@mbodeantor This is actually not possible to do, it is required for the final import to work because the root path has to be put in sys.path to recognize where a module is being imported from. They could be moved to before all the import statements instead of in the middle, if you want that done instead.

mbodeantor commented 10 months ago

Ah okay, no makes sense as is then.