dekalb county boc spider

[rhetr]

Summary

Issue: #17

Checklist

• [x] Tests are implemented
• [x] All tests are passing
• [x] Style checks run (see documentation for more details)
• [x] Style checks are passing
• [x] Code comments from template removed

[rhetr]

spider.parse yields requests so extra steps need to be taken to yield meetings. first create the additional requests folder:

mkdir tests/files/dekalb_county_boc_requests

then collect a list of the request.urls:

for request in meeting_requests:
    print(request.url)

copy and save this into a file requests.txt, then run the following in bash:

cat requests.txt | while read a; do scrapy fetch --nolog "$a" > tests/files/dekalb_county_boc_requests/$(basename $a).html; done

once the file_responses are created, the relevant Request can be associated by response.request = request (see code for details). gonna try the same technique for atl_boe soon