search

City-Bureau / city-scrapers

Scrape, standardize and share public meetings from local government websites
https://cityscrapers.org
MIT License
332 stars 310 forks source link

WIP : 0728 Northeastern Illinois University #930

Closed bugsz closed 4 years ago

bugsz commented 4 years ago

I have finished parsing the first website, which include basic description of meetings. Now the problem is that I don't know how to parse two sites in different structure. Besides, tests are also unavailable.

I'd like to receive advice from everyone : )

bugsz commented 4 years ago

I'm running into a problem when I try to write test code for it. I cant find the proper function to call in parsed_items = [item for item in spider.parse(test_response)] .Any suggestion on it? @pjsier

pjsier commented 4 years ago

@WhatAreYouSaying since you're first parsing the documents page and then the detail page you'll need to go in that order in the tests. Something like:

for item in spider.parse(test_docs_response):
  pass
parsed_items = [item for item in spider._parse_detail(test_response)]
bugsz commented 4 years ago

Unfortunately, I'm not able to finish it immediately due to the final. I'll continue working on it and finish it within a month.Sorry for that. @pjsier

bugsz commented 4 years ago

@pjsier I'm not sure what's going on in CI test. It says it couldn't install dependencies.

pjsier commented 4 years ago

Closing since this hasn't been active in a while, but feel free to reopen!