search

City-Bureau / city-scrapers

Scrape, standardize and share public meetings from local government websites
https://cityscrapers.org
MIT License
334 stars 311 forks source link

WIP: 0848-spider-wayne_state_university #884

Closed moisesjsanchez closed 4 years ago

moisesjsanchez commented 5 years ago

Hello, this is my first time writing a scraper with python using scrapy. I am having some issues getting the code to run as of now, I get the error "response not defined" when running PyTest. I was wondering if I can please get some feedback about what I have written so far? Thank you!

moisesjsanchez commented 5 years ago

Thank you so much for the help so far! However, I just ran into an issue with my git. My current commit is not making to this branch and I have been working on this issue for two days with no prevail. I can try dragging and dropping my current second draft via message on this thread? But if this would be too much I can just leave this project for another user and wait till another issue arises. Thank you once again.

pjsier commented 5 years ago

Hmm git issues can be tricky. Here's one resource that can be helpful for debugging: https://sethrobertson.github.io/GitFixUm/fixup.html

Worst case scenario I would say to manually copy the files you've changed out of the repo, re-clone it and then move the files back inside and then commit again. It's not ideal, but it should be relatively straightforward since you've only added a few files here. Happy to help debug on Slack as well!

moisesjsanchez commented 5 years ago

Hello. I just made my second draft. When I test some of these sections in scrapy shell I got most of the functions to work. However, I have an issue with my _validate_location function keep returning the ValueError even through Room BC is found in the xpath? I also fixed my git issue, I think for the most part. Thank you for the help once again!

pjsier commented 4 years ago

Closing since this hasn't been active in a while, but feel free to reopen!