Closed jlkravitz closed 9 years ago
Already ditching all the lectures?
— Jeremy
On Sat, Nov 29, 2014 at 2:13 PM, Joshua Kravitz notifications@github.com wrote:
Now uses selenium, which is needed since the site is made through angular. You can merge this Pull Request by running: git pull https://github.com/jlkravitz/scpd-scraper master Or you can view, comment on it, or merge it online at: https://github.com/jkeesh/scpd-scraper/pull/17 -- Commit Summary --
- Add full access script
- Use clearer url
- Refactor browser out of course download so we don't need 2-step auth for every course
- Lots of cleanup (not in working condition)
- Add phantom and selenium so we can use js
- Add requirements.txt
- Start incorporating selenium
- Better format for specifiying class names
- Clean up scpd access script
- Make try/catch have less code
- Working with selenium
- Remove js script
Update README -- File Changes -- M .gitignore (1) M README.md (71) A package.json (5) A requirements.txt (3) M scrape.py (452) -- Patch Links -- https://github.com/jkeesh/scpd-scraper/pull/17.patch https://github.com/jkeesh/scpd-scraper/pull/17.diff
Reply to this email directly or view it on GitHub: https://github.com/jkeesh/scpd-scraper/pull/17
It happens
cool that this is still being used (or at least forked). ill merge this but dont even have the credentials anymore to test it
yeah...it's kind of a pain to test since you have to log in every time, enter two-step code, etc. But I know the basic usage definitely works.
gotta remove "," in the package.json - correct line is
"chromedriver": "^2.10.0-1"
looks like the server is html5 now. Does anyone have it working? What do you do with the two factor. RIght now it times out for me once I manually type in the 2factor code.
I'm not a student any more so I don't use or maintain it. However I do merge in patches people make since it still seems to be used
— Jeremy
On Sat, Mar 7, 2015 at 7:35 PM, keith siilats notifications@github.com wrote:
gotta remove "," in the package.json - correct line is "chromedriver": "^2.10.0-1"
looks like the server is html5 now. Does anyone have it working? What do you do with the two factor. RIght now it times out for me once I manually type in the 2factor code.
Reply to this email directly or view it on GitHub: https://github.com/jkeesh/scpd-scraper/pull/17#issuecomment-77730933
Now uses selenium, which is needed since the site is made through angular.