parse_fces.py Authentication Issues on Certain Networks

[chrisying]

When running parse_fces.py, it fails at cmu_auth.authenticate(url, username, password) on line 106. It appears that the website that hosts the FCEs requires cookies which the script is not providing.

i.e. when trying to send a post request via cmu_auth to https://student.smartevals.com/reporting/SurveyResults.aspx?q=all&div=683&st=meantext&c=all&lvty=all%2Fall%2Fall&cat=all&u=all&dep=all&xp=t&hg=t&ds=normal&y=all&t=all&b=all&sat=all&srfall=Y&gb=CourseID&cg=all

The response is

[jez]

Hmm, we were having some trouble with this in the days before we released it. The FCE's page is a pain, but maybe we'll have some time to fix it.

Luckily there is a workaround. The FCE's page allows anyone who's logged in to manually download the data for each school. You can log in, download the data, and then run parse_fces.py manually on the files you've downloaded.

[justingallagher]

This is actually an issue I've been trying to debug for the last few days, as it seems to show up sporadically. I'd appreciate any details you can give me:

1. What is the exact error you get when running the script? Trying to use cmu_auth to directly query the smartevals.com site doesn't give useful results because we have to pass some form data to properly authenticate (which is done by the script).
2. What network are you running the script on? I've been able to replicate the problem only on CMU networks, and things run smoothly on my home network. If you have access to a non-CMU network, it might be worth trying to run the script there.

Thanks!

[justingallagher]

This might be fixed as of fdadd2b, testing would be appreciated.