search

oal55 / interviewbit-crawler

Python script that logs in to interview bit, fetches the user's solved questions and saves them into their local.
4 stars 5 forks source link

does not process javascript #3

Open vaibhavsood opened 6 years ago

vaibhavsood commented 6 years ago

Hi,

For me these scripts dont correctly fetch the code files, in utilities.py the regex to find 'PROBLEM_SOLVED' does not work. on investigation, a plain session.get(url) just fetches the html without executing the javascript, did you use any libs like dryscape or spynner to preprocess the javascript?

page = session.get(url); page = page.text
    # get links and statuses
    links = rgx_link.findall(page)
    stats = [PROBLEM_SOLVED in x for x in rgx_stat.findall(page)]
oal55 commented 6 years ago

Nah, the ones on this repo are the only scripts I used. Their devs seem to have changed the hyperlinks. They have a different structure now, that's why you couldn't fetch the problems. I'll take a look at it next week.

Thanks for the issue