search

rasbt / datacollect

A collection of tools to collect and download various data.
GNU General Public License v3.0
209 stars 97 forks source link

Links are not working #3

Closed gauravlath07 closed 7 years ago

gauravlath07 commented 7 years ago

EXTREMELY uselful code. But I couldnt get it to work because https://www.dreamteamfc.com/c/statistics/players/ALL/ is outdated.

rasbt commented 7 years ago

yeah, I haven't used the script for ~1 1/2 years and they probably changed their website/layout since then. But if you happen to update it, I'd welcome a PR

gauravlath07 commented 7 years ago

You got a sick repository mate.

I have just started data dev. And was wondering if you could suggest me some ways to web scrape javascript enabled pages. I know I should be googling this stuff…but I couldn’t get any of them to work. And the other links suggested javascript solutions which i don’t want to venture into.

Would really appreciate advice from a veteran :)

Cheers !

On Feb 1, 2017, at 7:26 PM, Sebastian Raschka notifications@github.com wrote:

yeah, I haven't used the script for ~1 1/2 years and they probably changed their website/layout since then. But if you happen to update it, I'd welcome a PR

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/rasbt/datacollect/issues/3#issuecomment-276828085, or mute the thread https://github.com/notifications/unsubscribe-auth/AR0jAIOh9VB3qn9UXyQqm5Q3ZHc7He5Uks5rYSK7gaJpZM4L0gfh.

rasbt commented 7 years ago

To be honest, I don't have much experience with scraping data from javascript enabled pages -- I was usually aiming for the HTML/static stuff. (When I worked with JavaScript-enabled pages, they usually came with specific APIs, or the data was available through raw json). I heard dryscape is quite good though, have a look: https://github.com/niklasb/dryscrape

rasbt commented 7 years ago

I hope to find some time one day to update all of this, but for now, let me close this issue since there already seems to be a similar one here: #2