search

lugnitdgp / freepl_main

1 stars 10 forks source link

Automate the score calculation system. Find an API to fetch detailed score. #2

Open modihere opened 6 years ago

modihere commented 6 years ago

Presently the score of each player is calculated manually by using pen and paper and then updated manually in the web app. We need to make the system automated. A basic idea about web scraping using selenium and a fairly good command over django web framework is required

Arrowheadahp commented 5 years ago

hello i already know beautysoup so can i use that to find the scores of the players. i can make it as a function in a seperate file

modihere commented 5 years ago

How will you scrape a dynamic site? Any idea?

Arrowheadahp commented 5 years ago

while true: f=urllib2.urlopen('...') .... time.sleep(5)

modihere commented 5 years ago

That's just for a normal html site. How will you manage the clicks and new tabs?

Arrowheadahp commented 5 years ago

i googled what you asked i have to use selenium webdriver for that i can still learn that can i ask why do we need to use new tabs or clicks we store the url of the site to see the score with the player credentials in the csv file

modihere commented 5 years ago

From where will you take the stats of each players? There is no such free api available. You'll have to automate the whole thing of opening a browser and going to the site clicking the relevant links to fetch the score. After that you'll be able to scrape the html for the relevant data to be stored in a csv file.