search

dangoldin / yahoo-ffl

Yahoo Fantasy Football Stats
MIT License
29 stars 10 forks source link

Blank csv #1

Closed problemss closed 10 years ago

problemss commented 10 years ago

I can't tell you how much I needed this. I was scrapping from Accuscore and just realized they are using 2012 data and my draft is tomorrow....arg.

Anyway this looks to be just what I need. The only issue I have is the process looks like it completed, but the csv file is blank. Any idea on this?

problemss commented 10 years ago

Would the XPath map cause this or would it generate other errors? If it will and I need to update it, do you have any pointers when finding the class? I inspected the webpage and the class for each number looks the same.

dangoldin commented 10 years ago

You may need to be change the URLs so it pulls the data for your league id and users your username/password. I'd copy the URLs it uses into your browser to make sure everything loads properly.

I also noticed that they changed their design so that may affect the HTML structure which would require updating the XPath map as you said.

problemss commented 10 years ago

Thank you. I will give this a try and let you know what I had to do!

problemss commented 10 years ago

Looks like i got it to work. These are my edits. The only issue is the position is formatted as 'team - position' and the opponent has a space in front. I tried the substring-after commands, but you have to call it from the beginning in XPath 1.0. I might replace the stat rows portion so I can do it or just format it in excel after importing....

stat_rows = hxs.select('//section[@id="players-table"]/div/table/tbody/tr') xpath_map = { 'name': 'td[contains(@class,"player")]/div/div/div[contains(@class,"ysf-player-name")]/a/text()', 'position': 'td[contains(@class,"player")]/div/div/div[contains(@class,"ysf-player-name")]/span/text()', 'opp': 'td[contains(@class,"player")]/div/div/div[contains(@class,"ysf-player-detail")]/span/span/text()', 'passing_yds': 'td[9]/text()', 'passing_tds': 'td[10]/text()', 'passing_int': 'td[11]/text()', 'rushing_yds': 'td[12]/text()', 'rushing_tds': 'td[13]/text()', 'receiving_yds': 'td[14]/text()', 'receiving_tds': 'td[15]/text()', 'return_tds': 'td[16]/text()', 'misc_twopt': 'td[17]/text()', 'fumbles': 'td[18]/text()', 'points': 'td[5]/text()', }

dangoldin commented 10 years ago

Yea - I would just do it in Excel or if you really want to do it in Python you can do a quick hack inside that for loop. Something like "if col_name == 'position': do substring"

It also looks as if the columns are different depending on the league settings so different people will need to have a different xpath map.