cwendt94 / espn-api

ESPN Fantasy API! (Football, Basketball)
MIT License
659 stars 213 forks source link

Up to date team projections #61

Closed voicelessreason closed 4 years ago

voicelessreason commented 5 years ago

Hi, not sure how else to ask you so this is an issue (can be closed if having open ones bother you).

I'm currently trying to poke around and locate up to date matchup projections. I know that throughout the game the matchup screen in the app updates the projections to reflect how players have performed so far. I wanted to incorporate that information into an app that I'm working on, but found that the projected points in the box_player object will always reflect the value that was projected before the match began. I've been pouring through the various payloads but so far having a really tough time interpreting what the stats correlate to. Any advice you can share in terms of determining the values of certain data points, or have you thought any about where this info might be found? Since its displayed in the matchup view I figured there was a good chance it would be in box_scores, but I just can't figure out what any of the data points are.

cwendt94 commented 5 years ago

Yeah so the best way I have been able to parse ESPN’s payload is by using chromes dev tools on ESPN fantasy website. I watch the network traffic and view the endpoints and data returned for each page and data.

For the problem you are trying to solve I have looked at it a little. It seems matchup and the fantasy cast on the website do not call any ESPN API endpoints. I think they might use web sockets for the live updates. I saw this also when leagues were live drafting the app and website didn’t call any endpoints. So I do not know if it is possible to grab this data.

voicelessreason commented 5 years ago

Oh man, that is really interesting, and that would definitely explain why that data is not available directly in the API response. I wonder if they are worried about making it accessible because it could reveal something about what they're doing algorithmically to arrive at those mid-match projections, or if its just an architectural choice insofar as they need to grab the data at a higher refresh rate than the rest of it.

At any rate, thanks so much for your response. I really appreciate all the work that you've done to get this tool off the ground 👍

cwendt94 commented 5 years ago

Yeah possibly. I know you can use chrome developer tools and specifically look at web sockets traffic but I haven’t had much time to look at it yet. Maybe there is a way!

No problem! I really enjoy being able to apply programming to fantasy football which I am really passionate about!

pmracek commented 5 years ago

I have the same problem. Last year, I could scrape the projections out of the HTML with requests and beautifulsoup. Since the site redesign, the HTML returned to requests isn't the final code displayed to a user in an interactive browser session.

loupalladino commented 5 years ago

The nesting they use now is nothing short of completely non-sensical. The div class values used to actually be intuitive but now they are a lot of the same class names repeated over and over, which makes things so much more difficult to scrape with bs4.

I know that projected totals are in a few places. Have we looked at the main scoreboard page for all of the league, i.e.:

https://fantasy.espn.com/football/league/scoreboard?leagueId=nnnnnn

That has a Projected Total that on the surface appears to be at a minimum scrape-able off the web page:

<div class="statusLabel">Proj Total:<span class="statusValue fw-bold">108</span></div>

This is outside both the main box score for the matchup as well as FantasyCast. I wonder if this yields anything different API-wise?