JaseZiv / worldfootballR

A wrapper for extracting world football (soccer) data from FBref, Transfermark, Understat
https://jaseziv.github.io/worldfootballR/
466 stars 60 forks source link

Plans for fotmob? #61

Closed tonyelhabr closed 2 years ago

tonyelhabr commented 2 years ago

Hi, I'm curious if you're planning on incorporating fotmob. For example

library(jsonlite)
url <- 'https://www.fotmob.com/matchDetails?matchId=3561958'
resp <- jsonlite::fromJSON(url)
names(resp$content$shotmap$shots)
#>  [1] "id"                    "eventType"             "teamId"               
#>  [4] "playerId"              "playerName"            "x"                    
#>  [7] "y"                     "min"                   "minAdded"             
#> [10] "isBlocked"             "isOnTarget"            "blockedX"             
#> [13] "blockedY"              "goalCrossedY"          "goalCrossedZ"         
#> [16] "expectedGoals"         "expectedGoalsOnTarget" "shotType"             
#> [19] "situation"             "period"                "isOwnGoal"            
#> [22] "onGoalShot"            "firstName"             "lastName"

In addition to what understat offers, this has xGoT (expectedGoalsOnTarget) and some of that features that go into it (e.g. goalCrossedZ). Also, fotmob has more leagues than understat.

If you don't have plans, I may just write a package for it, and we can maybe sync up down the road

JaseZiv commented 2 years ago

Thanks so much for trenching out @tonyelhabr.

I have emailed them for permission to use their data so am waiting for a response.

mid they’re ok with it, I’ll start writing some functions to include it in this package. Are you happy for me to reach out for help when needed? Or you could contribute to this library if you like…

tonyelhabr commented 2 years ago

I'd be happy to either help out if you have questions or do it all myself if you'd like.

FWIW I was looking at what's done in https://github.com/bgrnwd/fotmob. I think there should be functions for matches, match data, leagues, teams, and player data. I think these can be developed independently, so that might be an opportunity to split up work if you'd like me to help.

JaseZiv commented 2 years ago

This is brilliant!

Ok, once I hear from fotmob, I'll reach out to you.

tonyelhabr commented 2 years ago

fwiw it seems that they do allow scraping on most parts of the site

library(polite)
base_url <- 'https://www.fotmob.com/'
robotstxt::get_robotstxt(base_url)
#> [robots.txt]
#> --------------------------------------
#> 
#> User-agent: *
#> Disallow: /Accounts/
#> Disallow: /Predictions/
#> Disallow: /Users/
#> 
#> Sitemap: https://data.fotmob.com/sitemap/sitemap-recent-matches-and-news.xml
JaseZiv commented 2 years ago

Integrated in v0.4.0 8df0b2c406a7162bb65b7589d253b78ada1b432f