JaseZiv / worldfootballR

A wrapper for extracting world football (soccer) data from FBref, Transfermark, Understat
https://jaseziv.github.io/worldfootballR/
444 stars 60 forks source link

Issue with `fotmob_get_season_stats( )` #201

Closed mcavs closed 1 year ago

mcavs commented 1 year ago

I received the following error to call the season stats by using fotmob_get_season_stats( ). How can I solve this?

> epl_team_xg_2021 <- fotmob_get_season_stats(
+   country = "ENG",
+   league_name = "Premier League",
+   season_name = "2020/2021",
+   stat_name = "Expected goals",
+   team_or_player = "team"
+ )

Error: lexical error: invalid char in json text.
                                       NA
                     (right here) ------^
JaseZiv commented 1 year ago

Tagging @tonyelhabr here as this does appear to be an issue.

I have recreated it locally using version 0.6.1.4000 and can confirm the error is occurring.

tonyelhabr commented 1 year ago

i'll take a look at it tomorrow

tonyelhabr commented 1 year ago

i'll take a look at it tomorrow

Ah, this is quite involved. Seems that they have changed the way that they store the data.

mcavs commented 1 year ago

@tonyelhabr is this error solved? Because I still received the same error...

tonyelhabr commented 1 year ago

Ah yes it's broken again. fotmob changes their website very frequently. as you can see, there was no failed test for line 188 in test-fotmob.R when the PR merged (that test has essentially the same code as yours)

tonyelhabr commented 1 year ago

@mcavs can you try it again? i just tried the function call and it worked.

for some context: i was debugging the issue last night and saw some big changes that Fotmob implemented on the backend (different format for build id, parameters in the query component of the URL), but I came back to it this morning and no longer saw the same changes

mcavs commented 1 year ago

@tonyelhabr it works now. Thanks for your interest!

cjones46 commented 1 year ago

@tonyelhabr I'm still receiving the lexical error running the exact same command. I've updated the worldfootballR package and restarted R. Any suggestions?

tonyelhabr commented 1 year ago

@cjones46

  1. what version of {jsonlite} do you have? (you can use packageVersion("jsonlite"))
  2. do you see something like this in your inspector mode when you go to Premier League > Stats > Players 2022/2023 > Top Scorer (See All button). specifically, i'm interested in seeing the GET request for premier-league-players.json

image

cjones46 commented 1 year ago

@tonyelhabr Thanks so much for the quick and helpful reply!

Appears I have the latest version of jsonlite; see output below.

> packageVersion("jsonlite")
[1] ‘1.8.0’
> update.packages("jsonlite")
> packageVersion("jsonlite")
[1] ‘1.8.0’
> epl_team_xg_2021 <- fotmob_get_season_stats(country = "ENG", league_name = "Premier League", season_name = "2020/2021", stat_name = "Expected goals",team_or_player = "team")
Error: lexical error: invalid char in json text.
                                       NA
                     (right here) ------^

I believe the below screenshot illustrates that the get request for premier-league-players.json is working.

image

tonyelhabr commented 1 year ago

@cjones46 thanks for the package version printout and the screenshot! can you try to update your jsonlite version to the development version using remotes::install_github("jeroen/jsonlite")? i have version 1.8.2.

i thought that the new internal .fromJSON() function I wrote would work for when the user has a jsonlite version below 1.8.1, but that may not be the case. (See here for the notification about an internal change to jsonlite::fromJSON().)

Depending on whether or not updating your package version resolves the issue, we may need to add a minimum version requirement to the DESCRIPTION file.

cjones46 commented 1 year ago

@tonyelhabr Updated the package, verified that my version is > 1.8.1, but still getting the same error unfortunately.

> packageVersion("jsonlite")
[1] ‘1.8.3’
> epl_team_xg_2021 <- fotmob_get_season_stats(country = "ENG", league_name = "Premier League", season_name = "2020/2021", stat_name = "Expected goals",team_or_player = "team")
Error: lexical error: invalid char in json text.
                                       NA
                     (right here) ------^
tonyelhabr commented 1 year ago

@cjones46 hmm ok i'll look into this more this evening

tonyelhabr commented 1 year ago

@cjones46 I realized that I forgot to ask what version of the {worldfootballR} package you have. Do you have at least 0.6.1.5000?

cjones46 commented 1 year ago

@tonyelhabr

Doesn't appear that I do; I ran update.packages() on it at the beginning of this conversation and thought that meant I had the latest version, but it doesn't seem to have given me a version 0.6.1.5000. How do I obtain that version?

> packageVersion("worldfootballR")
[1] ‘0.6.1’
> update.packages("worldfootballR")
> packageVersion("worldfootballR")
[1] ‘0.6.1’
tonyelhabr commented 1 year ago

Try remotes::install_github("JaseZiv/worldfootballR")

cjones46 commented 1 year ago

@tonyelhabr

That worked! Thank you so so much for your help and for the great work you guys do to keep up this wonderful package!

tonyelhabr commented 1 year ago

@tonyelhabr

That worked! Thank you so so much for your help and for the great work you guys do to keep up this wonderful package!

awesome!