search

8-bit-sheep / googleAnalyticsR

Use the Google Analytics API from R
https://8-bit-sheep.com/googleAnalyticsR/
Other
257 stars 76 forks source link

ga_account_list failing to parse response for UA data #413

Closed Rifmaz closed 2 weeks ago

Rifmaz commented 1 year ago

What goes wrong

Hello,

I am currently unable to get an output from the ga_account_list() function as the function fails. The error message is copied below.

Steps to reproduce the problem

I'd like to mention that I've updated my R to version 4.2.1 and all my packages have also been updated to the most recent version (as of the 31st of October 2022):

Previous versions installed that were still working:

I have installed the previous version of the above packages but I'm still getting the error. I suspect there may be some other dependencies and that is why?

Regarding the gar_parse_error.rds file, I can't share it since it contains client information.

Expected output

Actual output

Before you run your code, please run:

options(googleAuthR.verbose=2) and copy-paste the console output here. 

my_accounts <- ga_account_list()
2022-11-02 09:25:05 > Request:  https://www.googleapis.com/analytics/v3/management/accountSummaries/
Error : Can't combine `..1` <data.frame> and `..1457` <list>.
Error: API Data failed to parse.  
             Wrote diagnostic object to 'gar_parse_error.rds', use googleAuthR::gar_debug_parsing('gar_parse_error.rds') to 
             debug the data_parse_function.

'API Data failed to parse' diagnostics

Regarding the gar_parse_error.rds file, I can't share it since it contains client information.

Session Info

> sessionInfo()
R version 4.2.1 (2022-06-23 ucrt)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19044)

Matrix products: default

locale:
[1] LC_COLLATE=English_United Kingdom.utf8  LC_CTYPE=English_United Kingdom.utf8   
[3] LC_MONETARY=English_United Kingdom.utf8 LC_NUMERIC=C                           
[5] LC_TIME=English_United Kingdom.utf8    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] forcats_0.5.2          stringr_1.4.1          dplyr_1.0.10           purrr_0.3.5            readr_2.1.3           
 [6] tidyr_1.2.1            tibble_3.1.8           ggplot2_3.3.6          tidyverse_1.3.2        googleAuthR_2.0.0.9000
[11] googleAnalyticsR_1.1.0

loaded via a namespace (and not attached):
 [1] lubridate_1.8.0           assertthat_0.2.1          digest_0.6.30             utf8_1.2.2               
 [5] R6_2.5.1                  cellranger_1.1.0          backports_1.4.1           reprex_2.0.2             
 [9] evaluate_0.17             httr_1.4.4                pillar_1.8.1              rlang_1.0.6              
[13] googlesheets4_1.0.1       curl_4.3.3                readxl_1.4.1              rstudioapi_0.14          
[17] whisker_0.4               rmarkdown_2.17            googledrive_2.0.0         munsell_0.5.0            
[21] broom_1.0.1               compiler_4.2.1            modelr_0.1.9              xfun_0.34                
[25] askpass_1.1               pkgconfig_2.0.3           htmltools_0.5.3           openssl_2.0.4            
[29] tidyselect_1.2.0          fansi_1.0.3               crayon_1.5.2              tzdb_0.3.0               
[33] dbplyr_2.2.1              withr_2.5.0               rappdirs_0.3.3            grid_4.2.1               
[37] jsonlite_1.8.3            gtable_0.3.1              lifecycle_1.0.3           DBI_1.1.3                
[41] magrittr_2.0.3            scales_1.2.1              cli_3.4.1                 stringi_1.7.8            
[45] cachem_1.0.6              fs_1.5.2                  xml2_1.3.3                ellipsis_0.3.2           
[49] generics_0.1.3            vctrs_0.4.2               tools_4.2.1               measurementProtocol_0.1.0
[53] glue_1.6.2                hms_1.1.2                 fastmap_1.1.0             yaml_2.3.6               
[57] colorspace_2.0-3          gargle_1.2.1              rvest_1.0.3               memoise_2.0.1            
[61] knitr_1.40                haven_2.5.1               usethis_2.1.6
MarkEdmondson1234 commented 1 year ago

Odd as this is a very old function without issues before. Are you using your own client id?

The important bit in the parse error object is the structure (str()) so if you can show that there is a much better chance to fix it.

I can only think the api response has changed for some reason.

Rifmaz commented 1 year ago

I'm not using my own client id. The authentication is the default one.

I've pasted below the redacted and truncated gargle file (left a few anonymised values in, the rest I've deleted):

List of 3
 $ request       :List of 4
  ..$ req_url     : chr "https://www.googleapis.com/analytics/v3/management/accountSummaries/"
  ..$ request_type: chr "GET"
  ..$ the_body    : NULL
  ..$ customConfig: NULL
 $ response      :List of 3
  ..$ data_parse_args: list()
  ..$ data_parse_func:function (x)  
  ..$ content        :List of 6
  .. ..$ kind        : chr "analytics#accountSummaries"
  .. ..$ username    : chr "xxx@xxx.com"
  .. ..$ totalResults: int 766
  .. ..$ startIndex  : int 1
  .. ..$ itemsPerPage: int 1000
  .. ..$ items       :'data.frame': 766 obs. of  4 variables:
  .. .. ..$ id           : chr [1:766]
  .. .. ..$ kind         : chr [1:766]
  .. .. ..$ name         : chr [1:766]
  .. .. ..$ webProperties:List of 766
  .. .. .. ..$ :'data.frame':   7 obs. of  7 variables:
  .. .. .. .. ..$ kind                 : chr [1:7] 
  .. .. .. .. ..$ id                   : chr [1:7] 
  .. .. .. .. ..$ name                 : chr [1:7] 
  .. .. .. .. ..$ internalWebPropertyId: chr [1:7] 
  .. .. .. .. ..$ level                : chr [1:7] 
  .. .. .. .. ..$ websiteUrl           : chr [1:7] 
  .. .. .. .. ..$ profiles             :List of 7
  .. .. .. .. .. ..$ :'data.frame': 1 obs. of  4 variables:
  .. .. .. .. .. .. ..$ kind: chr "analytics#profileSummary"
  .. .. .. .. .. .. ..$ id  : chr "xxxxx"
  .. .. .. .. .. .. ..$ name: chr "Une crêpe plus que parfaite"
  .. .. .. .. .. .. ..$ type: chr "WEB"
  .. .. .. .. .. ..$ :'data.frame': 3 obs. of  4 variables:
  .. .. .. .. .. .. ..$ kind: chr [1:3] 
  .. .. .. .. .. .. ..$ id  : chr [1:3] 
  .. .. .. .. .. .. ..$ name: chr [1:3] 
  .. .. .. .. .. .. ..$ type: chr [1:3] 
  .. .. .. .. .. ..$ :'data.frame': 1 obs. of  4 variables:
  .. .. .. .. .. .. ..$ kind: chr 
  .. .. .. .. .. .. ..$ id  : chr 
  .. .. .. .. .. .. ..$ name: chr 
  .. .. .. .. .. .. ..$ type: chr 
  .. .. .. .. .. ..$ :'data.frame': 2 obs. of  4 variables:
  .. .. .. .. .. .. ..$ kind: chr [1:2] 
  .. .. .. .. .. .. ..$ id  : chr [1:2] 
  .. .. .. .. .. .. ..$ name: chr [1:2] 
  .. .. .. .. .. .. ..$ type: chr [1:2] 
  .. .. .. .. .. ..$ :'data.frame': 2 obs. of  4 variables:
  .. .. .. .. .. .. ..$ kind: chr [1:2] 
  .. .. .. .. .. .. ..$ id  : chr [1:2] 
  .. .. .. .. .. .. ..$ name: chr [1:2] 
  .. .. .. .. .. .. ..$ type: chr [1:2] 
  .. .. .. .. .. ..$ :'data.frame': 2 obs. of  4 variables:
  .. .. .. .. .. .. ..$ kind: chr [1:2] 
  .. .. .. .. .. .. ..$ id  : chr [1:2] 
  .. .. .. .. .. .. ..$ name: chr [1:2] 
  .. .. .. .. .. .. ..$ type: chr [1:2] 
  .. .. .. .. .. ..$ :'data.frame': 13 obs. of  4 variables:
  .. .. .. .. .. .. ..$ kind: chr [1:13] 
  .. .. .. .. .. .. ..$ id  : chr [1:13] 
  .. .. .. .. .. .. ..$ name: chr [1:13] 
  .. .. .. .. .. .. ..$ type: chr [1:13] 
  .. .. .. ..$ :'data.frame':   0 obs. of  0 variables
  .. .. .. ..$ :'data.frame':   0 obs. of  0 variables
  .. .. .. ..$ :'data.frame':   1 obs. of  7 variables:
  .. .. .. .. ..$ kind                 : chr "analytics#webPropertySummary"
  .. .. .. .. ..$ id                   : chr "UA-xxxxxx-x"
  .. .. .. .. ..$ name                 : chr "xxxxxx.fr"
  .. .. .. .. ..$ internalWebPropertyId: chr "xxxx"
  .. .. .. .. ..$ level                : chr "PREMIUM"
  .. .. .. .. ..$ websiteUrl           : chr "http://www.xxxxxxx.fr"
  .. .. .. .. ..$ profiles             :List of 1
  .. .. .. .. .. ..$ :'data.frame': 1 obs. of  4 variables:
  .. .. .. .. .. .. ..$ kind: chr "analytics#profileSummary"
  .. .. .. .. .. .. ..$ id  : chr "xxxxxx"
  .. .. .. .. .. .. ..$ name: chr 
  .. .. .. .. .. .. ..$ type: chr 
  .. .. .. ..$ :'data.frame':   0 obs. of  0 variables
  .. .. .. ..$ :'data.frame':   3 obs. of  7 variables:
  .. .. .. .. ..$ kind                 : chr [1:3] 
  .. .. .. .. ..$ id                   : chr [1:3] 
  .. .. .. .. ..$ name                 : chr [1:3] 
  .. .. .. .. ..$ internalWebPropertyId: chr [1:3] 
  .. .. .. .. ..$ level                : chr [1:3] 
  .. .. .. .. ..$ websiteUrl           : chr [1:3] 
  .. .. .. .. ..$ profiles             :List of 3
  .. .. .. .. .. ..$ :'data.frame': 1 obs. of  4 variables:
  .. .. .. .. .. .. ..$ kind: chr "analytics#profileSummary"
  .. .. .. .. .. .. ..$ id  : chr "xxxxxx"
  .. .. .. .. .. .. ..$ name: chr 
  .. .. .. .. .. .. ..$ type: chr 
  .. .. .. .. .. ..$ :'data.frame': 0 obs. of  0 variables
  .. .. .. .. .. ..$ :'data.frame': 1 obs. of  4 variables:
  .. .. .. .. .. .. ..$ kind: chr "analytics#profileSummary"
  .. .. .. .. .. .. ..$ id  : chr "xxxxxx"
  .. .. .. .. .. .. ..$ name: chr 
  .. .. .. .. .. .. ..$ type: chr 
  .. .. .. ..$ :'data.frame':   4 obs. of  7 variables:
  .. .. .. .. ..$ kind                 : chr [1:4] 
  .. .. .. .. ..$ id                   : chr [1:4] 
  .. .. .. .. ..$ name                 : chr [1:4] 
  .. .. .. .. ..$ internalWebPropertyId: chr [1:4] 
  .. .. .. .. ..$ level                : chr [1:4] 
  .. .. .. .. ..$ websiteUrl           : chr [1:4] 
  .. .. .. .. ..$ profiles             :List of 4
  .. .. .. .. .. ..$ :'data.frame': 28 obs. of  4 variables:
  .. .. .. .. .. .. ..$ kind: chr [1:28] 
  .. .. .. .. .. .. ..$ id  : chr [1:28] 
  .. .. .. .. .. .. ..$ name: chr [1:28] 
  .. .. .. .. .. .. ..$ type: chr [1:28] 
  .. .. .. .. .. ..$ :'data.frame': 10 obs. of  4 variables:
  .. .. .. .. .. .. ..$ kind: chr [1:10] 
  .. .. .. .. .. .. ..$ id  : chr [1:10] 
  .. .. .. .. .. .. ..$ name: chr [1:10] 
  .. .. .. .. .. .. ..$ type: chr [1:10] 
  .. .. .. .. .. ..$ :'data.frame': 10 obs. of  4 variables:
  .. .. .. .. .. .. ..$ kind: chr [1:10] 
  .. .. .. .. .. .. ..$ id  : chr [1:10] 
  .. .. .. .. .. .. ..$ name: chr [1:10] 
  .. .. .. .. .. .. ..$ type: chr [1:10] 
  .. .. .. .. .. ..$ :'data.frame': 3 obs. of  4 variables:
  .. .. .. .. .. .. ..$ kind: chr [1:3] 
  .. .. .. .. .. .. ..$ id  : chr [1:3] 
  .. .. .. .. .. .. ..$ name: chr [1:3] 
  .. .. .. .. .. .. ..$ type: chr [1:3] 
  .. .. .. ..$ :'data.frame':   1 obs. of  7 variables:
  .. .. .. .. ..$ kind                 : chr "analytics#webPropertySummary"
  .. .. .. .. ..$ id                   : chr "UA-xxxxxxxx-1"
  .. .. .. .. ..$ name                 : chr "xxxxxx.fr"
  .. .. .. .. ..$ internalWebPropertyId: chr "xxxxxxxx"
  .. .. .. .. ..$ level                : chr "STANDARD"
  .. .. .. .. ..$ websiteUrl           : chr "http://xxxxxxxxx.fr"
  .. .. .. .. ..$ profiles             :List of 1
  .. .. .. .. .. ..$ :'data.frame': 1 obs. of  4 variables:
  .. .. .. .. .. .. ..$ kind: chr "analytics#profileSummary"
  .. .. .. .. .. .. ..$ id  : chr "xxxxxxxx"
  .. .. .. .. .. .. ..$ name: chr 
  .. .. .. .. .. .. ..$ type: chr 
  .. .. .. ..$ :'data.frame':   9 obs. of  7 variables:
  .. .. .. .. ..$ kind                 : chr [1:9] 
  .. .. .. .. ..$ id                   : chr [1:9] 
  .. .. .. .. ..$ name                 : chr [1:9] 
  .. .. .. .. ..$ internalWebPropertyId: chr [1:9] 
  .. .. .. .. ..$ level                : chr [1:9] 
  .. .. .. .. ..$ websiteUrl           : chr [1:9] 
  .. .. .. .. ..$ profiles             :List of 9
  .. .. .. .. .. ..$ :'data.frame': 22 obs. of  4 variables:
  .. .. .. .. .. .. ..$ kind: chr [1:22] 
  .. .. .. .. .. .. ..$ id  : chr [1:22] 
  .. .. .. .. .. .. ..$ name: chr [1:22] 
  .. .. .. .. .. .. ..$ type: chr [1:22] 
  .. .. .. .. .. ..$ :'data.frame': 1 obs. of  4 variables:
  .. .. .. .. .. .. ..$ kind: chr 
  .. .. .. .. .. .. ..$ id  : chr 
  .. .. .. .. .. .. ..$ name: chr 
  .. .. .. .. .. .. ..$ type: chr 
  .. .. .. .. .. ..$ :'data.frame': 1 obs. of  4 variables:
  .. .. .. .. .. .. ..$ kind: chr 
  .. .. .. .. .. .. ..$ id  : chr 
  .. .. .. .. .. .. ..$ name: chr 
  .. .. .. .. .. .. ..$ type: chr 
  .. .. .. .. .. ..$ :'data.frame': 1 obs. of  4 variables:
  .. .. .. .. .. .. ..$ kind: chr 
  .. .. .. .. .. .. ..$ id  : chr 
  .. .. .. .. .. .. ..$ name: chr 
  .. .. .. .. .. .. ..$ type: chr 
  .. .. .. .. .. ..$ :'data.frame': 1 obs. of  4 variables:
  .. .. .. .. .. .. ..$ kind: chr 
  .. .. .. .. .. .. ..$ id  : chr 
  .. .. .. .. .. .. ..$ name: chr 
  .. .. .. .. .. .. ..$ type: chr 
  .. .. .. .. .. ..$ :'data.frame': 1 obs. of  4 variables:
  .. .. .. .. .. .. ..$ kind: chr 
  .. .. .. .. .. .. ..$ id  : chr 
  .. .. .. .. .. .. ..$ name: chr 
  .. .. .. .. .. .. ..$ type: chr 
  .. .. .. .. .. ..$ :'data.frame': 1 obs. of  4 variables:
  .. .. .. .. .. .. ..$ kind: chr 
  .. .. .. .. .. .. ..$ id  : chr 
  .. .. .. .. .. .. ..$ name: chr 
  .. .. .. .. .. .. ..$ type: chr 
  .. .. .. .. .. ..$ :'data.frame': 1 obs. of  4 variables:
  .. .. .. .. .. .. ..$ kind: chr 
  .. .. .. .. .. .. ..$ id  : chr 
  .. .. .. .. .. .. ..$ name: chr 
  .. .. .. .. .. .. ..$ type: chr 
  .. .. .. .. .. ..$ :'data.frame': 1 obs. of  4 variables:
  .. .. .. .. .. .. ..$ kind: chr 
  .. .. .. .. .. .. ..$ id  : chr 
  .. .. .. .. .. .. ..$ name: chr 
  .. .. .. .. .. .. ..$ type: chr 
  .. .. .. .. [list output truncated]
 $ authentication:List of 1
  ..$ token:Classes 'Gargle2.0', 'Token2.0', 'Token', 'R6'  ── <Token (via gargle)> ───────────────────────────────────────────────────────────────────────────────────────────────── oauth_endpoint: google            app: googleAuthR.client_id=xxxxxxxxxx-          email: 'xxxxxxx@xxxxxxx.com'         scopes: ...analytics, ...analytics.edit, ...analytics.manage.users, ...analytics.readonly, ...analytics.user.deletion, ...userinfo.email    credentials: access_token, expires_in, refresh_token, scope, token_type, id_token 
 - attr(*, "class")= chr "gar_parse_error"
Rifmaz commented 1 year ago

Hi Mark,

I've done a full reinstall of R and RStudio (including deleting R library / RStudio folders after uninstall to wipe any files that may cause conflicts). Sadly the error remains (but slightly more informative):

Error in list_unchop(x = x, indices = indices, ptype = ptype, name_spec = name_spec,  : 
  Can't combine `x[[1]]` <data.frame> and `x[[1459]]` <list>.
Error: API Data failed to parse.  
             Wrote diagnostic object to 'gar_parse_error.rds', use googleAuthR::gar_debug_parsing('gar_parse_error.rds') to 
             debug the data_parse_function.
MarkEdmondson1234 commented 1 year ago

It looks to me you may have some unusual GA structure, perhaps a web property with no views? The key area is in the content$items$webproperties object so some str(content$items$webproperties, 2)on that may reveal an entry which does not have the same number of columns as the normal ones. Without access to the error object through, I can't do much more as I need to be able to replicate it to fix.

Rifmaz commented 1 year ago

Hi Mark! I think I've cracked why this is happening: some of the account IDs are returning NULL when I run the ga_webproperty_list() function for them. I suspect the ga_account_list() doesn't handle NULL values well but I appreciate I may be quite wrong.

I have also observed that certain accounts are missing 1 column (i.e. data.frames returned have 13 vs 14 columns). This appears to be the industryVertical column. Potentially this is a related issue?

I've attached a screenshot of the list returned when I run the following code: my_properties_list <- apply(my_accounts,1, function(x) { tryCatch(ga_webproperty_list(x['id']), error = function(e){} ) } image

MarkEdmondson1234 commented 1 year ago

Funny this only comes up now after many years, I guess you may have some unique GA account structure? If you run the command in the error message it will output the request and response objects that can be used to update the parsing script. You can edit out sensitive values, it's the structure that's important not the values.

Rifmaz commented 1 year ago

Hi Mark! I've attached the redacted gar_parse_error as an RData file. Thanks for looking into this!

And yes, you are right, I seem to have some weird GA account permissions (account access but without any child property access) gar_parse_error.zip

gronerik commented 2 weeks ago

UA is now gone (sniff), so closing this.