Google Analytics 4 API Error with R - Request failed 500 - with a specific query, other queries work as intended

cevaboyz commented 2 years ago

What goes wrong

Getting a 500 error whit a specific query

Steps to reproduce the problem

top_products_sold_last_week <- googleAnalyticsR::ga_data(property_id,
                                                   date_range = c(last_monday, last_sunday),
                                                   metrics = c("itemPurchaseQuantity", "itemRevenue", "averagePurchaseRevenue"), 
                                                   dimensions = c("nthWeek", "itemId"),limit = -1)

Expected output

Actual output

Request failed [500]. Retrying in 1.8 seconds...
Request failed [500]. Retrying in 2.4 seconds...
i 2022-08-08 10:43:33 > Request Status Code:  500
i 2022-08-08 10:43:33 > Trying again:  1  of  1
i 2022-08-08 10:43:36 > All attempts failed.
Error in `abort_http()`:
! http_500 Internal error encountered.

Before you run your code, please run:

options(googleAuthR.verbose=2) and copy-paste the console output here.

i 2022-08-08 14:12:08 > Request:  https://analyticsdata.googleapis.com/v1beta/properties/XXXXXXXXXXX:runReport/
i 2022-08-08 14:12:09 > Body JSON parsed to:  {"metrics":[{"name":"itemPurchaseQuantity"},{"name":"itemRevenue"},{"name":"averagePurchaseRevenue"}],"metricAggregations":["TOTAL","MAXIMUM","MINIMUM"],"dimensions":[{"name":"nthWeek"},{"name":"itemId"}],"dateRanges":[{"startDate":"2022-08-01","endDate":"2022-08-07"}],"keepEmptyRows":true,"limit":100000,"returnPropertyQuota":true}
Auto-refreshing stale OAuth token.
Request failed [500]. Retrying in 1.6 seconds...
Request failed [500]. Retrying in 1.9 seconds...
i 2022-08-08 14:12:22 > Request Status Code:  500
i 2022-08-08 14:12:22 > API error:  Internal error encountered.
i 2022-08-08 14:12:22 > Trying again:  1  of  1
i 2022-08-08 14:12:24 > All attempts failed.
i 2022-08-08 14:12:24 > Custom error 500 Internal error encountered.
Error in `abort_http()`:
! http_500 Internal error encountered.
Run `rlang::last_error()` to see where the error occurred.

Session Info

R version 4.1.0 (2021-05-18)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19044)

Matrix products: default

locale:
[1] LC_COLLATE=English_United States.1252  LC_CTYPE=English_United States.1252   
[3] LC_MONETARY=English_United States.1252 LC_NUMERIC=C                          
[5] LC_TIME=English_United States.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] gt_0.6.0        forcats_0.5.1   stringr_1.4.0   dplyr_1.0.9     purrr_0.3.4    
 [6] readr_2.1.2     tidyr_1.2.0     tibble_3.1.8    ggplot2_3.3.6   tidyverse_1.3.2
[11] scales_1.2.0    lubridate_1.8.0

loaded via a namespace (and not attached):
 [1] assertthat_0.2.1          digest_0.6.29             utf8_1.2.2               
 [4] R6_2.5.1                  cellranger_1.1.0          backports_1.4.1          
 [7] reprex_2.0.1              httr_1.4.3                pillar_1.8.0             
[10] rlang_1.0.4               googlesheets4_1.0.0       curl_4.3.2               
[13] readxl_1.4.0              rstudioapi_0.13           whisker_0.4              
[16] googleAuthR_2.0.0         checkmate_2.1.0           googledrive_2.0.0        
[19] munsell_0.5.0             broom_1.0.0               compiler_4.1.0           
[22] modelr_0.1.8              pkgconfig_2.0.3           askpass_1.1              
[25] htmltools_0.5.3           openssl_2.0.2             tidyselect_1.1.2         
[28] fansi_1.0.3               crayon_1.5.1              tzdb_0.3.0               
[31] dbplyr_2.2.1              withr_2.5.0               rappdirs_0.3.3           
[34] grid_4.1.0                jsonlite_1.8.0            gtable_0.3.0             
[37] lifecycle_1.0.1           DBI_1.1.3                 magrittr_2.0.3           
[40] cli_3.3.0                 stringi_1.7.6             cachem_1.0.6             
[43] farver_2.1.1              fs_1.5.2                  remotes_2.4.2            
[46] xml2_1.3.3                ellipsis_0.3.2            generics_0.1.3           
[49] vctrs_0.4.1               tools_4.1.0               measurementProtocol_0.1.0
[52] glue_1.6.2                hms_1.1.1                 pkgload_1.3.0            
[55] fastmap_1.1.0             colorspace_2.0-3          gargle_1.2.0             
[58] googleAnalyticsR_1.0.1    rvest_1.0.2               memoise_2.0.1            
[61] haven_2.5.0               usethis_2.1.6

MarkEdmondson1234 commented 2 years ago

Thanks, as I mentioned in the SO answer you can always get around errors like this if you have valid json using raw_json

raw <- '{"dimensions":[{"name":"itemId"},{"name":"nthWeek"}],"metrics":[{"name":"averagePurchaseRevenue"},{"name":"itemPurchaseQuantity"},{"name":"itemRevenue"}],"dateRanges":[{"startDate":"2022-08-01","endDate":"2022-08-07"}]}
'

ga_data(propertyId = propety_id, raw_json = raw)
# Making API request with raw JSON:  {"dimensions":[{"name":"itemId"},{"name":"nthWeek"}],"metric ...

# # A tibble: 10,000 × 5
   itemId  nthWeek averagePurchaseRevenue itemPurchaseQuantity itemRevenue
   <chr>   <chr>                    <dbl>                <dbl>       <dbl>
 1 5016395 0000                    15177.                    2      3390. 
 2 5048751 0000                     9378.                   44      8798.

When I compare the working vs bugged call I found it worked if I removed the metricAggregations that by default are added to each GA4 API call when using ga_data():

try <- '{"metrics":[{"name":"itemPurchaseQuantity"},{"name":"itemRevenue"},{"name":"averagePurchaseRevenue"}],"metricAggregations":["TOTAL","MAXIMUM","MINIMUM"],"dimensions":[{"name":"nthWeek"},{"name":"itemId"}],"dateRanges":[{"startDate":"2022-08-01","endDate":"2022-08-07"}],"keepEmptyRows":true,"limit":10}'

# errors
ga_data(propertyId = propety_id, raw_json = try)

Removing metricAggregations fixes it

try <- '{"metrics":[{"name":"itemPurchaseQuantity"},{"name":"itemRevenue"},{"name":"averagePurchaseRevenue"}],"dimensions":[{"name":"nthWeek"},{"name":"itemId"}],"dateRanges":[{"startDate":"2022-08-01","endDate":"2022-08-07"}],"keepEmptyRows":true,"limit":10,"returnPropertyQuota":true}'

# works
ga_data(propertyId = propety_id, raw_json = try)

I'll change it to make metricAggregations opt in rather than on every call, if it causes this behaviour it may be also affecting others.

thanks for the report :)

cevaboyz commented 2 years ago

@MarkEdmondson1234 Thank you, it was an oversight on my part that I did not see that you could use the raw json to make the call. I'm glad my oversight could bring something positive and thank you for your work!

MarkEdmondson1234 commented 2 years ago

I'll keep this open to track the code change

8-bit-sheep / googleAnalyticsR