search

google / site-kit-wp

Site Kit is a one-stop solution for WordPress users to use everything Google has to offer to make them successful on the web.
https://sitekit.withgoogle.com
Apache License 2.0
1.22k stars 278 forks source link

Refine data logged in error stats collection #2360

Open adamsilverstein opened 3 years ago

adamsilverstein commented 3 years ago

Bug Description

In https://github.com/google/site-kit-wp/issues/1999, we added logging for plugin data requests so we could better understand API errors and monitor error rates.

The initial implementation includes the full error message, however this turns out to be overly verbose. The error message includes the full details and may error messages be localized into the user's language. Because of this, a single error code might result in many message variations. These extra variations don't include useful information and wind up making reviewing and reporting on these errors less useful.

Resolution In trackAPIError we should skip sending the error.message, sending only labelMeta: https://github.com/google/site-kit-wp/blob/2c069f2ca8684dbb69e4603941d9beb90ddb2b10/assets/js/util/api.js#L56

Do not alter or remove anything below. The following sections will be managed by moderators only.

Acceptance criteria

Implementation Brief

Test Coverage

Visual Regression Changes

QA Brief

Changelog entry

felixarntz commented 3 years ago

I'm 50/50 on whether we should do this. While the translations make our current tracking somewhat incorrect, I think having the message content can convey helpful information. Also not all content is translated anyway, that mostly (only?) applies to strings that come from our plugin but not APIs.

Maybe we can update our tracking approach to count/group results by only the error code?

adamsilverstein commented 3 years ago

Maybe we can update our tracking approach to count/group results by only the error code?

Good point. I tested and am able to extract the reason and code reliably using a RegEx, so I can report based on those data points. Still, after reviewing the errors so far, I have a feeling that the message data is mostly extra noise, and what we really want to know is the the code and reason.

Let's talk more about the value of the extra message data at our next team sync and decide if we should keep it.