ResourceExhausted errors from the tablet and gate should be parsed into more specific errors

[zmagg]

Currently, ResourceExhausted is used as a catch all for many more specific error types.

e.g. tablet error

rpc error: code = ResourceExhausted desc = User 'user_name' has exceeded the 'max_user_connections' resource (current value: xyz) (errno 1226) (sqlstate 42000) (CallerID: user_name): Sql: "begin", BindVars: {}

We should define more specific error codes for specific errors that can be used in both metrics and logs.

We can split ResourceExhausted into:

• max row limit exceeded
• transaction pool connection limit exceeded
• ...

[dweitzman]

grpc packet size exceeded is another

[dweitzman]

The error codes should probably also be different at a grpc / mysql level:

• connection limit seems like a retryable error
• grpc packet size is probably not retryable

[RicardoLorenzo]

I' was looking into this and I'm a bit confused. This error is documented by MySQL and it specifically states which resource limit is reached and its current value.

Error number: 1226; Symbol: ER_USER_LIMIT_REACHED; SQLSTATE: 42000

User '%s' has exceeded the '%s' resource (current value: %ld)

I believe it could be confusing trying to replace MySQL error messages with others.

Hypothetically, an additional error code constants can be defined which can be used by parsing the MySQL error message here looking for the resource which limit was exceeded and printing/incrementing a different error code.

Now my concern is if this is going too far. IMHO presenting the mySQL error code as it is received is good as an admin can easily refer the documentation for help.

Update: Ahh, I can see the issue is how the MySQL error codes are grouped and the limit reached error code is only one of them. I wonder if the additional error codes should be defined to match MySQL ones 1 by 1.