Is your feature request related to a problem? Please describe
VictoriaMetrics components are designed to return actionable and detailed error messages to the user. The text of the messages contains all the needed information for user to understand what needs to be changed, or fixed, or re-configured.
Due to many details, such messages could also expose sensitive data like IP addresses, pod names, etc. This could be especially critical if VictoriaMetrics is used as a public datasource for many users.
Describe the solution you'd like
While it is possible to control error messages exposed by VM components, we still don't have control over error messages returned by the used libraries, including Go std lib.
Accounting for this, it could make sense to add functionality to lib/httpserver to mask all returned errors with placeholders:
error when searching for metricIDs in the current indexdb: the number of matching timeseries exceeds 1000000; => resource usage limit exceeded: narrow down series selector; uid: <uid>
couldn't start executing the request in 10.000 seconds => backend is temporarily unavailable; uid: <uid>
Such transformed errors should be user-facing - returned via HTTP to the user/caller. The actual error message should be logged as usual.
The <uid> part in masked errors should work as a unique identificator between masked message and actual log message. It means that when masking the error, VM should add a unique ID to both: logged message and returned message. Later, user can report the error message they received, and platform operator will be able to find the corresponded full error message in logs.
Functionality to mask error message should be toggleable.
Describe alternatives you've considered
Use a reverse-proxy that would intercept all responses and perform the similar masking actions.
Is your feature request related to a problem? Please describe
VictoriaMetrics components are designed to return actionable and detailed error messages to the user. The text of the messages contains all the needed information for user to understand what needs to be changed, or fixed, or re-configured.
Due to many details, such messages could also expose sensitive data like IP addresses, pod names, etc. This could be especially critical if VictoriaMetrics is used as a public datasource for many users.
Describe the solution you'd like
While it is possible to control error messages exposed by VM components, we still don't have control over error messages returned by the used libraries, including Go std lib. Accounting for this, it could make sense to add functionality to lib/httpserver to mask all returned errors with placeholders:
error when searching for metricIDs in the current indexdb: the number of matching timeseries exceeds 1000000;=>resource usage limit exceeded: narrow down series selector; uid: <uid>couldn't start executing the request in 10.000 seconds=>backend is temporarily unavailable; uid: <uid>Such transformed errors should be user-facing - returned via HTTP to the user/caller. The actual error message should be logged as usual.
The
<uid>part in masked errors should work as a unique identificator between masked message and actual log message. It means that when masking the error, VM should add a unique ID to both: logged message and returned message. Later, user can report the error message they received, and platform operator will be able to find the corresponded full error message in logs.Functionality to mask error message should be toggleable.
Describe alternatives you've considered
Use a reverse-proxy that would intercept all responses and perform the similar masking actions.
Additional information
Related to https://github.com/VictoriaMetrics/VictoriaMetrics/issues/7588