search

pingcap / tidb

TiDB is an open-source, cloud-native, distributed, MySQL-Compatible database for elastic scale and real-time analytics. Try AI-powered Chat2Query free at : https://www.pingcap.com/tidb-serverless/
https://pingcap.com
Apache License 2.0
37.04k stars 5.82k forks source link

Error logging: TiDB should log errors in a table in the database #20614

Open ghimsim opened 3 years ago

ghimsim commented 3 years ago

There is currently no way to look at an error log in TiDB. This is particularly useful when I do a batch operation like loading data into TiDB. During a LOAD DATA operation, there will likely be errors in the data and having this log will enable users to go in and see where the errors are.

ghost commented 3 years ago

A feature like this was just added in MySQL 8.0.22. We should take a look and see if the interfaces make sense for TiDB. There is already pre-existing support for information_schema.slow_query which maps to reading a log file.

Note that for the use case that you mentioned (LOAD DATA failures), I would categorize most of these as a client error. In many cases these shouldn't be written to the server's error log when semantically the server is behaving as expected (and it is the client behaving incorrectly, or violating constraints, resource limits etc). There is also a high risk of logging (and leaking) sensitive data when logging client errors, so I expect this delineation will only get stronger in newer releases.

TiDB currently lacks server-side observability into client errors. There is an existing FR assigned to me in https://github.com/pingcap/tidb/issues/14433 to add this. Please feel free to comment and add requirements.