lihongjie0209
commented
3 years ago MySQL 5.7 Performance Tuning
The good news is that MySQL 5.7 has significantly better default values. Morgan Tocker created a page with a complete list of features in MySQL 5.7, and is a great reference point. For example, the following variables are set by default:
- innodb_file_per_table=ON
- innodb_stats_on_metadata = OFF
- innodb_buffer_pool_instances = 8 (or 1 if innodb_buffer_pool_size < 1GB)
- query_cache_type = 0; query_cache_size = 0; (disabling mutex)
In MySQL 5.7, there are only four really important variables that need to be changed. However, there are other InnoDB and global MySQL variables that might need to be tuned for a specific workload and hardware.
To start, add the following settings to my.cnf under the [mysqld] section. You will need to restart MySQL:
[mysqld]
# other variables here
innodb_buffer_pool_size = 1G # (adjust value here, 50%-70% of total RAM)
innodb_log_file_size = 256M
innodb_flush_log_at_trx_commit = 1 # may change to 2 or 0
innodb_flush_method = O_DIRECT| Variable | Value |
|---|---|
| innodb_buffer_pool_size | Start with 50% 70% of total RAM. Does not need to be larger than the database size |
| innodb_flush_log_at_trx_commit | 1 (Default)0/2 (more performance, less reliability) |
| innodb_log_file_size | 128M – 2G (does not need to be larger than buffer pool) |
| innodb_flush_method | O_DIRECT (avoid double buffering) |
In this blog, we’re going to discuss the top ten MySQL performance tuning settings that you can implement after an installation.
When we are hired for a MySQL performance audit, we are expected to review the MySQL configuration and to suggest improvements. Many people are surprised because in most cases, we only suggest changing a few MySQL performance tuning settings after installation – even though hundreds of options are available. The goal of this post is to give you a list of some of the most critical settings.
We already made such suggestions in the past here on this blog a few years ago, but things have changed a lot in the MySQL world since then!
Before we start MySQL performance tuning:
Even experienced people can make mistakes that can cause a lot of trouble. So before blindly applying the recommendations of this post, please keep in mind the following items:
SET GLOBAL. It is very handy and it allows you to quickly revert the change if it creates any problem. But in the end, you want the setting to be adjusted permanently in the configuration file.[mysqld]section)innodb_buffer_pool_sizeshould be set in bytes whilemax_connectionis dimensionless.Basic settings
Here are 3 MySQL performance tuning settings that you should always look at. If you do not, you are very likely to run into problems very quickly.
innodb_buffer_pool_size: this is the #1 setting to look at for any installation using InnoDB. The buffer pool is where data and indexes are cached: having it as large as possible will ensure you use memory and not disks for most read operations. Typical values are 5-6GB (8GB RAM), 20-25GB (32GB RAM), 100-120GB (128GB RAM).innodb_log_file_size: this is the size of the redo logs. The redo logs are used to make sure writes are fast and durable and also during crash recovery. Up to MySQL 5.1, it was hard to adjust, as you wanted both large redo logs for good performance and small redo logs for fast crash recovery. Fortunately, crash recovery performance has improved a lot since MySQL 5.5 so you can now have good write performance and fast crash recovery. Until MySQL 5.5 the total redo log size was limited to 4GB (the default is to have 2 log files). This has been lifted in MySQL 5.6.Starting with
innodb_log_file_size = 512M(giving 1GB of redo logs) should give you plenty of room for writes. If you know your application is write-intensive and you are using MySQL 5.6, you can start withinnodb_log_file_size = 4G.max_connections: if you are often facing the ‘Too many connections’ error,max_connectionsis too low. It is very frequent that because the application does not close connections to the database correctly, you need much more than the default 151 connections. The main drawback of high values for max_connections (like 1000 or more) is that the server will become unresponsive if for any reason it has to run 1000 or more active transactions. Using a connection pool at the application level or a thread pool at the MySQL level can help here.InnoDB settings
InnoDB has been the default storage engine since MySQL 5.5 and it is much more frequently used than any other storage engine. That’s why it should be configured carefully.
innodb_file_per_table: this setting will tell InnoDB if it should store data and indexes in the shared tablespace (innodb_file_per_table = OFF) or in a separate .ibd file for each table (innodb_file_per_table= ON). Having a file per table allows you to reclaim space when dropping, truncating or rebuilding a table. It is also needed for some advanced features such as compression. However it does not provide any performance benefit. The main scenario when you do NOT want file per table is when you have a very high number of tables (say 10k+).With MySQL 5.6, the default value is ON so you have nothing to do in most cases. For previous versions, you should set it to ON prior to loading data as it has an effect on newly created tables only.
innodb_flush_log_at_trx_commit: the default setting of 1 means that InnoDB is fully ACID compliant. It is the best value when your primary concern is data safety, for instance on a master. However, it can have a significant overhead on systems with slow disks because of the extra fsyncs that are needed to flush each change to the redo logs. Setting it to 2 is a bit less reliable because committed transactions will be flushed to the redo logs only once a second, but that can be acceptable on some situations for a master and that is definitely a good value for a replica. 0 is even faster but you are more likely to lose some data in case of a crash: it is only a good value for a replica.innodb_flush_method: this setting controls how data and logs are flushed to disk. Popular values are O_DIRECT when you have a hardware RAID controller with a battery-protected write-back cache and fdatasync (default value) for most other scenarios. sysbench is a good tool to help you choose between the 2 values.
innodb_log_buffer_size: this is the size of the buffer for transactions that have not been committed yet. The default value (1MB) is usually fine but as soon as you have transactions with large blob/text fields, the buffer can fill up very quickly and trigger extra I/O load. Look at the Innodb_log_waits status variable and if it is not 0, increase
innodb_log_buffer_size.Other settings
query_cache_size: the query cache is a well-known bottleneck that can be seen even when concurrency is moderate. The best option is to disable it from day 1 by settingquery_cache_size = 0(now the default on MySQL 5.6) and to use other ways to speed up read queries: good indexing, adding replicas to spread the read load or using an external cache (memcache or redis for instance). If you have already built your MySQL application with the query cache enabled and if you have never noticed any problem, the query cache may be beneficial for you. So you should be cautious if you decide to disable it.log_bin: enabling binary logging is mandatory if you want the server to act as a replication master. If so, don’t forget to also setserver_idto a unique value. It is also useful for a single server when you want to be able to do point-in-time recovery: restore your latest backup and apply the binary logs. Once created, binary log files are kept forever. So if you do not want to run out of disk space, you should either purge old files with PURGE BINARY LOGS or setexpire_logs_daysto specify after how many days the logs will be automatically purged.Binary logging, however, is not free, so if you do not need for instance on a replica that is not a master, it is recommended to keep it disabled.
skip_name_resolve: when a client connects, the server will perform hostname resolution, and when DNS is slow, establishing the connection will become slow as well. It is therefore recommended to start the server withskip-name-resolveto disable all DNS lookups. The only limitation is that theGRANTstatements must then use IP addresses only, so be careful when adding this setting to an existing system.Conclusion
There are of course other settings that can make a difference depending on your workload or your hardware: low memory and fast disks, high concurrency, write-intensive workloads, for instance, are cases when you will need specific tuning. However, the goal here is to give you a few MySQL performance tuning to allow you to quickly get a sane MySQL configuration without spending too much time on changing non-essential MySQL settings or on reading documentation to understand which settings do matter to you.