about mariadb innodb_buffer_pool_size in k8s

[zhuochengs]

Hi I have a question: the configration of innodb_buffer_pool_size in k8s. I have a node in k8s, and this node have 125 GBs memory and 112 cpus(logic). now I deploy the mariadb pod on this node, and it's my resource configration like that:

    resources: 
    requests:
      memory: "80Gi"
      cpu: "50"
    limits:
      memory: "80Gi"
      cpu: "50"

and then I saw some log says that do not set the pod's memory over the request's memory*70%, so I set the configration like that: innodb_buffer_pool_size = 56000M

But now I find the mariadb's pod's memory increased very fast, it almost above the 56000M.

So I want know, if I want my mariadb's pod use memory as much as possible in this db-node, how to set the right configration, about the k8s request 、limit resource( CPU and memory), and the mariadb innodb_buffer_pool_size ? Thanks!

ps: this is my current configration about mariadb:

config: |-
    [mysqld]
    skip-name-resolve
    explicit_defaults_for_timestamp
    basedir=/opt/bitnami/mariadb
    port=3306
    socket=/opt/bitnami/mariadb/tmp/mysql.sock
    tmpdir=/opt/bitnami/mariadb/tmp
    bind-address=0.0.0.0
    pid-file=/opt/bitnami/mariadb/tmp/mysqld.pid
    log-error=/opt/bitnami/mariadb/logs/mysqld.log
    character-set-server=utf8mb4
    collation-server=utf8mb4_bin
    # optimize
    max_allowed_packet = 1024M
    innodb_log_file_size = 1G
    innodb_flush_method = O_DIRECT
    innodb_flush_log_at_trx_commit = 1
    innodb_log_buffer_size = 64M
    innodb_buffer_pool_size=56000M
    table_open_cache = 512
    sort_buffer_size = 2M
    read_buffer_size = 2M
    read_rnd_buffer_size = 8M
    thread_cache_size = 8
    query_cache_size = 32M
    max_heap_table_size=1024M
    tmp_table_size=1024M
    max_connections=65535
    max_connect_errors=65535
    wait_timeout=172800
    interactive_timeout=172800
    connect_timeout=30
    # log settings
    expire_logs_days=7
    # close strict 
    sql_mode=IGNORE_SPACE,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION

    [client]
    port=3306
    socket=/opt/bitnami/mariadb/tmp/mysql.sock
    default-character-set=utf8mb4

    [manager]
    port=3306
    socket=/opt/bitnami/mariadb/tmp/mysql.sock
    pid-file=/opt/bitnami/mariadb/tmp/mysqld.pid

[wglambert]

This isn't a Kubernetes or MariaDB forum but I'll give it my best

innodb_buffer_pool_size isn't the same thing as container memory, it occupies memory but there's still other operations/processes that can consume memory. Staying under 70% of the pod's request is a best practice to not go oom. You're setting 70% of that as the baseline. But actually it's more than that because the calculation that MariaDB uses for the innodb_buffer_pool_size is performed in chunks. https://dev.mysql.com/doc/refman/5.7/en/innodb-buffer-pool-resize.html

Chunk size is defined by the innodb_buffer_pool_chunk_size configuration option, which has a default of 128M. For more information, see Configuring InnoDB Buffer Pool Chunk Size.

Buffer pool size must always be equal to or a multiple of innodb_buffer_pool_chunk_size innodb_buffer_pool_instances. If you configure innodb_buffer_pool_size to a value that is not equal to or a multiple of innodb_buffer_pool_chunk_size innodb_buffer_pool_instances, buffer pool size is automatically adjusted to a value that is equal to or a multiple of innodb_buffer_pool_chunk_size * innodb_buffer_pool_instances.

Also Kubernetes resource requests are just information for the kube-scheduler to fit the pod appropriately, it doesn't actually bestow the pod with a set amount of memory, it's just a number meant to represent how much that pod should use as an average which is completely subjective to its average workload. Then the limit will actually restrict the amount of memory the pod will use, it actually implements a memory_limit on the container (different depending on your container runtime) https://docs.docker.com/engine/docker-overview/#control-groups

For further questions you should ask a Kubernetes/MariaDB resource/forum or Stack Overflow.

[zhuochengs]

Thanks a lot! @wglambert !