search

geerlingguy / ansible-role-mysql

Ansible Role - MySQL
https://galaxy.ansible.com/geerlingguy/mysql/
MIT License
1.05k stars 865 forks source link

MySQL 5.7 daemon won't start on CentOS 7 #418

Closed jchoponis closed 3 years ago

jchoponis commented 4 years ago

I'm trying to use this module to spin up a vanilla MySQL 5.7 instance on a plain CentOS 7 VM some testing and it fails at the step named "Ensure MySQL is started and enabled on boot.".

Based on the list of evidence below, I'm hoping someone else might look at this and spot the issue. This looks like some kind of permissions bug/issue per the systemctl output of: Nov 06 09:31:40 centos7x64-1 mysqld_safe[15562]: /usr/bin/mysqld_safe: line 616: /var/log/mysqld.err: Permission denied, but I'm not 100% sure yet.

In case anyone notes I'm using password authentication (and exposing those passwords below), that's OK for me as this is just an isolated, expendable lab environment. As we know, robust deployments should use SSH keys, and that is just not a goal for this specific exercise.

Any insights about this issue welcome - thanks for reading!

PLAY [db] **********************************************************************************************************************************************************************************************************

TASK [Gathering Facts] *********************************************************************************************************************************************************************************************
ok: [192.168.131.135]

TASK [Install the MySQL repo.] *************************************************************************************************************************************************************************************
changed: [192.168.131.135]

TASK [Override variables for MySQL (RedHat).] **********************************************************************************************************************************************************************
ok: [192.168.131.135]

TASK [geerlingguy.mysql : include_tasks] ***************************************************************************************************************************************************************************
included: /Users/jchoponis/git/ansible-playground/roles/geerlingguy.mysql/tasks/variables.yml for 192.168.131.135

TASK [geerlingguy.mysql : Include OS-specific variables.] **********************************************************************************************************************************************************
ok: [192.168.131.135] => (item=/Users/jchoponis/git/ansible-playground/roles/geerlingguy.mysql/vars/RedHat-7.yml)

TASK [geerlingguy.mysql : Define mysql_packages.] ******************************************************************************************************************************************************************
skipping: [192.168.131.135]

TASK [geerlingguy.mysql : Define mysql_daemon.] ********************************************************************************************************************************************************************
skipping: [192.168.131.135]

TASK [geerlingguy.mysql : Define mysql_slow_query_log_file.] *******************************************************************************************************************************************************
ok: [192.168.131.135]

TASK [geerlingguy.mysql : Define mysql_log_error.] *****************************************************************************************************************************************************************
skipping: [192.168.131.135]

TASK [geerlingguy.mysql : Define mysql_syslog_tag.] ****************************************************************************************************************************************************************
skipping: [192.168.131.135]

TASK [geerlingguy.mysql : Define mysql_pid_file.] ******************************************************************************************************************************************************************
skipping: [192.168.131.135]

TASK [geerlingguy.mysql : Define mysql_config_file.] ***************************************************************************************************************************************************************
ok: [192.168.131.135]

TASK [geerlingguy.mysql : Define mysql_config_include_dir.] ********************************************************************************************************************************************************
ok: [192.168.131.135]

TASK [geerlingguy.mysql : Define mysql_socket.] ********************************************************************************************************************************************************************
skipping: [192.168.131.135]

TASK [geerlingguy.mysql : Define mysql_supports_innodb_large_prefix.] **********************************************************************************************************************************************
ok: [192.168.131.135]

TASK [geerlingguy.mysql : include_tasks] ***************************************************************************************************************************************************************************
included: /Users/jchoponis/git/ansible-playground/roles/geerlingguy.mysql/tasks/setup-RedHat.yml for 192.168.131.135

TASK [geerlingguy.mysql : Ensure MySQL packages are installed.] ****************************************************************************************************************************************************
changed: [192.168.131.135]

TASK [geerlingguy.mysql : include_tasks] ***************************************************************************************************************************************************************************
skipping: [192.168.131.135]

TASK [geerlingguy.mysql : include_tasks] ***************************************************************************************************************************************************************************
skipping: [192.168.131.135]

TASK [geerlingguy.mysql : Check if MySQL packages were installed.] *************************************************************************************************************************************************
ok: [192.168.131.135]

TASK [geerlingguy.mysql : include_tasks] ***************************************************************************************************************************************************************************
included: /Users/jchoponis/git/ansible-playground/roles/geerlingguy.mysql/tasks/configure.yml for 192.168.131.135

TASK [geerlingguy.mysql : Get MySQL version.] **********************************************************************************************************************************************************************
ok: [192.168.131.135]

TASK [geerlingguy.mysql : Copy my.cnf global MySQL configuration.] *************************************************************************************************************************************************
changed: [192.168.131.135]

TASK [geerlingguy.mysql : Verify mysql include directory exists.] **************************************************************************************************************************************************
skipping: [192.168.131.135]

TASK [geerlingguy.mysql : Copy my.cnf override files into include directory.] **************************************************************************************************************************************

TASK [geerlingguy.mysql : Create slow query log file (if configured).] *********************************************************************************************************************************************
skipping: [192.168.131.135]

TASK [geerlingguy.mysql : Create datadir if it does not exist] *****************************************************************************************************************************************************
ok: [192.168.131.135]

TASK [geerlingguy.mysql : Set ownership on slow query log file (if configured).] ***********************************************************************************************************************************
skipping: [192.168.131.135]

TASK [geerlingguy.mysql : Create error log file (if configured).] **************************************************************************************************************************************************
skipping: [192.168.131.135]

TASK [geerlingguy.mysql : Set ownership on error log file (if configured).] ****************************************************************************************************************************************
skipping: [192.168.131.135]

TASK [geerlingguy.mysql : Ensure MySQL is started and enabled on boot.] ********************************************************************************************************************************************
fatal: [192.168.131.135]: FAILED! => {"changed": false, "msg": "Unable to start service mysqld: Job for mysqld.service failed because a timeout was exceeded. See \"systemctl status mysqld.service\" and \"journalctl -xe\" for details.\n"}

RUNNING HANDLER [geerlingguy.mysql : restart mysql] ****************************************************************************************************************************************************************

PLAY RECAP *********************************************************************************************************************************************************************************************************
192.168.131.135            : ok=16   changed=3    unreachable=0    failed=1    skipped=14   rescued=0    ignored=0

Here's the journalctl output:

[root@centos7x64-1 justin]# journalctl -xe
-- The result is failed.
Nov 06 09:21:39 centos7x64-1 systemd[1]: Unit mysqld.service entered failed state.
Nov 06 09:21:39 centos7x64-1 systemd[1]: mysqld.service failed.
Nov 06 09:21:39 centos7x64-1 polkitd[686]: Unregistered Authentication Agent for unix-process:10831:38466 (system bus name :1.58, object path /org/freedesktop/PolicyKit1/AuthenticationAgent, locale en_US.UTF-8) (disconnected from bus)
Nov 06 09:21:39 centos7x64-1 sudo[10821]: pam_unix(sudo:session): session closed for user root
Nov 06 09:21:39 centos7x64-1 systemd[1]: mysqld.service holdoff time over, scheduling restart.
Nov 06 09:21:39 centos7x64-1 systemd[1]: Stopped MySQL Community Server.
-- Subject: Unit mysqld.service has finished shutting down
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit mysqld.service has finished shutting down.
Nov 06 09:21:39 centos7x64-1 systemd[1]: Starting MySQL Community Server...
-- Subject: Unit mysqld.service has begun start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit mysqld.service has begun starting up.
Nov 06 09:21:40 centos7x64-1 mysqld_safe[13222]: /usr/bin/mysqld_safe: line 616: /var/log/mysqld.err: Permission denied
Nov 06 09:21:40 centos7x64-1 mysqld_safe[13222]: 201106 09:21:40 mysqld_safe error: log-error set to '/var/log/mysqld.err', however file don't exists. Create writable for user 'mysql'.
Nov 06 09:21:40 centos7x64-1 mysqld_safe[13222]: /usr/bin/mysqld_safe: line 138: /var/log/mysqld.err: Permission denied
Nov 06 09:21:40 centos7x64-1 systemd[1]: mysqld.service: main process exited, code=exited, status=1/FAILURE
Nov 06 09:22:39 centos7x64-1 sshd[9937]: Received disconnect from 192.168.131.1 port 52017:11: disconnected by user
Nov 06 09:22:39 centos7x64-1 sshd[9937]: Disconnected from 192.168.131.1 port 52017
Nov 06 09:22:39 centos7x64-1 sshd[9933]: pam_unix(sshd:session): session closed for user justin
Nov 06 09:22:39 centos7x64-1 systemd-logind[685]: Removed session 2.
-- Subject: Session 2 has been terminated
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- Documentation: http://www.freedesktop.org/wiki/Software/systemd/multiseat
-- 
-- A session with the ID 2 has been terminated.
Nov 06 09:24:59 centos7x64-1 unix_chkpwd[14316]: password check failed for user (justin)
Nov 06 09:24:59 centos7x64-1 sshd[14305]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=ip-192-168-131-1.us-east-2.compute.internal  user=justin
Nov 06 09:25:01 centos7x64-1 sshd[14305]: Failed password for justin from 192.168.131.1 port 52333 ssh2
Nov 06 09:25:05 centos7x64-1 unix_chkpwd[14332]: password check failed for user (justin)
Nov 06 09:25:07 centos7x64-1 sshd[14305]: Failed password for justin from 192.168.131.1 port 52333 ssh2
Nov 06 09:25:14 centos7x64-1 sshd[14305]: Accepted password for justin from 192.168.131.1 port 52333 ssh2
Nov 06 09:25:15 centos7x64-1 systemd-logind[685]: New session 3 of user justin.
-- Subject: A new session 3 has been created for user justin
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- Documentation: http://www.freedesktop.org/wiki/Software/systemd/multiseat
-- 
-- A new session with the ID 3 has been created for the user justin.
-- 
-- The leading process of the session is 14305.
Nov 06 09:25:15 centos7x64-1 systemd[1]: Started Session 3 of user justin.
-- Subject: Unit session-3.scope has finished start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit session-3.scope has finished starting up.
-- 
-- The start-up result is done.
Nov 06 09:25:15 centos7x64-1 sshd[14305]: pam_unix(sshd:session): session opened for user justin by (uid=0)
Nov 06 09:25:28 centos7x64-1 sudo[14391]:   justin : TTY=pts/0 ; PWD=/home/justin ; USER=root ; COMMAND=/bin/su
Nov 06 09:25:28 centos7x64-1 sudo[14391]: pam_unix(sudo:session): session opened for user root by justin(uid=0)
Nov 06 09:25:28 centos7x64-1 su[14425]: (to root) justin on pts/0
Nov 06 09:25:28 centos7x64-1 su[14425]: pam_unix(su:session): session opened for user root by justin(uid=0)

Here's the systemctl output:

[root@centos7x64-1 justin]# systemctl status mysqld
● mysqld.service - MySQL Community Server
   Loaded: loaded (/usr/lib/systemd/system/mysqld.service; enabled; vendor preset: disabled)
   Active: activating (start-post) (Result: exit-code) since Fri 2020-11-06 09:31:39 EST; 10s ago
  Process: 15562 ExecStart=/usr/bin/mysqld_safe --basedir=/usr (code=exited, status=1/FAILURE)
  Process: 15550 ExecStartPre=/usr/bin/mysql-systemd-start pre (code=exited, status=0/SUCCESS)
 Main PID: 15562 (code=exited, status=1/FAILURE);         : 15563 (mysql-systemd-s)
   CGroup: /system.slice/mysqld.service
           └─control
             ├─15563 /bin/bash /usr/bin/mysql-systemd-start post
             └─16104 sleep 1

Nov 06 09:31:39 centos7x64-1 systemd[1]: Starting MySQL Community Server...
Nov 06 09:31:40 centos7x64-1 mysqld_safe[15562]: /usr/bin/mysqld_safe: line 616: /var/log/mysqld.err: Permission denied
Nov 06 09:31:40 centos7x64-1 systemd[1]: mysqld.service: main process exited, code=exited, status=1/FAILURE

My ansible version on the mac os box driving the playbook:

$ ansible --version
ansible 2.10.3
  config file = /Users/jchoponis/git/ansible-playground/ansible.cfg
  configured module search path = ['/Users/jchoponis/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python3.9/site-packages/ansible
  executable location = /usr/local/bin/ansible
  python version = 3.9.0 (default, Nov  4 2020, 19:24:28) [Clang 12.0.0 (clang-1200.0.32.21)]

CentOS version: 

[root@centos7x64-1 justin]# cat /etc/redhat-release
CentOS Linux release 7.8.2003 (Core)

Role version per roles/geerlingguy.mysql/meta/.galaxy_install_info

install_date: Fri Nov  6 14:08:24 2020
version: 3.3.0

And the playbook:

- hosts: db
  become: yes
  roles:
    - { role: geerlingguy.mysql }
  vars:
    mysql_root_password: Tango-Alpha-98745
    mysql_databases:
      - name: example_db
        encoding: latin1
        collation: latin1_general_ci
    mysql_users:
      - name: example_user
        host: "%"
        password: Tango-Alpha-12345
        priv: "example_db.*:ALL"
  pre_tasks:
    - name: Install the MySQL repo.
      yum:
        name: http://repo.mysql.com/mysql-community-release-el7-5.noarch.rpm
        state: present
      when: ansible_os_family == "RedHat"
    - name: Override variables for MySQL (RedHat).
      set_fact:
        mysql_daemon: mysqld
        mysql_packages: ['mysql-server']
        mysql_log_error: /var/log/mysqld.err
        mysql_syslog_tag: mysqld
        mysql_pid_file: /var/run/mysqld/mysqld.pid
        mysql_socket: /var/lib/mysql/mysql.sock
      when: ansible_os_family == "RedHat"

The bash script I'm using to run that playbook.

ansible-playbook ./install-mysql-5.7.yml -i ./hosts -u justin --ask-pass -K

Also, I'm using Mac OS 10.15.7 and VMware Fusion 11.5.6, in case these matter.

vovtz commented 3 years ago

I have the same issue and I’m using an almost identical setup, but with Vagrant and VirtualBox. I noticed it skips task Create error log file (if configured)., therefore Ansible didn’t create file /var/log/mysqld.err and the mysqld user is not allowed to create that file in runtime.

My first idea was that the task was being skipped because of its tag skip_ansible_galaxy (guess the name pushed me in the wrong direction). Turns out that the when condition is the cause. Solution is to also configure mysql_log. For example:

    - name: Override variables for MySQL (RedHat)
      set_fact:
          # (…)
          mysql_log: /var/log/mysqld
          mysql_log_error: syslog: /var/log/mysqld.err
          # (…)
      when: ansible_os_family == "RedHat"
AntonNguyen97 commented 3 years ago

any updates regarding this issue?

stale[bot] commented 3 years ago

This issue has been marked 'stale' due to lack of recent activity. If there is no further activity, the issue will be closed in another 30 days. Thank you for your contribution!

Please read this blog post to see the reasons why I mark issues as stale.

stale[bot] commented 3 years ago

This issue has been closed due to inactivity. If you feel this is in error, please reopen the issue or file a new issue with the relevant details.