search

quantum / esos

An open source, high performance, block-level storage platform.
http://www.esos-project.com/
Other
284 stars 58 forks source link

Pacemaker Error #72

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 

i activate drab and corosync + pacemaker but pacemaker does not start and in 
the log i have a strange message

 04 15:57:13 [3309] esosc1.dytech.local        cib:     info: crm_get_peer:     Node 167837953 has uuid 167837953
Apr 04 15:57:13 [3309] esosc1.dytech.local        cib:     info: 
qb_ipcs_us_publish:     server name: cib_ro
Apr 04 15:57:13 [3309] esosc1.dytech.local        cib:     info: 
qb_ipcs_us_publish:     server name: cib_rw
Apr 04 15:57:13 [3309] esosc1.dytech.local        cib:     info: 
qb_ipcs_us_publish:     server name: cib_shm
Apr 04 15:57:13 [3309] esosc1.dytech.local        cib:     info: cib_init: 
    Starting cib mainloop
Apr 04 15:57:13 [3309] esosc1.dytech.local        cib:     info: 
pcmk_cpg_membership:    Joined[0.0] cib.167837953
Apr 04 15:57:13 [3309] esosc1.dytech.local        cib:     info: 
pcmk_cpg_membership:    Member[0.0] cib.167837953
Apr 04 15:57:14 [3309] esosc1.dytech.local        cib:    error: 
qb_sys_mmap_file_open:  couldn't open file 
/dev/shm/qb-cib_rw-control-3309-3310-9: Permission denied (13)
Apr 04 15:57:14 [3309] esosc1.dytech.local        cib:    error: 
qb_sys_mmap_file_open:  couldn't open file 
/usr/var/run/qb-cib_rw-control-3309-3310-9: No such file or directory (2)
Apr 04 15:57:14 [3309] esosc1.dytech.local        cib:    error: 
qb_ipcs_us_connect:     couldn't create file for mmap (3309-3310-9): No such file 
or directory (2)
Apr 04 15:57:14 [3309] esosc1.dytech.local        cib:    error: 
handle_new_connection:  Error in connection setup (3309-3310-9): No such file 
or directory (2)
Apr 04 15:57:14 [3310] esosc1.dytech.local stonith-ng:     info: 
crm_ipc_connect:    Could not establish cib_rw connection: No such file or 
directory (2)
Apr 04 15:57:14 [3309] esosc1.dytech.local        cib:    error: 
qb_sys_mmap_file_open:  couldn't open file 
/dev/shm/qb-cib_shm-request-3309-3314-10-header: Permission denied (13)
Apr 04 15:57:14 [3309] esosc1.dytech.local        cib:    error: 
qb_sys_mmap_file_open:  couldn't open file 
/usr/var/run/qb-cib_shm-request-3309-3314-10-header: No such file or directory 
(2)
Apr 04 15:57:14 [3309] esosc1.dytech.local        cib:    error: qb_rb_open: 
    couldn't create file for mmap
Apr 04 15:57:14 [3309] esosc1.dytech.local        cib:    error: 
qb_ipcs_shm_rb_open:    qb_rb_open:cib_shm-request-3309-3314-10: No such file or 
directory (2)
Apr 04 15:57:14 [3309] esosc1.dytech.local        cib:    error: 
qb_ipcs_shm_connect:    shm connection FAILED: No such file or directory (2)
Apr 04 15:57:14 [3309] esosc1.dytech.local        cib:    error: 
handle_new_connection:  Error in connection setup (3309-3314-10): No such file 
or directory (2)
Apr 04 15:57:14 [3314] esosc1.dytech.local       crmd:     info: 
crm_ipc_connect:    Could not establish cib_shm connection: No such file or 
directory (2)
Apr 04 15:57:14 [3314] esosc1.dytech.local       crmd:     info: 
do_cib_control:     Could not connect to the CIB service: Transport endpoint is 
not connected
Apr 04 15:57:14 [3314] esosc1.dytech.local       crmd:  warning: 
do_cib_control:     Couldn't complete CIB registration 1 times... pause and retry
Apr 04 15:57:16 [3309] esosc1.dytech.local        cib:    error: 
qb_sys_mmap_file_open:  couldn't open file 
/dev/shm/qb-cib_rw-control-3309-3310-11: Permission denied (13)
Apr 04 15:57:16 [3309] esosc1.dytech.local        cib:    error: 
qb_sys_mmap_file_open:  couldn't open file 
/usr/var/run/qb-cib_rw-control-3309-3310-11: No such file or directory (2)
Apr 04 15:57:16 [3309] esosc1.dytech.local        cib:    error: 
qb_ipcs_us_connect:     couldn't create file for mmap (3309-3310-11): No such file 
or directory (2)
Apr 04 15:57:16 [3309] esosc1.dytech.local        cib:    error: 
handle_new_connection:  Error in connection setup (3309-3310-11): No such file 
or directory (2)
Apr 04 15:57:16 [3310] esosc1.dytech.local stonith-ng:     info: 
crm_ipc_connect:    Could not establish cib_rw connection: No such file or 
directory (2)
Apr 04 15:57:16 [3314] esosc1.dytech.local       crmd:     info: 
crm_timer_popped:   Wait Timer (I_NULL) just popped (2000ms)
Apr 04 15:57:16 [3309] esosc1.dytech.local        cib:    error: 
qb_sys_mmap_file_open:  couldn't open file 
/dev/shm/qb-cib_shm-request-3309-3314-12-header: Permission denied (13)
Apr 04 15:57:16 [3309] esosc1.dytech.local        cib:    error: 
qb_sys_mmap_file_open:  couldn't open file 
/usr/var/run/qb-cib_shm-request-3309-3314-12-header: No such file or directory 
(2)
Apr 04 15:57:16 [3309] esosc1.dytech.local        cib:    error: qb_rb_open: 
    couldn't create file for mmap
Apr 04 15:57:1

What steps will reproduce the problem?
1.Configure Corosync
2.start Corosync
3.start pacemaker

What is the expected output? What do you see instead?
 04 15:57:13 [3309] esosc1.dytech.local        cib:     info: crm_get_peer:     Node 167837953 has uuid 167837953
Apr 04 15:57:13 [3309] esosc1.dytech.local        cib:     info: 
qb_ipcs_us_publish:     server name: cib_ro
Apr 04 15:57:13 [3309] esosc1.dytech.local        cib:     info: 
qb_ipcs_us_publish:     server name: cib_rw
Apr 04 15:57:13 [3309] esosc1.dytech.local        cib:     info: 
qb_ipcs_us_publish:     server name: cib_shm
Apr 04 15:57:13 [3309] esosc1.dytech.local        cib:     info: cib_init: 
    Starting cib mainloop
Apr 04 15:57:13 [3309] esosc1.dytech.local        cib:     info: 
pcmk_cpg_membership:    Joined[0.0] cib.167837953
Apr 04 15:57:13 [3309] esosc1.dytech.local        cib:     info: 
pcmk_cpg_membership:    Member[0.0] cib.167837953
Apr 04 15:57:14 [3309] esosc1.dytech.local        cib:    error: 
qb_sys_mmap_file_open:  couldn't open file 
/dev/shm/qb-cib_rw-control-3309-3310-9: Permission denied (13)
Apr 04 15:57:14 [3309] esosc1.dytech.local        cib:    error: 
qb_sys_mmap_file_open:  couldn't open file 
/usr/var/run/qb-cib_rw-control-3309-3310-9: No such file or directory (2)
Apr 04 15:57:14 [3309] esosc1.dytech.local        cib:    error: 
qb_ipcs_us_connect:     couldn't create file for mmap (3309-3310-9): No such file 
or directory (2)
Apr 04 15:57:14 [3309] esosc1.dytech.local        cib:    error: 
handle_new_connection:  Error in connection setup (3309-3310-9): No such file 
or directory (2)
Apr 04 15:57:14 [3310] esosc1.dytech.local stonith-ng:     info: 
crm_ipc_connect:    Could not establish cib_rw connection: No such file or 
directory (2)
Apr 04 15:57:14 [3309] esosc1.dytech.local        cib:    error: 
qb_sys_mmap_file_open:  couldn't open file 
/dev/shm/qb-cib_shm-request-3309-3314-10-header: Permission denied (13)
Apr 04 15:57:14 [3309] esosc1.dytech.local        cib:    error: 
qb_sys_mmap_file_open:  couldn't open file 
/usr/var/run/qb-cib_shm-request-3309-3314-10-header: No such file or directory 
(2)
Apr 04 15:57:14 [3309] esosc1.dytech.local        cib:    error: qb_rb_open: 
    couldn't create file for mmap
Apr 04 15:57:14 [3309] esosc1.dytech.local        cib:    error: 
qb_ipcs_shm_rb_open:    qb_rb_open:cib_shm-request-3309-3314-10: No such file or 
directory (2)
Apr 04 15:57:14 [3309] esosc1.dytech.local        cib:    error: 
qb_ipcs_shm_connect:    shm connection FAILED: No such file or directory (2)
Apr 04 15:57:14 [3309] esosc1.dytech.local        cib:    error: 
handle_new_connection:  Error in connection setup (3309-3314-10): No such file 
or directory (2)
Apr 04 15:57:14 [3314] esosc1.dytech.local       crmd:     info: 
crm_ipc_connect:    Could not establish cib_shm connection: No such file or 
directory (2)
Apr 04 15:57:14 [3314] esosc1.dytech.local       crmd:     info: 
do_cib_control:     Could not connect to the CIB service: Transport endpoint is 
not connected
Apr 04 15:57:14 [3314] esosc1.dytech.local       crmd:  warning: 
do_cib_control:     Couldn't complete CIB registration 1 times... pause and retry
Apr 04 15:57:16 [3309] esosc1.dytech.local        cib:    error: 
qb_sys_mmap_file_open:  couldn't open file 
/dev/shm/qb-cib_rw-control-3309-3310-11: Permission denied (13)
Apr 04 15:57:16 [3309] esosc1.dytech.local        cib:    error: 
qb_sys_mmap_file_open:  couldn't open file 
/usr/var/run/qb-cib_rw-control-3309-3310-11: No such file or directory (2)
Apr 04 15:57:16 [3309] esosc1.dytech.local        cib:    error: 
qb_ipcs_us_connect:     couldn't create file for mmap (3309-3310-11): No such file 
or directory (2)
Apr 04 15:57:16 [3309] esosc1.dytech.local        cib:    error: 
handle_new_connection:  Error in connection setup (3309-3310-11): No such file 
or directory (2)
Apr 04 15:57:16 [3310] esosc1.dytech.local stonith-ng:     info: 
crm_ipc_connect:    Could not establish cib_rw connection: No such file or 
directory (2)
Apr 04 15:57:16 [3314] esosc1.dytech.local       crmd:     info: 
crm_timer_popped:   Wait Timer (I_NULL) just popped (2000ms)
Apr 04 15:57:16 [3309] esosc1.dytech.local        cib:    error: 
qb_sys_mmap_file_open:  couldn't open file 
/dev/shm/qb-cib_shm-request-3309-3314-12-header: Permission denied (13)
Apr 04 15:57:16 [3309] esosc1.dytech.local        cib:    error: 
qb_sys_mmap_file_open:  couldn't open file 
/usr/var/run/qb-cib_shm-request-3309-3314-12-header: No such file or directory 
(2)
Apr 04 15:57:16 [3309] esosc1.dytech.local        cib:    error: qb_rb_open: 
    couldn't create file for mmap
Apr 04 15:57:1

What version of the product are you using? On what operating system?
1-r622

Please provide any additional information below.

Original issue reported on code.google.com by dotcom...@gmail.com on 4 Apr 2014 at 4:08

GoogleCodeExporter commented 9 years ago 
i resolve , with change the userid in /etc/passwd for hacluster

hacluster:x:0:0:ha_cluster user:/var/lib/pacemaker/cores/hacluster:/sbin/nologin

but this is not a good solution, can you help me?

Original comment by dotcom...@gmail.com on 4 Apr 2014 at 4:18

GoogleCodeExporter commented 9 years ago 
Hi,

I just looked at a machine running an older version of ESOS and it definitely 
appears to be a permissions problem. On the older box I see files owned by 
'hacluster' in /dev/shm and not on a new build (as you described). I'm thinking 
right now it could be related to the permissions on the /dev/shm mount; I think 
these were updated not too many revisions ago. I'll look into it this evening.

In the mean time, I'm updating Pacemaker to 1.1.11... its something I've been 
meaning to do for a while, and this reminded me. =)

So, I'll build an image on my test machine with the new version and see what 
needs to be fixed. Should have something for ya before the weekend is over.

--Marc

Original comment by msmith...@gmail.com on 4 Apr 2014 at 7:44

GoogleCodeExporter commented 9 years ago 
I've fixed this in r626: 
https://googledrive.com/host/0B-MvNl-PpBFPbXplMmhwaElid0U/esos-0.1-r626.tar.xz

Could you please try the new version and confirm?

Thanks,

Marc

Original comment by msmith...@gmail.com on 5 Apr 2014 at 5:37

GoogleCodeExporter commented 9 years ago 
ok, fixed permission. 

but now i have:

local2.log:Apr  7 22:16:35 esosc2 corosync[2129]:  [QB    ] server name: 
votequorum
local2.log:Apr  7 22:16:35 esosc2 corosync[2129]:  [QB    ] server name: quorum
local2.log:Apr  7 22:50:11 esosc2 corosync[2129]:  [QB    ] couldn't create 
file for mmap
local2.log:Apr  7 22:50:11 esosc2 corosync[2129]:  [QB    ] 
qb_rb_open:quorum-response-2129-2256-29: No space left on device (28)
local2.log:Apr  7 22:50:11 esosc2 corosync[2129]:  [QB    ] shm connection 
FAILED: No space left on device (28)
local2.log:Apr  7 22:50:11 esosc2 corosync[2129]:  [QB    ] Error in connection 
setup (2129-2256-29): No space left on device (28)
messages:Apr  7 22:16:35 esosc2 corosync[2129]:  [QB    ] server name: cmap
messages:Apr  7 22:16:35 esosc2 corosync[2129]:  [QB    ] server name: cfg

Original comment by i...@ucspat.org on 7 Apr 2014 at 8:51

GoogleCodeExporter commented 9 years ago 
i just change fstab but not work!!

tmpfs       /dev/shm    tmpfs   size=128m,mode=1777     0 0

Original comment by i...@ucspat.org on 7 Apr 2014 at 10:31

GoogleCodeExporter commented 9 years ago 
ok, if i remount with size=128M working fine!

but on every reboot this is lost.
i don't  know why, i just write this size in fstab and save it.

Original comment by dotcom...@gmail.com on 8 Apr 2014 at 10:33

GoogleCodeExporter commented 9 years ago 
I'll make the change in trunk... for the time being, you have to do this to 
make the change stick:
mount /mnt/root
vi /mnt/root/etc/fstab
umount /mnt/root

Its a chicken-and-egg problem... the core file systems are mounted before the 
USB configuration is sync'd.

--Marc

Original comment by msmith...@gmail.com on 9 Apr 2014 at 1:35

GoogleCodeExporter commented 9 years ago 
I committed this and the new image should be posted in about an hour.

--Marc

Original comment by msmith...@gmail.com on 11 Apr 2014 at 1:11