search

renzhengeek / issues

0 stars 0 forks source link

ocfs2-test:multi nodes: xattr last for 24 hours #17

Open renzhengeek opened 8 years ago

renzhengeek commented 8 years ago

multi-xattr-test/xattr-multiple-test-log-run-x86_64-2015-09-24-10-28-27.log :

=====================Multiple nodes xattr testing starts: Thu Sep 24 10:28:27 CST 2015=====================
++++++++++xattr tests with "-b 512 -C 4096"++++++++++
Mkfsing device(-b 512 -C 4096): mkfs.ocfs2 1.8.2
Cluster stack: pcmk
Cluster name: hacluster
Stack Flags: 0x0
NOTE: Feature extended slot map may be enabled
Overwriting existing ocfs2 partition.
Proceed (y/N): Label: ocfs2-xattr-multi-test-x86_64
Features: sparse extended-slotmap backup-super unwritten inline-data strict-journal-super xattr indexed-dirs refcount discontig-bg
Block size: 512 (9 bits)
Cluster size: 4096 (12 bits)
Volume size: 53687091200 (13107200 clusters) (104857600 blocks)
Cluster groups: 3658 (tail covers 512 clusters, rest cover 3584 clusters)
Extent allocator size: 27262976 (26 groups)
Journal size: 33554432
Node slots: 2
Creating bitmaps: done
Initializing superblock: done
Writing system files: done
Writing superblock: done
Writing backup superblock: 3 block(s)
Formatting Journals: done
Growing extent allocator: done
Formatting slot map: done
Formatting quota files: done
Writing lost+found: done
mkfs.ocfs2 successful

[ PASS ]
Mounting device /dev/sdb to nodes(m1,m2):+ /usr/lib64/mpi/gcc/openmpi/bin/mpirun --allow-run-as-root -mca btl tcp,self -mca plm_rsh_agent ssh:rsh -np 2 --host m1,m2 sudo udevadm trigger
+ /usr/lib64/mpi/gcc/openmpi/bin/mpirun --allow-run-as-root -mca btl tcp,self -mca plm_rsh_agent ssh:rsh -np 2 --host m1,m2 /home/ocfs2test/bin/ocfs2/bin/command.py --mount -l ocfs2-xattr-multi-test-x86_64 -m /mnt/shared
[ PASS ]
[1] Check Namespace&Filetype of Multinode Xattr on Ocfs2:[ PASS ]
[2] Check Utility of Multinode Xattr on Ocfs2:[ PASS ]
[3] Check Max Multinode Xattr EA_Name_Length:[ PASS ]
[4] Check Max Multinode Xattr EA_Size:[ PASS ]
[5] Check Huge Multinode Xattr EA_Entry_Nums:[ PASS ]
[6] Check All Max Multinode Xattr Arguments Together:[ PASS ]
[7] Launch Concurrent Adding Test:[ PASS ]
[8] Launch MultiNode Xattr Stress Test:[ PASS ]
Umounting device /dev/sdb from nodes(m1,m2):+ /usr/lib64/mpi/gcc/openmpi/bin/mpirun --allow-run-as-root -mca btl tcp,self -mca plm_rsh_agent ssh:rsh -np 2 --host m1,m2 /home/ocfs2test/bin/ocfs2/bin/command.py --umount -m /mnt/shared
[ PASS ]
++++++++++xattr tests with "-b 512 -C 1048576"++++++++++
Mkfsing device(-b 512 -C 1048576): mkfs.ocfs2 1.8.2
Cluster stack: pcmk
Cluster name: hacluster
Stack Flags: 0x0
NOTE: Feature extended slot map may be enabled
Overwriting existing ocfs2 partition.
Proceed (y/N): Label: ocfs2-xattr-multi-test-x86_64
Features: sparse extended-slotmap backup-super unwritten inline-data strict-journal-super xattr indexed-dirs refcount discontig-bg
Block size: 512 (9 bits)
Cluster size: 1048576 (20 bits)
Volume size: 53687091200 (51200 clusters) (104857600 blocks)
Cluster groups: 15 (tail covers 1024 clusters, rest cover 3584 clusters)
Extent allocator size: 27262976 (26 groups)
Journal size: 33554432
Node slots: 2
Creating bitmaps: done
Initializing superblock: done
Writing system files: done
Writing superblock: done
Writing backup superblock: 3 block(s)
Formatting Journals: done
Growing extent allocator: done
Formatting slot map: done
Formatting quota files: done
Writing lost+found: done
mkfs.ocfs2 successful

[ PASS ]
Mounting device /dev/sdb to nodes(m1,m2):+ /usr/lib64/mpi/gcc/openmpi/bin/mpirun --allow-run-as-root -mca btl tcp,self -mca plm_rsh_agent ssh:rsh -np 2 --host m1,m2 sudo udevadm trigger
+ /usr/lib64/mpi/gcc/openmpi/bin/mpirun --allow-run-as-root -mca btl tcp,self -mca plm_rsh_agent ssh:rsh -np 2 --host m1,m2 /home/ocfs2test/bin/ocfs2/bin/command.py --mount -l ocfs2-xattr-multi-test-x86_64 -m /mnt/shared
[ PASS ]
[9] Check Namespace&Filetype of Multinode Xattr on Ocfs2:[ PASS ]
[10] Check Utility of Multinode Xattr on Ocfs2:[ PASS ]
[11] Check Max Multinode Xattr EA_Name_Length:[ PASS ]
[12] Check Max Multinode Xattr EA_Size:[ PASS ]
[13] Check Huge Multinode Xattr EA_Entry_Nums:[ PASS ]
[14] Check All Max Multinode Xattr Arguments Together:[ PASS ]
[15] Launch Concurrent Adding Test:
renzhengeek commented 8 years ago

multi-xattr-test/xattr-multiple-test-log-x86_64-2015-09-24-10-28-27.log:

=====================Multiple nodes xattr testing starts: Thu Sep 24 10:28:27 CST 2015=====================
++++++++++xattr tests with "-b 512 -C 4096"++++++++++

==========================================================
[1] Check Namespace&Filetype of Multinode Xattr on Ocfs2:
==========================================================
Testing Binary:     /usr/bin/sudo -u root /usr/lib64/mpi/gcc/openmpi/bin/mpirun --allow-run-as-root -mca plm_rsh_agent ssh:rsh -mca btl tcp,self  -np 4 --host m1,m2 /home/ocfs2test/bin/ocfs2/bin/xattr-multi-test -i 1 -x 20 -n user -t normal -l 50 -s 200 /mnt/shared/multi_xattr_test_place
********user mode on normal********
******************************************************
**************Round 0 test running...*****************
******************************************************
Test 1: Creating commonfile /mnt/shared/multi_xattr_test_place/test_normal-0 on m1(rank 0),to perform 20 EAs.
Test 2: Performancing Xattrs on /mnt/shared/multi_xattr_test_place/test_normal-0,all ranks take race to do update/remove/add/read 20 EAs.
Test 3: Listing all EAs of file /mnt/shared/multi_xattr_test_place/test_normal-0 by all nodes.
Test 4: Removing all EAs on file /mnt/shared/multi_xattr_test_place/test_normal-0.
Test 5: Verifying if all EAs removed from file /mnt/shared/multi_xattr_test_place/test_normal-0 by rank0.
Testing Binary:     /usr/bin/sudo -u root /usr/lib64/mpi/gcc/openmpi/bin/mpirun --allow-run-as-root -mca plm_rsh_agent ssh:rsh -mca btl tcp,self  -np 4 --host m1,m2 /home/ocfs2test/bin/ocfs2/bin/xattr-multi-test -i 1 -x 20 -n user -t directory -l 50 -s 200 /mnt/shared/multi_xattr_test_place
********user mode on directory********
******************************************************
**************Round 0 test running...*****************
******************************************************
Test 1: Creating directory /mnt/shared/multi_xattr_test_place/test_directory-0 on m1(rank 0),to perform 20 EAs.
Test 2: Performancing Xattrs on /mnt/shared/multi_xattr_test_place/test_directory-0,all ranks take race to do update/remove/add/read 20 EAs.
Test 3: Listing all EAs of file /mnt/shared/multi_xattr_test_place/test_directory-0 by all nodes.
Test 4: Removing all EAs on file /mnt/shared/multi_xattr_test_place/test_directory-0.
Test 5: Verifying if all EAs removed from file /mnt/shared/multi_xattr_test_place/test_directory-0 by rank0.
Testing Binary:     /usr/bin/sudo -u root /usr/lib64/mpi/gcc/openmpi/bin/mpirun --allow-run-as-root -mca plm_rsh_agent ssh:rsh -mca btl tcp,self  -np 4 --host m1,m2 /home/ocfs2test/bin/ocfs2/bin/xattr-multi-test -i 1 -x 20 -n user -t symlink -l 50 -s 200 /mnt/shared/multi_xattr_test_place
********user mode on symlink********
******************************************************
**************Round 0 test running...*****************
******************************************************
Test 1: Creating symlink /mnt/shared/multi_xattr_test_place/test_symlink-0 on m1(rank 0),to perform 20 EAs.
Test 2: Performancing Xattrs on /mnt/shared/multi_xattr_test_place/test_symlink-0,all ranks take race to do update/remove/add/read 20 EAs.
Failed at lsetxattr(add,errno:1,Operation not permitted) on /mnt/shared/multi_xattr_test_place/test_symlink-0:xattr_name=user.a4Gd6Sn6Rf6Lu6Vm2Jy5Al1Qj6Mi1Fx0Pb4Qt1Y000000,xattr_value_sz=187,xattr_value=abcdefghijklmnopqrsuser.a4Gd6Sn6Rf6Lu6Vm2Jy5Al1Qj6Mi1Fx0Pb4Qt1Y0000004Lw3Cy7Sa0Ik6Jj7Ah3Xw7Tn8Nj1Xa8Yl4Lp2Dk4Gs8Fd2Vd2Td0Gw8Aj6Pj8Qh7Qu3Rx7Uf5Hk7Ei2Nb1Pk6Hm5Jb4Ms00186srqponmlkjihgfedcba
Rank:0 on Host(m1) abort!
--------------------------------------------------------------------------
MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_WORLD 
with errorcode 1.

NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes.
You may or may not see output from other processes, depending on
exactly when Open MPI kills them.
--------------------------------------------------------------------------
[m2][[22779,1],2][btl_tcp_frag.c:215:mca_btl_tcp_frag_recv] mca_btl_tcp_frag_recv: readv failed: Connection reset by peer (104)
Testing Binary:     /usr/bin/sudo -u root /usr/lib64/mpi/gcc/openmpi/bin/mpirun --allow-run-as-root -mca plm_rsh_agent ssh:rsh -mca btl tcp,self  -np 4 --host m1,m2 /home/ocfs2test/bin/ocfs2/bin/xattr-multi-test -i 1 -x 20 -n trusted -t normal -l 50 -s 200 /mnt/shared/multi_xattr_test_place
********trusted mode on normal********
******************************************************
**************Round 0 test running...*****************
******************************************************
Test 1: Creating commonfile /mnt/shared/multi_xattr_test_place/test_normal-0 on m1(rank 0),to perform 20 EAs.
Test 2: Performancing Xattrs on /mnt/shared/multi_xattr_test_place/test_normal-0,all ranks take race to do update/remove/add/read 20 EAs.
Test 3: Listing all EAs of file /mnt/shared/multi_xattr_test_place/test_normal-0 by all nodes.
Test 4: Removing all EAs on file /mnt/shared/multi_xattr_test_place/test_normal-0.
Test 5: Verifying if all EAs removed from file /mnt/shared/multi_xattr_test_place/test_normal-0 by rank0.
Testing Binary:     /usr/bin/sudo -u root /usr/lib64/mpi/gcc/openmpi/bin/mpirun --allow-run-as-root -mca plm_rsh_agent ssh:rsh -mca btl tcp,self  -np 4 --host m1,m2 /home/ocfs2test/bin/ocfs2/bin/xattr-multi-test -i 1 -x 20 -n trusted -t directory -l 50 -s 200 /mnt/shared/multi_xattr_test_place
********trusted mode on directory********
******************************************************
**************Round 0 test running...*****************
******************************************************
Test 1: Creating directory /mnt/shared/multi_xattr_test_place/test_directory-0 on m1(rank 0),to perform 20 EAs.
Test 2: Performancing Xattrs on /mnt/shared/multi_xattr_test_place/test_directory-0,all ranks take race to do update/remove/add/read 20 EAs.
Test 3: Listing all EAs of file /mnt/shared/multi_xattr_test_place/test_directory-0 by all nodes.
Test 4: Removing all EAs on file /mnt/shared/multi_xattr_test_place/test_directory-0.
Test 5: Verifying if all EAs removed from file /mnt/shared/multi_xattr_test_place/test_directory-0 by rank0.
Testing Binary:     /usr/bin/sudo -u root /usr/lib64/mpi/gcc/openmpi/bin/mpirun --allow-run-as-root -mca plm_rsh_agent ssh:rsh -mca btl tcp,self  -np 4 --host m1,m2 /home/ocfs2test/bin/ocfs2/bin/xattr-multi-test -i 1 -x 20 -n trusted -t symlink -l 50 -s 200 /mnt/shared/multi_xattr_test_place
********trusted mode on symlink********
******************************************************
**************Round 0 test running...*****************
******************************************************
Test 1: Creating symlink /mnt/shared/multi_xattr_test_place/test_symlink-0 on m1(rank 0),to perform 20 EAs.
Test 2: Performancing Xattrs on /mnt/shared/multi_xattr_test_place/test_symlink-0,all ranks take race to do update/remove/add/read 20 EAs.
Test 3: Listing all EAs of file /mnt/shared/multi_xattr_test_place/test_symlink-0 by all nodes.
Test 4: Removing all EAs on file /mnt/shared/multi_xattr_test_place/test_symlink-0.
Test 5: Verifying if all EAs removed from file /mnt/shared/multi_xattr_test_place/test_symlink-0 by rank0.

==========================================================
[2] Check Utility of Multinode Xattr on Ocfs2:
==========================================================
Testing Binary:     /usr/lib64/mpi/gcc/openmpi/bin/mpirun -mca plm_rsh_agent ssh:rsh -mca btl tcp,self  -np 4 --host m1,m2 /home/ocfs2test/bin/ocfs2/bin/xattr-multi-test -i 1 -x 10 -n user -t normal -l 50 -s 100 /mnt/shared/multi_xattr_test_place
******************************************************
**************Round 0 test running...*****************
******************************************************
Test 1: Creating commonfile /mnt/shared/multi_xattr_test_place/test_normal-0 on m1(rank 0),to perform 10 EAs.
Test 2: Performancing Xattrs on /mnt/shared/multi_xattr_test_place/test_normal-0,all ranks take race to do update/remove/add/read 10 EAs.
Test 3: Listing all EAs of file /mnt/shared/multi_xattr_test_place/test_normal-0 by all nodes.
Test 4: Removing all EAs on file /mnt/shared/multi_xattr_test_place/test_normal-0.
Test 5: Verifying if all EAs removed from file /mnt/shared/multi_xattr_test_place/test_normal-0 by rank0.
Testing Binary:     /usr/lib64/mpi/gcc/openmpi/bin/mpirun -mca plm_rsh_agent ssh:rsh -mca btl tcp,self  -np 4 --host m1,m2 /home/ocfs2test/bin/ocfs2/bin/xattr-multi-test -i 1 -x 10 -n user -t normal -l 50 -s 100 /mnt/shared/multi_xattr_test_place
******************************************************
**************Round 0 test running...*****************
******************************************************
Test 1: Creating commonfile /mnt/shared/multi_xattr_test_place/test_normal-0 on m1(rank 0),to perform 10 EAs.
Test 2: Performancing Xattrs on /mnt/shared/multi_xattr_test_place/test_normal-0,all ranks take race to do update/remove/add/read 10 EAs.
Test 3: Listing all EAs of file /mnt/shared/multi_xattr_test_place/test_normal-0 by all nodes.
Test 4: Removing all EAs on file /mnt/shared/multi_xattr_test_place/test_normal-0.
Test 5: Verifying if all EAs removed from file /mnt/shared/multi_xattr_test_place/test_normal-0 by rank0.
Testing Binary:     /usr/lib64/mpi/gcc/openmpi/bin/mpirun -mca plm_rsh_agent ssh:rsh -mca btl tcp,self  -np 4 --host m1,m2 /home/ocfs2test/bin/ocfs2/bin/xattr-multi-test -i 1 -x 10 -n user -t normal -l 50 -s 100 /mnt/shared/multi_xattr_test_place
******************************************************
**************Round 0 test running...*****************
******************************************************
Test 1: Creating commonfile /mnt/shared/multi_xattr_test_place/test_normal-0 on m1(rank 0),to perform 10 EAs.
Test 2: Performancing Xattrs on /mnt/shared/multi_xattr_test_place/test_normal-0,all ranks take race to do update/remove/add/read 10 EAs.
Test 3: Listing all EAs of file /mnt/shared/multi_xattr_test_place/test_normal-0 by all nodes.
Test 4: Removing all EAs on file /mnt/shared/multi_xattr_test_place/test_normal-0.
Test 5: Verifying if all EAs removed from file /mnt/shared/multi_xattr_test_place/test_normal-0 by rank0.
Testing Binary:     /usr/lib64/mpi/gcc/openmpi/bin/mpirun -mca plm_rsh_agent ssh:rsh -mca btl tcp,self  -np 4 --host m1,m2 /home/ocfs2test/bin/ocfs2/bin/xattr-multi-test -i 1 -x 10 -n user -t normal -l 50 -s 100 /mnt/shared/multi_xattr_test_place
******************************************************
**************Round 0 test running...*****************
******************************************************
Test 1: Creating commonfile /mnt/shared/multi_xattr_test_place/test_normal-0 on m1(rank 0),to perform 10 EAs.
Test 2: Performancing Xattrs on /mnt/shared/multi_xattr_test_place/test_normal-0,all ranks take race to do update/remove/add/read 10 EAs.
Test 3: Listing all EAs of file /mnt/shared/multi_xattr_test_place/test_normal-0 by all nodes.
Test 4: Removing all EAs on file /mnt/shared/multi_xattr_test_place/test_normal-0.
Test 5: Verifying if all EAs removed from file /mnt/shared/multi_xattr_test_place/test_normal-0 by rank0.

==========================================================
[3] Check Max Multinode Xattr EA_Name_Length:
==========================================================
Testing Binary:     /usr/lib64/mpi/gcc/openmpi/bin/mpirun -mca plm_rsh_agent ssh:rsh -mca btl tcp,self  -np 4 --host m1,m2 /home/ocfs2test/bin/ocfs2/bin/xattr-multi-test -i 1 -x 4 -n user -t normal -l 255 -s 300 /mnt/shared/multi_xattr_test_place
******************************************************
**************Round 0 test running...*****************
******************************************************
Test 1: Creating commonfile /mnt/shared/multi_xattr_test_place/test_normal-0 on m1(rank 0),to perform 4 EAs.
Test 2: Performancing Xattrs on /mnt/shared/multi_xattr_test_place/test_normal-0,all ranks take race to do update/remove/add/read 4 EAs.
Test 3: Listing all EAs of file /mnt/shared/multi_xattr_test_place/test_normal-0 by all nodes.
Test 4: Removing all EAs on file /mnt/shared/multi_xattr_test_place/test_normal-0.
Test 5: Verifying if all EAs removed from file /mnt/shared/multi_xattr_test_place/test_normal-0 by rank0.

==========================================================
[4] Check Max Multinode Xattr EA_Size:
==========================================================
Testing Binary:     /usr/lib64/mpi/gcc/openmpi/bin/mpirun -mca plm_rsh_agent ssh:rsh -mca btl tcp,self  -np 4 --host m1,m2 /home/ocfs2test/bin/ocfs2/bin/xattr-multi-test -i 1 -x 1 -n user -t normal -l 50 -s 65536 /mnt/shared/multi_xattr_test_place
******************************************************
**************Round 0 test running...*****************
******************************************************
Test 1: Creating commonfile /mnt/shared/multi_xattr_test_place/test_normal-0 on m1(rank 0),to perform 1 EAs.
Test 2: Performancing Xattrs on /mnt/shared/multi_xattr_test_place/test_normal-0,all ranks take race to do update/remove/add/read 1 EAs.
Test 3: Listing all EAs of file /mnt/shared/multi_xattr_test_place/test_normal-0 by all nodes.
Test 4: Removing all EAs on file /mnt/shared/multi_xattr_test_place/test_normal-0.
Test 5: Verifying if all EAs removed from file /mnt/shared/multi_xattr_test_place/test_normal-0 by rank0.

==========================================================
[5] Check Huge Multinode Xattr EA_Entry_Nums:
==========================================================
Testing Binary:     /usr/lib64/mpi/gcc/openmpi/bin/mpirun -mca plm_rsh_agent ssh:rsh -mca btl tcp,self  -np 4 --host m1,m2 /home/ocfs2test/bin/ocfs2/bin/xattr-multi-test -i 1 -x 10000 -n user -t normal -l 100 -s 200 /mnt/shared/multi_xattr_test_place
Warning:list size exceed,due to (xattr_name_sz + 1) * xattr_nums was greater than 65536,will not launch list test!
Warning:list size exceed,due to (xattr_name_sz + 1) * xattr_nums was greater than 65536,will not launch list test!
******************************************************
**************Round 0 test running...*****************
******************************************************
Test 1: Creating commonfile /mnt/shared/multi_xattr_test_place/test_normal-0 on m1(rank 0),to perform 10000 EAs.
Warning:list size exceed,due to (xattr_name_sz + 1) * xattr_nums was greater than 65536,will not launch list test!
Warning:list size exceed,due to (xattr_name_sz + 1) * xattr_nums was greater than 65536,will not launch list test!
Test 2: Performancing Xattrs on /mnt/shared/multi_xattr_test_place/test_normal-0,all ranks take race to do update/remove/add/read 10000 EAs.
Test 3: Removing all EAs on file /mnt/shared/multi_xattr_test_place/test_normal-0.
Test 4: Verifying if all EAs removed from file /mnt/shared/multi_xattr_test_place/test_normal-0 by rank0.

==========================================================
[6] Check All Max Multinode Xattr Arguments Together:
==========================================================
Testing Binary:     /usr/lib64/mpi/gcc/openmpi/bin/mpirun -mca plm_rsh_agent ssh:rsh -mca btl tcp,self  -np 4 --host m1,m2 /home/ocfs2test/bin/ocfs2/bin/xattr-multi-test -i 1 -x 1000 -n user -t normal -l 255 -s 65536 /mnt/shared/multi_xattr_test_place
Warning:list size exceed,due to (xattr_name_sz + 1) * xattr_nums was greater than 65536,will not launch list test!
******************************************************
**************Round 0 test running...*****************
Warning:list size exceed,due to (xattr_name_sz + 1) * xattr_nums was greater than 65536,will not launch list test!
******************************************************
Test 1: Creating commonfile /mnt/shared/multi_xattr_test_place/test_normal-0 on m1(rank 0),to perform 1000 EAs.
Warning:list size exceed,due to (xattr_name_sz + 1) * xattr_nums was greater than 65536,will not launch list test!
Warning:list size exceed,due to (xattr_name_sz + 1) * xattr_nums was greater than 65536,will not launch list test!
Test 2: Performancing Xattrs on /mnt/shared/multi_xattr_test_place/test_normal-0,all ranks take race to do update/remove/add/read 1000 EAs.
Test 3: Removing all EAs on file /mnt/shared/multi_xattr_test_place/test_normal-0.
Test 4: Verifying if all EAs removed from file /mnt/shared/multi_xattr_test_place/test_normal-0 by rank0.

==========================================================
[7] Launch Concurrent Adding Test:
==========================================================
Testing Binary:     /usr/lib64/mpi/gcc/openmpi/bin/mpirun -mca plm_rsh_agent ssh:rsh -mca btl tcp,self  -np 4 --host m1,m2 /home/ocfs2test/bin/ocfs2/bin/xattr-multi-test -i 1 -x 1000 -n user -t normal -l 255 -s 5000 -o -r -k /mnt/shared/multi_xattr_test_place
******************************************************
**************Round 0 test running...*****************
******************************************************
Test 1: Creating commonfile /mnt/shared/multi_xattr_test_place/test_normal-0 on m1(rank 0),to perform 1000 EAs.
Test 2: Performancing Xattr operations on /mnt/shared/multi_xattr_test_place/test_normal-0,all ranks take race to add 1000 EAs.

==========================================================
[8] Launch MultiNode Xattr Stress Test:
==========================================================
Testing Binary:     /usr/lib64/mpi/gcc/openmpi/bin/mpirun -mca plm_rsh_agent ssh:rsh -mca btl tcp,self  -np 4 --host m1,m2 /home/ocfs2test/bin/ocfs2/bin/xattr-multi-test -i 1 -x 2000 -n user -t normal -l 255 -s 5000  -r -k /mnt/shared/multi_xattr_test_place
Warning:list size exceed,due to (xattr_name_sz + 1) * xattr_nums was greater than 65536,Warning:list size exceed,due to (xattr_name_sz + 1) * xattr_nums was greater than 65536,will not launch list test!
will not launch list test!
******************************************************
**************Round 0 test running...*****************
******************************************************
Test 1: Creating commonfile /mnt/shared/multi_xattr_test_place/test_normal-0 on m1(rank 0),to perform 2000 EAs.
Warning:list size exceed,due to (xattr_name_sz + 1) * xattr_nums was greater than 65536,will not launch list test!
Warning:list size exceed,due to (xattr_name_sz + 1) * xattr_nums was greater than 65536,will not launch list test!
Test 2: Performancing Xattrs on /mnt/shared/multi_xattr_test_place/test_normal-0,all ranks take race to do update/remove/add/read 2000 EAs.

++++++++++xattr tests with "-b 512 -C 1048576"++++++++++

==========================================================
[9] Check Namespace&Filetype of Multinode Xattr on Ocfs2:
==========================================================
Testing Binary:     /usr/bin/sudo -u root /usr/lib64/mpi/gcc/openmpi/bin/mpirun --allow-run-as-root -mca plm_rsh_agent ssh:rsh -mca btl tcp,self  -np 4 --host m1,m2 /home/ocfs2test/bin/ocfs2/bin/xattr-multi-test -i 1 -x 20 -n user -t normal -l 50 -s 200 /mnt/shared/multi_xattr_test_place
********user mode on normal********
******************************************************
**************Round 0 test running...*****************
******************************************************
Test 1: Creating commonfile /mnt/shared/multi_xattr_test_place/test_normal-0 on m1(rank 0),to perform 20 EAs.
Test 2: Performancing Xattrs on /mnt/shared/multi_xattr_test_place/test_normal-0,all ranks take race to do update/remove/add/read 20 EAs.
Test 3: Listing all EAs of file /mnt/shared/multi_xattr_test_place/test_normal-0 by all nodes.
Test 4: Removing all EAs on file /mnt/shared/multi_xattr_test_place/test_normal-0.
Test 5: Verifying if all EAs removed from file /mnt/shared/multi_xattr_test_place/test_normal-0 by rank0.
Testing Binary:     /usr/bin/sudo -u root /usr/lib64/mpi/gcc/openmpi/bin/mpirun --allow-run-as-root -mca plm_rsh_agent ssh:rsh -mca btl tcp,self  -np 4 --host m1,m2 /home/ocfs2test/bin/ocfs2/bin/xattr-multi-test -i 1 -x 20 -n user -t directory -l 50 -s 200 /mnt/shared/multi_xattr_test_place
********user mode on directory********
******************************************************
**************Round 0 test running...*****************
******************************************************
Test 1: Creating directory /mnt/shared/multi_xattr_test_place/test_directory-0 on m1(rank 0),to perform 20 EAs.
Test 2: Performancing Xattrs on /mnt/shared/multi_xattr_test_place/test_directory-0,all ranks take race to do update/remove/add/read 20 EAs.
Test 3: Listing all EAs of file /mnt/shared/multi_xattr_test_place/test_directory-0 by all nodes.
Test 4: Removing all EAs on file /mnt/shared/multi_xattr_test_place/test_directory-0.
Test 5: Verifying if all EAs removed from file /mnt/shared/multi_xattr_test_place/test_directory-0 by rank0.
Testing Binary:     /usr/bin/sudo -u root /usr/lib64/mpi/gcc/openmpi/bin/mpirun --allow-run-as-root -mca plm_rsh_agent ssh:rsh -mca btl tcp,self  -np 4 --host m1,m2 /home/ocfs2test/bin/ocfs2/bin/xattr-multi-test -i 1 -x 20 -n user -t symlink -l 50 -s 200 /mnt/shared/multi_xattr_test_place
********user mode on symlink********
******************************************************
**************Round 0 test running...*****************
******************************************************
Test 1: Creating symlink /mnt/shared/multi_xattr_test_place/test_symlink-0 on m1(rank 0),to perform 20 EAs.
Test 2: Performancing Xattrs on /mnt/shared/multi_xattr_test_place/test_symlink-0,all ranks take race to do update/remove/add/read 20 EAs.
Failed at lsetxattr(add,errno:1,Operation not permitted) on /mnt/shared/multi_xattr_test_place/test_symlink-0:xattr_name=user.d6Rm4Xv1Ct4No4Yv6De7Hs4So6Bz8Td0Ej0Kw6F000000,xattr_value_sz=94,xattr_value=abcdefghijklmnopqrsuser.d6Rm4Xv1Ct4No4Yv6De7Hs4So6Bz8Td0Ej0Kw6F00000000093srqponmlkjihgfedcba
Rank:0 on Host(m1) abort!
--------------------------------------------------------------------------
MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_WORLD 
with errorcode 1.

NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes.
You may or may not see output from other processes, depending on
exactly when Open MPI kills them.
--------------------------------------------------------------------------
[m2][[4435,1],2][btl_tcp_frag.c:215:mca_btl_tcp_frag_recv] mca_btl_tcp_frag_recv: readv failed: Connection reset by peer (104)
Testing Binary:     /usr/bin/sudo -u root /usr/lib64/mpi/gcc/openmpi/bin/mpirun --allow-run-as-root -mca plm_rsh_agent ssh:rsh -mca btl tcp,self  -np 4 --host m1,m2 /home/ocfs2test/bin/ocfs2/bin/xattr-multi-test -i 1 -x 20 -n trusted -t normal -l 50 -s 200 /mnt/shared/multi_xattr_test_place
********trusted mode on normal********
******************************************************
**************Round 0 test running...*****************
******************************************************
Test 1: Creating commonfile /mnt/shared/multi_xattr_test_place/test_normal-0 on m1(rank 0),to perform 20 EAs.
Test 2: Performancing Xattrs on /mnt/shared/multi_xattr_test_place/test_normal-0,all ranks take race to do update/remove/add/read 20 EAs.
Test 3: Listing all EAs of file /mnt/shared/multi_xattr_test_place/test_normal-0 by all nodes.
Test 4: Removing all EAs on file /mnt/shared/multi_xattr_test_place/test_normal-0.
Test 5: Verifying if all EAs removed from file /mnt/shared/multi_xattr_test_place/test_normal-0 by rank0.
Testing Binary:     /usr/bin/sudo -u root /usr/lib64/mpi/gcc/openmpi/bin/mpirun --allow-run-as-root -mca plm_rsh_agent ssh:rsh -mca btl tcp,self  -np 4 --host m1,m2 /home/ocfs2test/bin/ocfs2/bin/xattr-multi-test -i 1 -x 20 -n trusted -t directory -l 50 -s 200 /mnt/shared/multi_xattr_test_place
********trusted mode on directory********
******************************************************
**************Round 0 test running...*****************
******************************************************
Test 1: Creating directory /mnt/shared/multi_xattr_test_place/test_directory-0 on m1(rank 0),to perform 20 EAs.
Test 2: Performancing Xattrs on /mnt/shared/multi_xattr_test_place/test_directory-0,all ranks take race to do update/remove/add/read 20 EAs.
Test 3: Listing all EAs of file /mnt/shared/multi_xattr_test_place/test_directory-0 by all nodes.
Test 4: Removing all EAs on file /mnt/shared/multi_xattr_test_place/test_directory-0.
Test 5: Verifying if all EAs removed from file /mnt/shared/multi_xattr_test_place/test_directory-0 by rank0.
Testing Binary:     /usr/bin/sudo -u root /usr/lib64/mpi/gcc/openmpi/bin/mpirun --allow-run-as-root -mca plm_rsh_agent ssh:rsh -mca btl tcp,self  -np 4 --host m1,m2 /home/ocfs2test/bin/ocfs2/bin/xattr-multi-test -i 1 -x 20 -n trusted -t symlink -l 50 -s 200 /mnt/shared/multi_xattr_test_place
********trusted mode on symlink********
******************************************************
**************Round 0 test running...*****************
******************************************************
Test 1: Creating symlink /mnt/shared/multi_xattr_test_place/test_symlink-0 on m1(rank 0),to perform 20 EAs.
Test 2: Performancing Xattrs on /mnt/shared/multi_xattr_test_place/test_symlink-0,all ranks take race to do update/remove/add/read 20 EAs.
Test 3: Listing all EAs of file /mnt/shared/multi_xattr_test_place/test_symlink-0 by all nodes.
Test 4: Removing all EAs on file /mnt/shared/multi_xattr_test_place/test_symlink-0.
Test 5: Verifying if all EAs removed from file /mnt/shared/multi_xattr_test_place/test_symlink-0 by rank0.

==========================================================
[10] Check Utility of Multinode Xattr on Ocfs2:
==========================================================
Testing Binary:     /usr/lib64/mpi/gcc/openmpi/bin/mpirun -mca plm_rsh_agent ssh:rsh -mca btl tcp,self  -np 4 --host m1,m2 /home/ocfs2test/bin/ocfs2/bin/xattr-multi-test -i 1 -x 10 -n user -t normal -l 50 -s 100 /mnt/shared/multi_xattr_test_place
******************************************************
**************Round 0 test running...*****************
******************************************************
Test 1: Creating commonfile /mnt/shared/multi_xattr_test_place/test_normal-0 on m1(rank 0),to perform 10 EAs.
Test 2: Performancing Xattrs on /mnt/shared/multi_xattr_test_place/test_normal-0,all ranks take race to do update/remove/add/read 10 EAs.
Test 3: Listing all EAs of file /mnt/shared/multi_xattr_test_place/test_normal-0 by all nodes.
Test 4: Removing all EAs on file /mnt/shared/multi_xattr_test_place/test_normal-0.
Test 5: Verifying if all EAs removed from file /mnt/shared/multi_xattr_test_place/test_normal-0 by rank0.
Testing Binary:     /usr/lib64/mpi/gcc/openmpi/bin/mpirun -mca plm_rsh_agent ssh:rsh -mca btl tcp,self  -np 4 --host m1,m2 /home/ocfs2test/bin/ocfs2/bin/xattr-multi-test -i 1 -x 10 -n user -t normal -l 50 -s 100 /mnt/shared/multi_xattr_test_place
******************************************************
**************Round 0 test running...*****************
******************************************************
Test 1: Creating commonfile /mnt/shared/multi_xattr_test_place/test_normal-0 on m1(rank 0),to perform 10 EAs.
Test 2: Performancing Xattrs on /mnt/shared/multi_xattr_test_place/test_normal-0,all ranks take race to do update/remove/add/read 10 EAs.
Test 3: Listing all EAs of file /mnt/shared/multi_xattr_test_place/test_normal-0 by all nodes.
Test 4: Removing all EAs on file /mnt/shared/multi_xattr_test_place/test_normal-0.
Test 5: Verifying if all EAs removed from file /mnt/shared/multi_xattr_test_place/test_normal-0 by rank0.
Testing Binary:     /usr/lib64/mpi/gcc/openmpi/bin/mpirun -mca plm_rsh_agent ssh:rsh -mca btl tcp,self  -np 4 --host m1,m2 /home/ocfs2test/bin/ocfs2/bin/xattr-multi-test -i 1 -x 10 -n user -t normal -l 50 -s 100 /mnt/shared/multi_xattr_test_place
******************************************************
**************Round 0 test running...*****************
******************************************************
Test 1: Creating commonfile /mnt/shared/multi_xattr_test_place/test_normal-0 on m1(rank 0),to perform 10 EAs.
Test 2: Performancing Xattrs on /mnt/shared/multi_xattr_test_place/test_normal-0,all ranks take race to do update/remove/add/read 10 EAs.
Test 3: Listing all EAs of file /mnt/shared/multi_xattr_test_place/test_normal-0 by all nodes.
Test 4: Removing all EAs on file /mnt/shared/multi_xattr_test_place/test_normal-0.
Test 5: Verifying if all EAs removed from file /mnt/shared/multi_xattr_test_place/test_normal-0 by rank0.
Testing Binary:     /usr/lib64/mpi/gcc/openmpi/bin/mpirun -mca plm_rsh_agent ssh:rsh -mca btl tcp,self  -np 4 --host m1,m2 /home/ocfs2test/bin/ocfs2/bin/xattr-multi-test -i 1 -x 10 -n user -t normal -l 50 -s 100 /mnt/shared/multi_xattr_test_place
******************************************************
**************Round 0 test running...*****************
******************************************************
Test 1: Creating commonfile /mnt/shared/multi_xattr_test_place/test_normal-0 on m1(rank 0),to perform 10 EAs.
Test 2: Performancing Xattrs on /mnt/shared/multi_xattr_test_place/test_normal-0,all ranks take race to do update/remove/add/read 10 EAs.
Test 3: Listing all EAs of file /mnt/shared/multi_xattr_test_place/test_normal-0 by all nodes.
Test 4: Removing all EAs on file /mnt/shared/multi_xattr_test_place/test_normal-0.
Test 5: Verifying if all EAs removed from file /mnt/shared/multi_xattr_test_place/test_normal-0 by rank0.

==========================================================
[11] Check Max Multinode Xattr EA_Name_Length:
==========================================================
Testing Binary:     /usr/lib64/mpi/gcc/openmpi/bin/mpirun -mca plm_rsh_agent ssh:rsh -mca btl tcp,self  -np 4 --host m1,m2 /home/ocfs2test/bin/ocfs2/bin/xattr-multi-test -i 1 -x 4 -n user -t normal -l 255 -s 300 /mnt/shared/multi_xattr_test_place
******************************************************
**************Round 0 test running...*****************
******************************************************
Test 1: Creating commonfile /mnt/shared/multi_xattr_test_place/test_normal-0 on m1(rank 0),to perform 4 EAs.
Test 2: Performancing Xattrs on /mnt/shared/multi_xattr_test_place/test_normal-0,all ranks take race to do update/remove/add/read 4 EAs.
Test 3: Listing all EAs of file /mnt/shared/multi_xattr_test_place/test_normal-0 by all nodes.
Test 4: Removing all EAs on file /mnt/shared/multi_xattr_test_place/test_normal-0.
Test 5: Verifying if all EAs removed from file /mnt/shared/multi_xattr_test_place/test_normal-0 by rank0.

==========================================================
[12] Check Max Multinode Xattr EA_Size:
==========================================================
Testing Binary:     /usr/lib64/mpi/gcc/openmpi/bin/mpirun -mca plm_rsh_agent ssh:rsh -mca btl tcp,self  -np 4 --host m1,m2 /home/ocfs2test/bin/ocfs2/bin/xattr-multi-test -i 1 -x 1 -n user -t normal -l 50 -s 65536 /mnt/shared/multi_xattr_test_place
******************************************************
**************Round 0 test running...*****************
******************************************************
Test 1: Creating commonfile /mnt/shared/multi_xattr_test_place/test_normal-0 on m1(rank 0),to perform 1 EAs.
Test 2: Performancing Xattrs on /mnt/shared/multi_xattr_test_place/test_normal-0,all ranks take race to do update/remove/add/read 1 EAs.
Test 3: Listing all EAs of file /mnt/shared/multi_xattr_test_place/test_normal-0 by all nodes.
Test 4: Removing all EAs on file /mnt/shared/multi_xattr_test_place/test_normal-0.
Test 5: Verifying if all EAs removed from file /mnt/shared/multi_xattr_test_place/test_normal-0 by rank0.

==========================================================
[13] Check Huge Multinode Xattr EA_Entry_Nums:
==========================================================
Testing Binary:     /usr/lib64/mpi/gcc/openmpi/bin/mpirun -mca plm_rsh_agent ssh:rsh -mca btl tcp,self  -np 4 --host m1,m2 /home/ocfs2test/bin/ocfs2/bin/xattr-multi-test -i 1 -x 10000 -n user -t normal -l 100 -s 200 /mnt/shared/multi_xattr_test_place
Warning:list size exceed,due to (xattr_name_sz + 1) * xattr_nums was greater than 65536,will not launch list test!
Warning:list size exceed,due to (xattr_name_sz + 1) * xattr_nums was greater than 65536,******************************************************
**************Round 0 test running...*****************
******************************************************
will not launch list test!
Test 1: Creating commonfile /mnt/shared/multi_xattr_test_place/test_normal-0 on m1(rank 0),to perform 10000 EAs.
Warning:list size exceed,due to (xattr_name_sz + 1) * xattr_nums was greater than 65536,will not launch list test!
Warning:list size exceed,due to (xattr_name_sz + 1) * xattr_nums was greater than 65536,will not launch list test!
Test 2: Performancing Xattrs on /mnt/shared/multi_xattr_test_place/test_normal-0,all ranks take race to do update/remove/add/read 10000 EAs.
Test 3: Removing all EAs on file /mnt/shared/multi_xattr_test_place/test_normal-0.
Test 4: Verifying if all EAs removed from file /mnt/shared/multi_xattr_test_place/test_normal-0 by rank0.

==========================================================
[14] Check All Max Multinode Xattr Arguments Together:
==========================================================
Testing Binary:     /usr/lib64/mpi/gcc/openmpi/bin/mpirun -mca plm_rsh_agent ssh:rsh -mca btl tcp,self  -np 4 --host m1,m2 /home/ocfs2test/bin/ocfs2/bin/xattr-multi-test -i 1 -x 1000 -n user -t normal -l 255 -s 65536 /mnt/shared/multi_xattr_test_place
Warning:list size exceed,due to (xattr_name_sz + 1) * xattr_nums was greater than 65536,will not launch list test!
Warning:list size exceed,due to (xattr_name_sz + 1) * xattr_nums was greater than 65536,******************************************************
**************Round 0 test running...*****************
******************************************************
will not launch list test!
Test 1: Creating commonfile /mnt/shared/multi_xattr_test_place/test_normal-0 on m1(rank 0),to perform 1000 EAs.
Warning:list size exceed,due to (xattr_name_sz + 1) * xattr_nums was greater than 65536,will not launch list test!
Warning:list size exceed,due to (xattr_name_sz + 1) * xattr_nums was greater than 65536,will not launch list test!
Test 2: Performancing Xattrs on /mnt/shared/multi_xattr_test_place/test_normal-0,all ranks take race to do update/remove/add/read 1000 EAs.
Test 3: Removing all EAs on file /mnt/shared/multi_xattr_test_place/test_normal-0.
Test 4: Verifying if all EAs removed from file /mnt/shared/multi_xattr_test_place/test_normal-0 by rank0.

==========================================================
[15] Launch Concurrent Adding Test:
==========================================================
Testing Binary:     /usr/lib64/mpi/gcc/openmpi/bin/mpirun -mca plm_rsh_agent ssh:rsh -mca btl tcp,self  -np 4 --host m1,m2 /home/ocfs2test/bin/ocfs2/bin/xattr-multi-test -i 1 -x 1000 -n user -t normal -l 255 -s 5000 -o -r -k /mnt/shared/multi_xattr_test_place
******************************************************
**************Round 0 test running...*****************
******************************************************
Test 1: Creating commonfile /mnt/shared/multi_xattr_test_place/test_normal-0 on m1(rank 0),to perform 1000 EAs.
Test 2: Performancing Xattr operations on /mnt/shared/multi_xattr_test_place/test_normal-0,all ranks take race to add 1000 EAs.