search

junneyang / zumastor

Automatically exported from code.google.com/p/zumastor
0 stars 1 forks source link

Hang In zumastor's patch #146

Open GoogleCodeExporter opened 9 years ago

GoogleCodeExporter commented 9 years ago 
Hello
Here is a hang problem.  I used lvm2 and xfs to create volume, 
and then create snapshot for this volume.  When the snapshot
was full, mount process was hanged.  I found this process hanged
because of zumastor's patch in __generic_make_request function. 
The mount process was hanged by this 
wait_event(q->throttle_wait, atomic_read(&q->available) >= need);

Is this a race condition problem described by FIXME??
The kernel version is Linux-2.6.24.2 with zumastor's patch.

Thanks a lot

Original issue reported on code.google.com by ej3Georg...@gmail.com on 23 May 2008 at 6:54

GoogleCodeExporter commented 9 years ago 
Did you create snapshot with ddsnap? Or you saw the problem without running 
ddsnap at
all? Could you post the sysrq-t trace?

Original comment by jiayin...@gmail.com on 23 May 2008 at 3:44

GoogleCodeExporter commented 9 years ago 
Sorry, my snapshot is created by lvm.  ...
The sysrq-t trace shows that : 

  task                PC stack   pid father
linuxrc       S 0fef0be0     0     1      0
Call Trace:
[cf813dc0] [cfa0e7f8] 0xcfa0e7f8 (unreliable)
[cf813e80] [c0009214] __switch_to+0x5c/0x74
[cf813ea0] [c0288644] schedule+0x2e8/0x320
[cf813ed0] [c0016f24] do_wait+0xcb0/0xe48
[cf813f40] [c00022e0] ret_from_syscall+0x0/0x3c
kthreadd      S 00000000     0     2      0
Call Trace:
[cf823ec0] [00021000] 0x21000 (unreliable)
[cf823f80] [c0009214] __switch_to+0x5c/0x74
[cf823fa0] [c0288644] schedule+0x2e8/0x320
[cf823fd0] [c0028104] kthreadd+0x74/0xfc
[cf823ff0] [c000461c] kernel_thread+0x44/0x60
ksoftirqd/0   S 00000000     0     3      2
Call Trace:
[cf827ea0] [00029000] 0x29000 (unreliable)
[cf827f60] [c0009214] __switch_to+0x5c/0x74
[cf827f80] [c0288644] schedule+0x2e8/0x320
[cf827fb0] [c0018c98] ksoftirqd+0x38/0xcc
[cf827fd0] [c0027ea0] kthread+0x48/0x84
[cf827ff0] [c000461c] kernel_thread+0x44/0x60
events/0      S 00000000     0     4      2
Call Trace:
[cf829e80] [00029000] 0x29000 (unreliable)
[cf829f40] [c0009214] __switch_to+0x5c/0x74
[cf829f60] [c0288644] schedule+0x2e8/0x320
[cf829f90] [c0023ec0] worker_thread+0x80/0xbc
[cf829fd0] [c0027ea0] kthread+0x48/0x84
[cf829ff0] [c000461c] kernel_thread+0x44/0x60
khelper       S 00000000     0     5      2
Call Trace:
[cf82be80] [c000e370] __enqueue_entity+0xb4/0xc8 (unreliable)
[cf82bf40] [c0009214] __switch_to+0x5c/0x74
[cf82bf60] [c0288644] schedule+0x2e8/0x320
[cf82bf90] [c0023ec0] worker_thread+0x80/0xbc
[cf82bfd0] [c0027ea0] kthread+0x48/0x84
[cf82bff0] [c000461c] kernel_thread+0x44/0x60
scsi_to_zylog S 00000000     0    60      2
Call Trace:
[cf8bde80] [36363834] 0x36363834 (unreliable)
[cf8bdf40] [c0009214] __switch_to+0x5c/0x74
[cf8bdf60] [c0288644] schedule+0x2e8/0x320
[cf8bdf90] [c0023ec0] worker_thread+0x80/0xbc
[cf8bdfd0] [c0027ea0] kthread+0x48/0x84
[cf8bdff0] [c000461c] kernel_thread+0x44/0x60
kblockd/0     S 00000000     0    61      2
Call Trace:
[cf8bfe80] [c000f370] try_to_wake_up+0x74/0x9c (unreliable)
[cf8bff40] [c0009214] __switch_to+0x5c/0x74
[cf8bff60] [c0288644] schedule+0x2e8/0x320
[cf8bff90] [c0023ec0] worker_thread+0x80/0xbc
[cf8bffd0] [c0027ea0] kthread+0x48/0x84
[cf8bfff0] [c000461c] kernel_thread+0x44/0x60
ata/0         S 00000000     0    68      2
Call Trace:
[cf83de80] [cf8142dc] 0xcf8142dc (unreliable)
[cf83df40] [c0009214] __switch_to+0x5c/0x74
[cf83df60] [c0288644] schedule+0x2e8/0x320
[cf83df90] [c0023ec0] worker_thread+0x80/0xbc
[cf83dfd0] [c0027ea0] kthread+0x48/0x84
[cf83dff0] [c000461c] kernel_thread+0x44/0x60
ata_aux       S 00000000     0    69      2
Call Trace:
[cf83fe80] [cf81431c] 0xcf81431c (unreliable)
[cf83ff40] [c0009214] __switch_to+0x5c/0x74
[cf83ff60] [c0288644] schedule+0x2e8/0x320
[cf83ff90] [c0023ec0] worker_thread+0x80/0xbc
[cf83ffd0] [c0027ea0] kthread+0x48/0x84
[cf83fff0] [c000461c] kernel_thread+0x44/0x60
khubd         S 00000000     0    74      2
Call Trace:
[cf8d3e40] [c000fbb4] __wake_up+0x24/0x3c (unreliable)
[cf8d3f00] [c0009214] __switch_to+0x5c/0x74
[cf8d3f20] [c0288644] schedule+0x2e8/0x320
[cf8d3f50] [c01b2730] hub_thread+0xa84/0xacc
[cf8d3fd0] [c0027ea0] kthread+0x48/0x84
[cf8d3ff0] [c000461c] kernel_thread+0x44/0x60
pdflush       S 00000000     0    98      2
Call Trace:
[cfa31e60] [00088210] 0x88210 (unreliable)
[cfa31f20] [c0009214] __switch_to+0x5c/0x74
[cfa31f40] [c0288644] schedule+0x2e8/0x320
[cfa31f70] [c00409e0] pdflush+0xbc/0x1b8
[cfa31fd0] [c0027ea0] kthread+0x48/0x84
[cfa31ff0] [c000461c] kernel_thread+0x44/0x60
pdflush       S 00000000     0    99      2
Call Trace:
[cfa33e60] [c007be80] __writeback_single_inode+0x284/0x29c (unreliable)
[cfa33f20] [c0009214] __switch_to+0x5c/0x74
[cfa33f40] [c0288644] schedule+0x2e8/0x320
[cfa33f70] [c00409e0] pdflush+0xbc/0x1b8
[cfa33fd0] [c0027ea0] kthread+0x48/0x84
[cfa33ff0] [c000461c] kernel_thread+0x44/0x60
kswapd0       S 00000000     0   100      2
Call Trace:
[cfa35e10] [c03d58a0] 0xc03d58a0 (unreliable)
[cfa35ed0] [c0009214] __switch_to+0x5c/0x74
[cfa35ef0] [c0288644] schedule+0x2e8/0x320
[cfa35f20] [c004429c] kswapd+0xb8/0x400
[cfa35fd0] [c0027ea0] kthread+0x48/0x84
[cfa35ff0] [c000461c] kernel_thread+0x44/0x60
aio/0         S 00000000     0   145      2
Call Trace:
[cfa6de80] [cf89455c] 0xcf89455c (unreliable)
[cfa6df40] [c0009214] __switch_to+0x5c/0x74
[cfa6df60] [c0288644] schedule+0x2e8/0x320
[cfa6df90] [c0023ec0] worker_thread+0x80/0xbc
[cfa6dfd0] [c0027ea0] kthread+0x48/0x84
[cfa6dff0] [c000461c] kernel_thread+0x44/0x60
button contro S 00000000     0   740      2
Call Trace:
[cfb45e80] [cf89451c] 0xcf89451c (unreliable)
[cfb45f40] [c0009214] __switch_to+0x5c/0x74
[cfb45f60] [c0288644] schedule+0x2e8/0x320
[cfb45f90] [c0023ec0] worker_thread+0x80/0xbc
[cfb45fd0] [c0027ea0] kthread+0x48/0x84
[cfb45ff0] [c000461c] kernel_thread+0x44/0x60
link change/0 S 00000000     0   778      2
Call Trace:
[cfb47e80] [cfa104e0] 0xcfa104e0 (unreliable)
[cfb47f40] [c0009214] __switch_to+0x5c/0x74
[cfb47f60] [c0288644] schedule+0x2e8/0x320
[cfb47f90] [c0023ec0] worker_thread+0x80/0xbc
[cfb47fd0] [c0027ea0] kthread+0x48/0x84
[cfb47ff0] [c000461c] kernel_thread+0x44/0x60
scsi_eh_0     S 00000000     0   821      2
Call Trace:
[cfb01e60] [c0002934] ret_from_except+0x0/0x18 (unreliable)
[cfb01f20] [c0009214] __switch_to+0x5c/0x74
[cfb01f40] [c0288644] schedule+0x2e8/0x320
[cfb01f70] [c01741c8] scsi_error_handler+0x4c/0x2d4
[cfb01fd0] [c0027ea0] kthread+0x48/0x84
[cfb01ff0] [c000461c] kernel_thread+0x44/0x60
usb-storage   S 00000000     0   822      2
Call Trace:
[cfb1fe20] [cfb1fed0] 0xcfb1fed0 (unreliable)
[cfb1fee0] [c0009214] __switch_to+0x5c/0x74
[cfb1ff00] [c0288644] schedule+0x2e8/0x320
[cfb1ff30] [c0288244] __down_interruptible+0xb4/0x104
[cfb1ff80] [c01cb230] usb_stor_control_thread+0x34/0x1fc
[cfb1ffd0] [c0027ea0] kthread+0x48/0x84
[cfb1fff0] [c000461c] kernel_thread+0x44/0x60
ksnapd        S 00000000     0   836      2
Call Trace:
[cfb03e80] [cf894c5c] 0xcf894c5c (unreliable)
[cfb03f40] [c0009214] __switch_to+0x5c/0x74
[cfb03f60] [c0288644] schedule+0x2e8/0x320
[cfb03f90] [c0023ec0] worker_thread+0x80/0xbc
[cfb03fd0] [c0027ea0] kthread+0x48/0x84
[cfb03ff0] [c000461c] kernel_thread+0x44/0x60
rpciod/0      S 00000000     0   838      2
Call Trace:
[cfa93e80] [cf89409c] 0xcf89409c (unreliable)
[cfa93f40] [c0009214] __switch_to+0x5c/0x74
[cfa93f60] [c0288644] schedule+0x2e8/0x320
[cfa93f90] [c0023ec0] worker_thread+0x80/0xbc
[cfa93fd0] [c0027ea0] kthread+0x48/0x84
[cfa93ff0] [c000461c] kernel_thread+0x44/0x60
linuxrc       S 0fef0be0     0   841      1
Call Trace:
[cf9fbdc0] [c0038cbc] unlock_page+0x48/0x5c (unreliable)
[cf9fbe80] [c0009214] __switch_to+0x5c/0x74
[cf9fbea0] [c0288644] schedule+0x2e8/0x320
[cf9fbed0] [c0016f24] do_wait+0xcb0/0xe48
[cf9fbf40] [c00022e0] ret_from_syscall+0x0/0x3c
rcS           S 0fef0c78     0   842    841
Call Trace:
[cf8d1dc0] [cf8e47fc] 0xcf8e47fc (unreliable)
[cf8d1e80] [c0009214] __switch_to+0x5c/0x74
[cf8d1ea0] [c0288644] schedule+0x2e8/0x320
[cf8d1ed0] [c0016f24] do_wait+0xcb0/0xe48
[cf8d1f40] [c00022e0] ret_from_syscall+0x0/0x3c
kjournald     S 00000000     0   932      2
Call Trace:
[cf8ebe60] [c0013f68] vprintk+0x2b4/0x2e0 (unreliable)
[cf8ebf20] [c0009214] __switch_to+0x5c/0x74
[cf8ebf40] [c0288644] schedule+0x2e8/0x320
[cf8ebf70] [c00beb54] kjournald+0x144/0x1b8
[cf8ebfd0] [c0027ea0] kthread+0x48/0x84
[cf8ebff0] [c000461c] kernel_thread+0x44/0x60
tar           ? 0fef1674     0   934      1
Call Trace:
[cf8efdf0] [c000f370] try_to_wake_up+0x74/0x9c (unreliable)
[cf8efeb0] [c0009214] __switch_to+0x5c/0x74
[cf8efed0] [c0288644] schedule+0x2e8/0x320
[cf8eff00] [c0015ee0] do_exit+0x6a8/0x6b0
[cf8eff30] [c0015f9c] sys_exit_group+0x0/0x8
[cf8eff40] [c00022e0] ret_from_syscall+0x0/0x3c
rcS2          S 0fef0c78     0   937    842
Call Trace:
[cfb05dc0] [cc5057f8] 0xcc5057f8 (unreliable)
[cfb05e80] [c0009214] __switch_to+0x5c/0x74
[cfb05ea0] [c0288644] schedule+0x2e8/0x320
[cfb05ed0] [c0016f24] do_wait+0xcb0/0xe48
[cfb05f40] [c00022e0] ret_from_syscall+0x0/0x3c
buzzerd       S 0ff6ffa8     0   941    937
Call Trace:
[cf981d70] [cf4e9500] 0xcf4e9500 (unreliable)
[cf981e30] [c0009214] __switch_to+0x5c/0x74
[cf981e50] [c0288644] schedule+0x2e8/0x320
[cf981e80] [c00fa7a8] do_msgrcv+0x27c/0x344
[cf981ee0] [c00fa8a0] sys_msgrcv+0x30/0x7c
[cf981f00] [c0006910] sys_ipc+0x1ac/0x25c
[cf981f40] [c00022e0] ret_from_syscall+0x0/0x3c
kjournald     S 00000000     0   953      2
Call Trace:
[cf983f20] [c0009214] __switch_to+0x5c/0x74
[cf983f40] [c0288644] schedule+0x2e8/0x320
[cf983f70] [c00beb54] kjournald+0x144/0x1b8
[cf983fd0] [c0027ea0] kthread+0x48/0x84
[cf983ff0] [c000461c] kernel_thread+0x44/0x60
scsi_eh_1     S 00000000     0   966      2
Call Trace:
[cd76be60] [c0379918] 0xc0379918 (unreliable)
[cd76bf20] [c0009214] __switch_to+0x5c/0x74
[cd76bf40] [c0288644] schedule+0x2e8/0x320
[cd76bf70] [c01741c8] scsi_error_handler+0x4c/0x2d4
[cd76bfd0] [c0027ea0] kthread+0x48/0x84
[cd76bff0] [c000461c] kernel_thread+0x44/0x60
scsi_eh_2     S 00000000     0   968      2
Call Trace:
[cd773e60] [c0379918] 0xc0379918 (unreliable)
[cd773f20] [c0009214] __switch_to+0x5c/0x74
[cd773f40] [c0288644] schedule+0x2e8/0x320
[cd773f70] [c01741c8] scsi_error_handler+0x4c/0x2d4
[cd773fd0] [c0027ea0] kthread+0x48/0x84
[cd773ff0] [c000461c] kernel_thread+0x44/0x60
scsi_eh_3     S 00000000     0   969      2
Call Trace:
[cd775f20] [c0009214] __switch_to+0x5c/0x74
[cd775f40] [c0288644] schedule+0x2e8/0x320
[cd775f70] [c01741c8] scsi_error_handler+0x4c/0x2d4
[cd775fd0] [c0027ea0] kthread+0x48/0x84
[cd775ff0] [c000461c] kernel_thread+0x44/0x60
scsi_eh_4     S 00000000     0   970      2
Call Trace:
[cf8f1f20] [c0009214] __switch_to+0x5c/0x74
[cf8f1f40] [c0288644] schedule+0x2e8/0x320
[cf8f1f70] [c01741c8] scsi_error_handler+0x4c/0x2d4
[cf8f1fd0] [c0027ea0] kthread+0x48/0x84
[cf8f1ff0] [c000461c] kernel_thread+0x44/0x60
xfslogd/0     S 00000000     0  1111      2
Call Trace:
[cc4cbe80] [00021000] 0x21000 (unreliable)
[cc4cbf40] [c0009214] __switch_to+0x5c/0x74
[cc4cbf60] [c0288644] schedule+0x2e8/0x320
[cc4cbf90] [c0023ec0] worker_thread+0x80/0xbc
[cc4cbfd0] [c0027ea0] kthread+0x48/0x84
[cc4cbff0] [c000461c] kernel_thread+0x44/0x60
xfsdatad/0    S 00000000     0  1113      2
Call Trace:
[cc463f40] [c0009214] __switch_to+0x5c/0x74
[cc463f60] [c0288644] schedule+0x2e8/0x320
[cc463f90] [c0023ec0] worker_thread+0x80/0xbc
[cc463fd0] [c0027ea0] kthread+0x48/0x84
[cc463ff0] [c000461c] kernel_thread+0x44/0x60
xfs_mru_cache S 00000000     0  1123      2
Call Trace:
[cc50be80] [7c0b4839] 0x7c0b4839 (unreliable)
[cc50bf40] [c0009214] __switch_to+0x5c/0x74
[cc50bf60] [c0288644] schedule+0x2e8/0x320
[cc50bf90] [c0023ec0] worker_thread+0x80/0xbc
[cc50bfd0] [c0027ea0] kthread+0x48/0x84
[cc50bff0] [c000461c] kernel_thread+0x44/0x60
cifsoplockd   S 00000000     0  1168      2
Call Trace:
[cc525e40] [ffffffff] 0xffffffff (unreliable)
[cc525f00] [c0009214] __switch_to+0x5c/0x74
[cc525f20] [c0288644] schedule+0x2e8/0x320
[cc525f50] [c0288b38] schedule_timeout+0x90/0xc0
[cc525f90] [d1bc5b58] cifs_oplock_thread+0x1a8/0x1cc [cifs]
[cc525fd0] [c0027ea0] kthread+0x48/0x84
[cc525ff0] [c000461c] kernel_thread+0x44/0x60
cifsdnotifyd  S 00000000     0  1170      2
Call Trace:
[cc44be70] [00021000] 0x21000 (unreliable)
[cc44bf30] [c0009214] __switch_to+0x5c/0x74
[cc44bf50] [c0288644] schedule+0x2e8/0x320
[cc44bf80] [c0288b38] schedule_timeout+0x90/0xc0
[cc44bfc0] [d1bc5ba4] cifs_dnotify_thread+0x28/0xa0 [cifs]
[cc44bfd0] [c0027ea0] kthread+0x48/0x84
[cc44bff0] [c000461c] kernel_thread+0x44/0x60
zylogd        S 0fec1008     0  1201      1
Call Trace:
[cc433d70] [cc433dc8] 0xcc433dc8 (unreliable)
[cc433e30] [c0009214] __switch_to+0x5c/0x74
[cc433e50] [c0288644] schedule+0x2e8/0x320
[cc433e80] [c00fa7a8] do_msgrcv+0x27c/0x344
[cc433ee0] [c00fa8a0] sys_msgrcv+0x30/0x7c
[cc433f00] [c0006910] sys_ipc+0x1ac/0x25c
[cc433f40] [c00022e0] ret_from_syscall+0x0/0x3c
syslog-ng     R running      0  1205      1
zylogger      ? 0fe8f674     0  1207      1
Call Trace:
[cc44fdf0] [c000f370] try_to_wake_up+0x74/0x9c (unreliable)
[cc44feb0] [c0009214] __switch_to+0x5c/0x74
[cc44fed0] [c0288644] schedule+0x2e8/0x320
[cc44ff00] [c0015ee0] do_exit+0x6a8/0x6b0
[cc44ff30] [c0015f9c] sys_exit_group+0x0/0x8
[cc44ff40] [c00022e0] ret_from_syscall+0x0/0x3c
uamd          S 0fc379f0     0  1212      1
Call Trace:
[cc539980] [fffbb448] 0xfffbb448 (unreliable)
[cc539a40] [c0009214] __switch_to+0x5c/0x74
[cc539a60] [c0288644] schedule+0x2e8/0x320
[cc539a90] [c0288b38] schedule_timeout+0x90/0xc0
[cc539ad0] [c006a40c] do_select+0x38c/0x3e4
[cc539d90] [c006a710] core_sys_select+0x2ac/0x360
[cc539ef0] [c006a8e0] sys_select+0x11c/0x234
[cc539f30] [c0006c40] ppc_select+0x8c/0x9c
[cc539f40] [c00022e0] ret_from_syscall+0x0/0x3c
cupsd         S 0fcff1a8     0  1225      1
Call Trace:
[cc42bd80] [cc506f00] 0xcc506f00 (unreliable)
[cc42be40] [c0009214] __switch_to+0x5c/0x74
[cc42be60] [c0288644] schedule+0x2e8/0x320
[cc42be90] [c0288ad4] schedule_timeout+0x2c/0xc0
[cc42bed0] [c008c28c] sys_epoll_wait+0x1c4/0x490
[cc42bf40] [c00022e0] ret_from_syscall+0x0/0x3c
zysh.sh       S 0fef0c78     0  1227    937
Call Trace:
[cc5f5dc0] [cc50c7f8] 0xcc50c7f8 (unreliable)
[cc5f5e80] [c0009214] __switch_to+0x5c/0x74
[cc5f5ea0] [c0288644] schedule+0x2e8/0x320
[cc5f5ed0] [c0016f24] do_wait+0xcb0/0xe48
[cc5f5f40] [c00022e0] ret_from_syscall+0x0/0x3c
zyshd         S 0f7d1c1c     0  1231   1227
Call Trace:
[cc42ddc0] [ca9217fc] 0xca9217fc (unreliable)
[cc42de80] [c0009214] __switch_to+0x5c/0x74
[cc42dea0] [c0288644] schedule+0x2e8/0x320
[cc42ded0] [c0016f24] do_wait+0xcb0/0xe48
[cc42df40] [c00022e0] ret_from_syscall+0x0/0x3c
sh            S 0fef0c78     0  1272   1205
Call Trace:
[cc5cddc0] [c0038cbc] unlock_page+0x48/0x5c (unreliable)
[cc5cde80] [c0009214] __switch_to+0x5c/0x74
[cc5cdea0] [c0288644] schedule+0x2e8/0x320
[cc5cded0] [c0016f24] do_wait+0xcb0/0xe48
[cc5cdf40] [c00022e0] ret_from_syscall+0x0/0x3c
zylogger      S 0feb0e88     0  1273   1272
Call Trace:
[cc627c80] [c0038cbc] unlock_page+0x48/0x5c (unreliable)
[cc627d40] [c0009214] __switch_to+0x5c/0x74
[cc627d60] [c0288644] schedule+0x2e8/0x320
[cc627d90] [c00625c0] pipe_wait+0x6c/0xa0
[cc627dd0] [c0062c88] pipe_read+0x380/0x3fc
[cc627e30] [c005c458] do_sync_read+0xb8/0x10c
[cc627ef0] [c005c574] vfs_read+0xc8/0x15c
[cc627f10] [c005c8bc] sys_read+0x4c/0x8c
[cc627f40] [c00022e0] ret_from_syscall+0x0/0x3c
sh            S 0fef0c78     0  1276   1205
Call Trace:
[cc635dc0] [c0038cbc] unlock_page+0x48/0x5c (unreliable)
[cc635e80] [c0009214] __switch_to+0x5c/0x74
[cc635ea0] [c0288644] schedule+0x2e8/0x320
[cc635ed0] [c0016f24] do_wait+0xcb0/0xe48
[cc635f40] [c00022e0] ret_from_syscall+0x0/0x3c
sh            S 0fef0c78     0  1277   1205
Call Trace:
[cc637dc0] [c0038cbc] unlock_page+0x48/0x5c (unreliable)
[cc637e80] [c0009214] __switch_to+0x5c/0x74
[cc637ea0] [c0288644] schedule+0x2e8/0x320
[cc637ed0] [c0016f24] do_wait+0xcb0/0xe48
[cc637f40] [c00022e0] ret_from_syscall+0x0/0x3c
zylogger      S 0feb0e88     0  1278   1276
Call Trace:
[cc513c80] [c0038cbc] unlock_page+0x48/0x5c (unreliable)
[cc513d40] [c0009214] __switch_to+0x5c/0x74
[cc513d60] [c0288644] schedule+0x2e8/0x320
[cc513d90] [c00625c0] pipe_wait+0x6c/0xa0
[cc513dd0] [c0062c88] pipe_read+0x380/0x3fc
[cc513e30] [c005c458] do_sync_read+0xb8/0x10c
[cc513ef0] [c005c574] vfs_read+0xc8/0x15c
[cc513f10] [c005c8bc] sys_read+0x4c/0x8c
[cc513f40] [c00022e0] ret_from_syscall+0x0/0x3c
zylogger      S 0feb0ec4     0  1279   1277
Call Trace:
[cc653c80] [00001c09] 0x1c09 (unreliable)
[cc653d40] [c0009214] __switch_to+0x5c/0x74
[cc653d60] [c0288644] schedule+0x2e8/0x320
[cc653d90] [c00625c0] pipe_wait+0x6c/0xa0
[cc653dd0] [c0062c88] pipe_read+0x380/0x3fc
[cc653e30] [c005c458] do_sync_read+0xb8/0x10c
[cc653ef0] [c005c574] vfs_read+0xc8/0x15c
[cc653f10] [c005c8bc] sys_read+0x4c/0x8c
[cc653f40] [c00022e0] ret_from_syscall+0x0/0x3c
zyshclient    S 0f7f90bc     0  1284   1227
Call Trace:
[cc633a30] [c0080b48] __find_get_block+0x204/0x21c (unreliable)
[cc633af0] [c0009214] __switch_to+0x5c/0x74
[cc633b10] [c0288644] schedule+0x2e8/0x320
[cc633b40] [c0288ad4] schedule_timeout+0x2c/0xc0
[cc633b80] [c006b058] do_sys_poll+0x2d4/0x3a8
[cc633f20] [c006b208] sys_poll+0x58/0xa4
[cc633f40] [c00022e0] ret_from_syscall+0x0/0x3c
md0_raid1     S 00000000     0  1324      2
Call Trace:
[cc667e40] [0000000a] 0xa (unreliable)
[cc667f00] [c0009214] __switch_to+0x5c/0x74
[cc667f20] [c0288644] schedule+0x2e8/0x320
[cc667f50] [c0288ad4] schedule_timeout+0x2c/0xc0
[cc667f90] [c01f7714] md_thread+0xc8/0x128
[cc667fd0] [c0027ea0] kthread+0x48/0x84
[cc667ff0] [c000461c] kernel_thread+0x44/0x60
ksnaphd       S 00000000     0  1383      2
Call Trace:
[cc79be80] [c0379948] 0xc0379948 (unreliable)
[cc79bf40] [c0009214] __switch_to+0x5c/0x74
[cc79bf60] [c0288644] schedule+0x2e8/0x320
[cc79bf90] [c0023ec0] worker_thread+0x80/0xbc
[cc79bfd0] [c0027ea0] kthread+0x48/0x84
[cc79bff0] [c000461c] kernel_thread+0x44/0x60
kcopyd        S 00000000     0  1385      2
Call Trace:
[cc7b7e80] [4804d8c9] 0x4804d8c9 (unreliable)
[cc7b7f40] [c0009214] __switch_to+0x5c/0x74
[cc7b7f60] [c0288644] schedule+0x2e8/0x320
[cc7b7f90] [c0023ec0] worker_thread+0x80/0xbc
[cc7b7fd0] [c0027ea0] kthread+0x48/0x84
[cc7b7ff0] [c000461c] kernel_thread+0x44/0x60
sh            ? 0fef1674     0  1390   1231
Call Trace:
[cc6e7df0] [c003ee68] free_hot_cold_page+0x164/0x18c (unreliable)
[cc6e7eb0] [c0009214] __switch_to+0x5c/0x74
[cc6e7ed0] [c0288644] schedule+0x2e8/0x320
[cc6e7f00] [c0015ee0] do_exit+0x6a8/0x6b0
[cc6e7f30] [c0015f9c] sys_exit_group+0x0/0x8
[cc6e7f40] [c00022e0] ret_from_syscall+0x0/0x3c
sh            S 0fef0c78     0  1430   1231
Call Trace:
[ca935dc0] [c0038cbc] unlock_page+0x48/0x5c (unreliable)
[ca935e80] [c0009214] __switch_to+0x5c/0x74
[ca935ea0] [c0288644] schedule+0x2e8/0x320
[ca935ed0] [c0016f24] do_wait+0xcb0/0xe48
[ca935f40] [c00022e0] ret_from_syscall+0x0/0x3c
mount         D 0ff6f1e8     0  1431   1430
Call Trace:
[ca9377b0] [c0084710] __bio_clone+0x90/0xbc (unreliable)
[ca937870] [c0009214] __switch_to+0x5c/0x74
[ca937890] [c0288644] schedule+0x2e8/0x320
[ca9378c0] [c01179a8] generic_make_request+0x184/0x3a4
[ca937930] [c0117cb0] submit_bio+0xe8/0xfc
[ca937970] [d1b9efa4] _xfs_buf_ioapply+0x2bc/0x300 [xfs]
[ca9379b0] [d1b9f040] xfs_buf_iorequest+0x58/0xa0 [xfs]
[ca9379c0] [d1ba5b5c] xfsbdstrat+0x30/0x50 [xfs]
[ca9379d0] [d1b859f4] xlog_bread+0xa8/0xec [xfs]
[ca9379f0] [d1b85f5c] xlog_find_verify_cycle+0x104/0x1e0 [xfs]
[ca937a40] [d1b86498] xlog_find_zeroed+0x200/0x29c [xfs]
[ca937a80] [d1b86558] xlog_find_head+0x24/0x458 [xfs]
[ca937ac0] [d1b86de4] xlog_find_tail+0x20/0x408 [xfs]
[ca937b00] [d1b8a328] xlog_recover+0x20/0xd0 [xfs]
[ca937b20] [d1b84020] xfs_log_mount+0xc8/0x144 [xfs]
[ca937b40] [d1b8c76c] xfs_mountfs+0x78c/0xb34 [xfs]
[ca937bf0] [d1b7eb50] xfs_ioinit+0x14/0x24 [xfs]
[ca937c00] [d1b93544] xfs_mount+0x2b4/0x370 [xfs]
[ca937c30] [d1ba6cb8] xfs_fs_fill_super+0xcc/0x244 [xfs]
[ca937ce0] [c005ebc4] get_sb_bdev+0x12c/0x188
[ca937d30] [d1ba6e4c] xfs_fs_get_sb+0x1c/0x2c [xfs]
[ca937d40] [c005ee54] vfs_kern_mount+0x5c/0xb4
[ca937d60] [c005eeec] do_kern_mount+0x40/0x10c
[ca937d80] [c007594c] do_mount+0x5b8/0x614
[ca937f10] [c0075c80] sys_mount+0x98/0xe8
[ca937f40] [c00022e0] ret_from_syscall+0x0/0x3c
xfsbufd       S 00000000     0  1432      2
Call Trace:
[ca92be60] [c0379918] 0xc0379918 (unreliable)
[ca92bf20] [c0009214] __switch_to+0x5c/0x74
[ca92bf40] [c0288644] schedule+0x2e8/0x320
[ca92bf70] [c0288b38] schedule_timeout+0x90/0xc0
[ca92bfb0] [d1ba0428] xfsbufd+0x5c/0x140 [xfs]
[ca92bfd0] [c0027ea0] kthread+0x48/0x84
[ca92bff0] [c000461c] kernel_thread+0x44/0x60

When I used printk to see the "need" in __generic_make_request, I found 
the throttle capacity were consumed completely in this function.

Original comment by ej3Georg...@gmail.com on 26 May 2008 at 12:38

GoogleCodeExporter commented 9 years ago 
Aha, so you applied the zumastor patch to the kernel,
then happened to create an lvm2 snapshot?  i.e. you
think the hang is a bug introduced by the zumastor
patch to the rest of the kernel, even when zumastor
is not in use?

Original comment by daniel.r...@gmail.com on 26 May 2008 at 2:45

GoogleCodeExporter commented 9 years ago 
I am not sure.  However, when I remove the zumastor's patch code, the mount 
process
would not hang.

Original comment by ej3Georg...@gmail.com on 27 May 2008 at 1:53

GoogleCodeExporter commented 9 years ago 
Thanks for the info.  Can you give us a recipe for reproducing the hang?

Original comment by daniel.r...@gmail.com on 27 May 2008 at 1:58

GoogleCodeExporter commented 9 years ago 
There is another hanged issue.  I used lvm2 to create volume on RAID-1(also 
created
by lvm2), and then created snapshot for the volume.  When I copied data to the 
volume
and filled up the snapshot, the system was hanged.  
This is the trace show: 
  task                PC stack   pid father
linuxrc       S 0fef0b34     0     1      0
Call Trace:
[cf813dc0] [100ae9c4] 0x100ae9c4 (unreliable)
[cf813e80] [c0009214] __switch_to+0x5c/0x74
[cf813ea0] [c02eeca4] schedule+0x2e8/0x320
[cf813ed0] [c0016f24] do_wait+0xcb0/0xe48
[cf813f40] [c00022e0] ret_from_syscall+0x0/0x3c
kthreadd      S 00000000     0     2      0
Call Trace:
[cf823ec0] [00021000] 0x21000 (unreliable)
[cf823f80] [c0009214] __switch_to+0x5c/0x74
[cf823fa0] [c02eeca4] schedule+0x2e8/0x320
[cf823fd0] [c0028104] kthreadd+0x74/0xfc
[cf823ff0] [c000461c] kernel_thread+0x44/0x60
ksoftirqd/0   S 00000000     0     3      2
Call Trace:
[cf827ea0] [00029000] 0x29000 (unreliable)
[cf827f60] [c0009214] __switch_to+0x5c/0x74
[cf827f80] [c02eeca4] schedule+0x2e8/0x320
[cf827fb0] [c0018c98] ksoftirqd+0x38/0xcc
[cf827fd0] [c0027ea0] kthread+0x48/0x84
[cf827ff0] [c000461c] kernel_thread+0x44/0x60
events/0      S 00000000     0     4      2
Call Trace:
[cf829e80] [00029000] 0x29000 (unreliable)
[cf829f40] [c0009214] __switch_to+0x5c/0x74
[cf829f60] [c02eeca4] schedule+0x2e8/0x320
[cf829f90] [c0023ec0] worker_thread+0x80/0xbc
[cf829fd0] [c0027ea0] kthread+0x48/0x84
[cf829ff0] [c000461c] kernel_thread+0x44/0x60
khelper       S 00000000     0     5      2
Call Trace:
[cf82be80] [c000e370] __enqueue_entity+0xb4/0xc8 (unreliable)
[cf82bf40] [c0009214] __switch_to+0x5c/0x74
[cf82bf60] [c02eeca4] schedule+0x2e8/0x320
[cf82bf90] [c0023ec0] worker_thread+0x80/0xbc
[cf82bfd0] [c0027ea0] kthread+0x48/0x84
[cf82bff0] [c000461c] kernel_thread+0x44/0x60
scsi_to_zylog S 00000000     0    60      2
Call Trace:
[cf8bde80] [ffffffff] 0xffffffff (unreliable)
[cf8bdf40] [c0009214] __switch_to+0x5c/0x74
[cf8bdf60] [c02eeca4] schedule+0x2e8/0x320
[cf8bdf90] [c0023ec0] worker_thread+0x80/0xbc
[cf8bdfd0] [c0027ea0] kthread+0x48/0x84
[cf8bdff0] [c000461c] kernel_thread+0x44/0x60
kblockd/0     S 00000000     0    61      2
Call Trace:
[cf8bfe80] [00800000] 0x800000 (unreliable)
[cf8bff40] [c0009214] __switch_to+0x5c/0x74
[cf8bff60] [c02eeca4] schedule+0x2e8/0x320
[cf8bff90] [c0023ec0] worker_thread+0x80/0xbc
[cf8bffd0] [c0027ea0] kthread+0x48/0x84
[cf8bfff0] [c000461c] kernel_thread+0x44/0x60
ata/0         S 00000000     0    68      2
Call Trace:
[cf83de80] [cf8142dc] 0xcf8142dc (unreliable)
[cf83df40] [c0009214] __switch_to+0x5c/0x74
[cf83df60] [c02eeca4] schedule+0x2e8/0x320
[cf83df90] [c0023ec0] worker_thread+0x80/0xbc
[cf83dfd0] [c0027ea0] kthread+0x48/0x84
[cf83dff0] [c000461c] kernel_thread+0x44/0x60
ata_aux       S 00000000     0    69      2
Call Trace:
[cf83fe80] [cf81431c] 0xcf81431c (unreliable)
[cf83ff40] [c0009214] __switch_to+0x5c/0x74
[cf83ff60] [c02eeca4] schedule+0x2e8/0x320
[cf83ff90] [c0023ec0] worker_thread+0x80/0xbc
[cf83ffd0] [c0027ea0] kthread+0x48/0x84
[cf83fff0] [c000461c] kernel_thread+0x44/0x60
khubd         S 00000000     0    74      2
Call Trace:
[cf8d3e40] [c000fbb4] __wake_up+0x24/0x3c (unreliable)
[cf8d3f00] [c0009214] __switch_to+0x5c/0x74
[cf8d3f20] [c02eeca4] schedule+0x2e8/0x320
[cf8d3f50] [c0218d8c] hub_thread+0xa84/0xacc
[cf8d3fd0] [c0027ea0] kthread+0x48/0x84
[cf8d3ff0] [c000461c] kernel_thread+0x44/0x60
pdflush       D 00000000     0    98      2
Call Trace:
[cfa31ac0] [c0084710] __bio_clone+0x90/0xbc (unreliable)
[cfa31b80] [c0009214] __switch_to+0x5c/0x74
[cfa31ba0] [c02eeca4] schedule+0x2e8/0x320
[cfa31bd0] [c017e004] generic_make_request+0x184/0x3a4
[cfa31c40] [c017e30c] submit_bio+0xe8/0xfc
[cfa31c80] [c0153a08] xfs_submit_ioend_bio+0x40/0x5c
[cfa31ca0] [c0153c3c] xfs_submit_ioend+0xcc/0x124
[cfa31cc0] [c0154df8] xfs_page_state_convert+0x684/0x6e8
[cfa31d70] [c0154f2c] xfs_vm_writepage+0xd0/0x128
[cfa31da0] [c003fafc] __writepage+0x24/0x68
[cfa31db0] [c0040360] write_cache_pages+0x1ec/0x33c
[cfa31e50] [c0154fa8] xfs_vm_writepages+0x24/0x34
[cfa31e60] [c0040520] do_writepages+0x48/0x78
[cfa31e70] [c007bd28] __writeback_single_inode+0x12c/0x29c
[cfa31ec0] [c007c2b8] sync_sb_inodes+0x1e8/0x284
[cfa31ef0] [c007c594] writeback_inodes+0x70/0xb4
[cfa31f10] [c003f780] background_writeout+0x8c/0xd8
[cfa31f70] [c0040a2c] pdflush+0x108/0x1b8
[cfa31fd0] [c0027ea0] kthread+0x48/0x84
[cfa31ff0] [c000461c] kernel_thread+0x44/0x60
pdflush       D 00000000     0    99      2
Call Trace:
[cfa33cd0] [ca7a0720] 0xca7a0720 (unreliable)
[cfa33d90] [c0009214] __switch_to+0x5c/0x74
[cfa33db0] [c02eeca4] schedule+0x2e8/0x320
[cfa33de0] [c02ef134] schedule_timeout+0x2c/0xc0
[cfa33e20] [c00fa31c] xfs_qm_dqunpin_wait+0x8c/0xac
[cfa33e60] [c00f952c] xfs_qm_dqflush+0x54/0x23c
[cfa33e80] [c00fda5c] xfs_qm_sync+0x11c/0x1bc
[cfa33eb0] [c014bcc8] xfs_sync+0x28/0x94
[cfa33ed0] [c015dfb8] xfs_fs_write_super+0x2c/0x48
[cfa33ee0] [c005e320] sync_supers+0x74/0xd0
[cfa33f00] [c003f854] wb_kupdate+0x44/0x12c
[cfa33f70] [c0040a2c] pdflush+0x108/0x1b8
[cfa33fd0] [c0027ea0] kthread+0x48/0x84
[cfa33ff0] [c000461c] kernel_thread+0x44/0x60
kswapd0       S 00000000     0   100      2
Call Trace:
[cfa35e10] [cf8c7590] 0xcf8c7590 (unreliable)
[cfa35ed0] [c0009214] __switch_to+0x5c/0x74
[cfa35ef0] [c02eeca4] schedule+0x2e8/0x320
[cfa35f20] [c004429c] kswapd+0xb8/0x400
[cfa35fd0] [c0027ea0] kthread+0x48/0x84
[cfa35ff0] [c000461c] kernel_thread+0x44/0x60
aio/0         S 00000000     0   145      2
Call Trace:
[cfa2be80] [cf89499c] 0xcf89499c (unreliable)
[cfa2bf40] [c0009214] __switch_to+0x5c/0x74
[cfa2bf60] [c02eeca4] schedule+0x2e8/0x320
[cfa2bf90] [c0023ec0] worker_thread+0x80/0xbc
[cfa2bfd0] [c0027ea0] kthread+0x48/0x84
[cfa2bff0] [c000461c] kernel_thread+0x44/0x60
xfslogd/0     S 00000000     0   159      2
Call Trace:
[cfacdf40] [c0009214] __switch_to+0x5c/0x74
[cfacdf60] [c02eeca4] schedule+0x2e8/0x320
[cfacdf90] [c0023ec0] worker_thread+0x80/0xbc
[cfacdfd0] [c0027ea0] kthread+0x48/0x84
[cfacdff0] [c000461c] kernel_thread+0x44/0x60
xfsdatad/0    S 00000000     0   160      2
Call Trace:
[cfacfe80] [ca4c5b60] 0xca4c5b60 (unreliable)
[cfacff40] [c0009214] __switch_to+0x5c/0x74
[cfacff60] [c02eeca4] schedule+0x2e8/0x320
[cfacff90] [c0023ec0] worker_thread+0x80/0xbc
[cfacffd0] [c0027ea0] kthread+0x48/0x84
[cfacfff0] [c000461c] kernel_thread+0x44/0x60
xfs_mru_cache S 00000000     0   164      2
Call Trace:
[cfa89e80] [cf894c1c] 0xcf894c1c (unreliable)
[cfa89f40] [c0009214] __switch_to+0x5c/0x74
[cfa89f60] [c02eeca4] schedule+0x2e8/0x320
[cfa89f90] [c0023ec0] worker_thread+0x80/0xbc
[cfa89fd0] [c0027ea0] kthread+0x48/0x84
[cfa89ff0] [c000461c] kernel_thread+0x44/0x60
button contro S 00000000     0   751      2
Call Trace:
[cfa43e80] [cf89479c] 0xcf89479c (unreliable)
[cfa43f40] [c0009214] __switch_to+0x5c/0x74
[cfa43f60] [c02eeca4] schedule+0x2e8/0x320
[cfa43f90] [c0023ec0] worker_thread+0x80/0xbc
[cfa43fd0] [c0027ea0] kthread+0x48/0x84
[cfa43ff0] [c000461c] kernel_thread+0x44/0x60
link change/0 S 00000000     0   789      2
Call Trace:
[cfa29e80] [cfa104e0] 0xcfa104e0 (unreliable)
[cfa29f40] [c0009214] __switch_to+0x5c/0x74
[cfa29f60] [c02eeca4] schedule+0x2e8/0x320
[cfa29f90] [c0023ec0] worker_thread+0x80/0xbc
[cfa29fd0] [c0027ea0] kthread+0x48/0x84
[cfa29ff0] [c000461c] kernel_thread+0x44/0x60
scsi_eh_0     S 00000000     0   832      2
Call Trace:
[cfb51e60] [c03b0000] 0xc03b0000 (unreliable)
[cfb51f20] [c0009214] __switch_to+0x5c/0x74
[cfb51f40] [c02eeca4] schedule+0x2e8/0x320
[cfb51f70] [c01da824] scsi_error_handler+0x4c/0x2d4
[cfb51fd0] [c0027ea0] kthread+0x48/0x84
[cfb51ff0] [c000461c] kernel_thread+0x44/0x60
usb-storage   S 00000000     0   833      2
Call Trace:
[cfb53e20] [c000d018] openpic_get_irq+0x14/0x78 (unreliable)
[cfb53ee0] [c0009214] __switch_to+0x5c/0x74
[cfb53f00] [c02eeca4] schedule+0x2e8/0x320
[cfb53f30] [c02ee8a4] __down_interruptible+0xb4/0x104
[cfb53f80] [c023188c] usb_stor_control_thread+0x34/0x1fc
[cfb53fd0] [c0027ea0] kthread+0x48/0x84
[cfb53ff0] [c000461c] kernel_thread+0x44/0x60
ksnapd        S 00000000     0   847      2
Call Trace:
[cfb57e80] [cf894b5c] 0xcf894b5c (unreliable)
[cfb57f40] [c0009214] __switch_to+0x5c/0x74
[cfb57f60] [c02eeca4] schedule+0x2e8/0x320
[cfb57f90] [c0023ec0] worker_thread+0x80/0xbc
[cfb57fd0] [c0027ea0] kthread+0x48/0x84
[cfb57ff0] [c000461c] kernel_thread+0x44/0x60
rpciod/0      S 00000000     0   849      2
Call Trace:
[cfb5df40] [c0009214] __switch_to+0x5c/0x74
[cfb5df60] [c02eeca4] schedule+0x2e8/0x320
[cfb5df90] [c0023ec0] worker_thread+0x80/0xbc
[cfb5dfd0] [c0027ea0] kthread+0x48/0x84
[cfb5dff0] [c000461c] kernel_thread+0x44/0x60
kjournald     S 00000000     0   947      2
Call Trace:
[cf9d7e60] [c0013f68] vprintk+0x2b4/0x2e0 (unreliable)
[cf9d7f20] [c0009214] __switch_to+0x5c/0x74
[cf9d7f40] [c02eeca4] schedule+0x2e8/0x320
[cf9d7f70] [c00beb54] kjournald+0x144/0x1b8
[cf9d7fd0] [c0027ea0] kthread+0x48/0x84
[cf9d7ff0] [c000461c] kernel_thread+0x44/0x60
buzzerd       S 0ff6ffa8     0   956      1
Call Trace:
[cf9dfd70] [cf4e9500] 0xcf4e9500 (unreliable)
[cf9dfe30] [c0009214] __switch_to+0x5c/0x74
[cf9dfe50] [c02eeca4] schedule+0x2e8/0x320
[cf9dfe80] [c0160e04] do_msgrcv+0x27c/0x344
[cf9dfee0] [c0160efc] sys_msgrcv+0x30/0x7c
[cf9dff00] [c0006910] sys_ipc+0x1ac/0x25c
[cf9dff40] [c00022e0] ret_from_syscall+0x0/0x3c
jffs2_gcd_mtd S 00000000     0   968      2
Call Trace:
[cf8e7e40] [c00ec39c] jffs2_free_node_frag+0x1c/0x2c (unreliable)
[cf8e7f00] [c0009214] __switch_to+0x5c/0x74
[cf8e7f20] [c02eeca4] schedule+0x2e8/0x320
[cf8e7f50] [c00f4aac] jffs2_garbage_collect_thread+0x84/0x118
[cf8e7ff0] [c000461c] kernel_thread+0x44/0x60
scsi_eh_1     S 00000000     0   981      2
Call Trace:
[cf847e60] [f5e41885] 0xf5e41885 (unreliable)
[cf847f20] [c0009214] __switch_to+0x5c/0x74
[cf847f40] [c02eeca4] schedule+0x2e8/0x320
[cf847f70] [c01da824] scsi_error_handler+0x4c/0x2d4
[cf847fd0] [c0027ea0] kthread+0x48/0x84
[cf847ff0] [c000461c] kernel_thread+0x44/0x60
scsi_eh_2     S 00000000     0   983      2
Call Trace:
[cf84be60] [54612398] 0x54612398 (unreliable)
[cf84bf20] [c0009214] __switch_to+0x5c/0x74
[cf84bf40] [c02eeca4] schedule+0x2e8/0x320
[cf84bf70] [c01da824] scsi_error_handler+0x4c/0x2d4
[cf84bfd0] [c0027ea0] kthread+0x48/0x84
[cf84bff0] [c000461c] kernel_thread+0x44/0x60
scsi_eh_3     S 00000000     0   984      2
Call Trace:
[cf84de60] [caa38a48] 0xcaa38a48 (unreliable)
[cf84df20] [c0009214] __switch_to+0x5c/0x74
[cf84df40] [c02eeca4] schedule+0x2e8/0x320
[cf84df70] [c01da824] scsi_error_handler+0x4c/0x2d4
[cf84dfd0] [c0027ea0] kthread+0x48/0x84
[cf84dff0] [c000461c] kernel_thread+0x44/0x60
scsi_eh_4     S 00000000     0   985      2
Call Trace:
[cf8eff20] [c0009214] __switch_to+0x5c/0x74
[cf8eff40] [c02eeca4] schedule+0x2e8/0x320
[cf8eff70] [c01da824] scsi_error_handler+0x4c/0x2d4
[cf8effd0] [c0027ea0] kthread+0x48/0x84
[cf8efff0] [c000461c] kernel_thread+0x44/0x60
cifsoplockd   S 00000000     0  1153      2
Call Trace:
[cc5d3f00] [c0009214] __switch_to+0x5c/0x74
[cc5d3f20] [c02eeca4] schedule+0x2e8/0x320
[cc5d3f50] [c02ef198] schedule_timeout+0x90/0xc0
[cc5d3f90] [d1b1bb58] cifs_oplock_thread+0x1a8/0x1cc [cifs]
[cc5d3fd0] [c0027ea0] kthread+0x48/0x84
[cc5d3ff0] [c000461c] kernel_thread+0x44/0x60
cifsdnotifyd  S 00000000     0  1155      2
Call Trace:
[cc5dff30] [c0009214] __switch_to+0x5c/0x74
[cc5dff50] [c02eeca4] schedule+0x2e8/0x320
[cc5dff80] [c02ef198] schedule_timeout+0x90/0xc0
[cc5dffc0] [d1b1bba4] cifs_dnotify_thread+0x28/0xa0 [cifs]
[cc5dffd0] [c0027ea0] kthread+0x48/0x84
[cc5dfff0] [c000461c] kernel_thread+0x44/0x60
zylogd        S 0fec1008     0  1185      1
Call Trace:
[cc409d70] [cc409dc8] 0xcc409dc8 (unreliable)
[cc409e30] [c0009214] __switch_to+0x5c/0x74
[cc409e50] [c02eeca4] schedule+0x2e8/0x320
[cc409e80] [c0160e04] do_msgrcv+0x27c/0x344
[cc409ee0] [c0160efc] sys_msgrcv+0x30/0x7c
[cc409f00] [c0006910] sys_ipc+0x1ac/0x25c
[cc409f40] [c00022e0] ret_from_syscall+0x0/0x3c
syslog-ng     R running      0  1189      1
uamd          S 0fc379f0     0  1196      1
Call Trace:
[cc477980] [c0018774] __do_softirq+0x5c/0xc0 (unreliable)
[cc477a40] [c0009214] __switch_to+0x5c/0x74
[cc477a60] [c02eeca4] schedule+0x2e8/0x320
[cc477a90] [c02ef198] schedule_timeout+0x90/0xc0
[cc477ad0] [c006a40c] do_select+0x38c/0x3e4
[cc477d90] [c006a710] core_sys_select+0x2ac/0x360
[cc477ef0] [c006a8e0] sys_select+0x11c/0x234
[cc477f30] [c0006c40] ppc_select+0x8c/0x9c
[cc477f40] [c00022e0] ret_from_syscall+0x0/0x3c
cupsd         S 0fcff1a8     0  1209      1
Call Trace:
[cc447d80] [cc4db780] 0xcc4db780 (unreliable)
[cc447e40] [c0009214] __switch_to+0x5c/0x74
[cc447e60] [c02eeca4] schedule+0x2e8/0x320
[cc447e90] [c02ef134] schedule_timeout+0x2c/0xc0
[cc447ed0] [c008c28c] sys_epoll_wait+0x1c4/0x490
[cc447f40] [c00022e0] ret_from_syscall+0x0/0x3c
zyshd         S 0f804008     0  1215      1
Call Trace:
[cc51dd70] [c0080b48] __find_get_block+0x204/0x21c (unreliable)
[cc51de30] [c0009214] __switch_to+0x5c/0x74
[cc51de50] [c02eeca4] schedule+0x2e8/0x320
[cc51de80] [c0160e04] do_msgrcv+0x27c/0x344
[cc51dee0] [c0160efc] sys_msgrcv+0x30/0x7c
[cc51df00] [c0006910] sys_ipc+0x1ac/0x25c
[cc51df40] [c00022e0] ret_from_syscall+0x0/0x3c
sh            S 0fef0c78     0  1256   1189
Call Trace:
[cc533dc0] [c0038cbc] unlock_page+0x48/0x5c (unreliable)
[cc533e80] [c0009214] __switch_to+0x5c/0x74
[cc533ea0] [c02eeca4] schedule+0x2e8/0x320
[cc533ed0] [c0016f24] do_wait+0xcb0/0xe48
[cc533f40] [c00022e0] ret_from_syscall+0x0/0x3c
zylogger      S 0feb0e88     0  1257   1256
Call Trace:
[cc5c1c80] [c0018774] __do_softirq+0x5c/0xc0 (unreliable)
[cc5c1d40] [c0009214] __switch_to+0x5c/0x74
[cc5c1d60] [c02eeca4] schedule+0x2e8/0x320
[cc5c1d90] [c00625c0] pipe_wait+0x6c/0xa0
[cc5c1dd0] [c0062c88] pipe_read+0x380/0x3fc
[cc5c1e30] [c005c458] do_sync_read+0xb8/0x10c
[cc5c1ef0] [c005c574] vfs_read+0xc8/0x15c
[cc5c1f10] [c005c8bc] sys_read+0x4c/0x8c
[cc5c1f40] [c00022e0] ret_from_syscall+0x0/0x3c
sh            S 0fef0c78     0  1258   1189
Call Trace:
[cc541dc0] [c0038cbc] unlock_page+0x48/0x5c (unreliable)
[cc541e80] [c0009214] __switch_to+0x5c/0x74
[cc541ea0] [c02eeca4] schedule+0x2e8/0x320
[cc541ed0] [c0016f24] do_wait+0xcb0/0xe48
[cc541f40] [c00022e0] ret_from_syscall+0x0/0x3c
zylogger      S 0feb0e88     0  1259   1258
Call Trace:
[cc54bc80] [c0038cbc] unlock_page+0x48/0x5c (unreliable)
[cc54bd40] [c0009214] __switch_to+0x5c/0x74
[cc54bd60] [c02eeca4] schedule+0x2e8/0x320
[cc54bd90] [c00625c0] pipe_wait+0x6c/0xa0
[cc54bdd0] [c0062c88] pipe_read+0x380/0x3fc
[cc54be30] [c005c458] do_sync_read+0xb8/0x10c
[cc54bef0] [c005c574] vfs_read+0xc8/0x15c
[cc54bf10] [c005c8bc] sys_read+0x4c/0x8c
[cc54bf40] [c00022e0] ret_from_syscall+0x0/0x3c
sh            S 0fef0c78     0  1260   1189
Call Trace:
[cc5b3dc0] [c0038cbc] unlock_page+0x48/0x5c (unreliable)
[cc5b3e80] [c0009214] __switch_to+0x5c/0x74
[cc5b3ea0] [c02eeca4] schedule+0x2e8/0x320
[cc5b3ed0] [c0016f24] do_wait+0xcb0/0xe48
[cc5b3f40] [c00022e0] ret_from_syscall+0x0/0x3c
zylogger      S 0feb0ec4     0  1261   1260
Call Trace:
[cc5c5c80] [c00bb488] journal_get_write_access+0x3c/0x54 (unreliable)
[cc5c5d40] [c0009214] __switch_to+0x5c/0x74
[cc5c5d60] [c02eeca4] schedule+0x2e8/0x320
[cc5c5d90] [c00625c0] pipe_wait+0x6c/0xa0
[cc5c5dd0] [c0062c88] pipe_read+0x380/0x3fc
[cc5c5e30] [c005c458] do_sync_read+0xb8/0x10c
[cc5c5ef0] [c005c574] vfs_read+0xc8/0x15c
[cc5c5f10] [c005c8bc] sys_read+0x4c/0x8c
[cc5c5f40] [c00022e0] ret_from_syscall+0x0/0x3c
md0_raid1     S 00000000     0  1308      2
Call Trace:
[cc559e40] [d18aa16c] mv_ial_ht_queuecommand+0x298/0x2e4 [mv_sata] (unreliable)
[cc559f00] [c0009214] __switch_to+0x5c/0x74
[cc559f20] [c02eeca4] schedule+0x2e8/0x320
[cc559f50] [c02ef134] schedule_timeout+0x2c/0xc0
[cc559f90] [c025dd70] md_thread+0xc8/0x128
[cc559fd0] [c0027ea0] kthread+0x48/0x84
[cc559ff0] [c000461c] kernel_thread+0x44/0x60
md0_resync    S 00000000     0  1313      2
Call Trace:
[cc557cd0] [cb6e9600] 0xcb6e9600 (unreliable)
[cc557d90] [c0009214] __switch_to+0x5c/0x74
[cc557db0] [c02eeca4] schedule+0x2e8/0x320
[cc557de0] [c02ef198] schedule_timeout+0x90/0xc0
[cc557e20] [c001d1c4] msleep_interruptible+0x1c/0x54
[cc557e30] [c0249534] sync_request+0x1f0/0x664
[cc557e90] [c026207c] md_do_sync+0x5b4/0xac4
[cc557f90] [c025ddac] md_thread+0x104/0x128
[cc557fd0] [c0027ea0] kthread+0x48/0x84
[cc557ff0] [c000461c] kernel_thread+0x44/0x60
xfsbufd       S 00000000     0  1368      2
Call Trace:
[cb58be60] [c0018774] __do_softirq+0x5c/0xc0 (unreliable)
[cb58bf20] [c0009214] __switch_to+0x5c/0x74
[cb58bf40] [c02eeca4] schedule+0x2e8/0x320
[cb58bf70] [c02ef198] schedule_timeout+0x90/0xc0
[cb58bfb0] [c0157b68] xfsbufd+0x5c/0x140
[cb58bfd0] [c0027ea0] kthread+0x48/0x84
[cb58bff0] [c000461c] kernel_thread+0x44/0x60
xfssyncd      S 00000000     0  1372      2
Call Trace:
[ca503f10] [c0009214] __switch_to+0x5c/0x74
[ca503f30] [c02eeca4] schedule+0x2e8/0x320
[ca503f60] [c02ef198] schedule_timeout+0x90/0xc0
[ca503fa0] [c015dde4] xfssyncd+0x40/0x184
[ca503fd0] [c0027ea0] kthread+0x48/0x84
[ca503ff0] [c000461c] kernel_thread+0x44/0x60
xfsbufd       S 00000000     0  1387      2
Call Trace:
[ca50be60] [c00072d0] do_IRQ+0x88/0xa0 (unreliable)
[ca50bf20] [c0009214] __switch_to+0x5c/0x74
[ca50bf40] [c02eeca4] schedule+0x2e8/0x320
[ca50bf70] [c02ef198] schedule_timeout+0x90/0xc0
[ca50bfb0] [c0157b68] xfsbufd+0x5c/0x140
[ca50bfd0] [c0027ea0] kthread+0x48/0x84
[ca50bff0] [c000461c] kernel_thread+0x44/0x60
xfssyncd      S 00000000     0  1388      2
Call Trace:
[cb72de50] [c0018774] __do_softirq+0x5c/0xc0 (unreliable)
[cb72df10] [c0009214] __switch_to+0x5c/0x74
[cb72df30] [c02eeca4] schedule+0x2e8/0x320
[cb72df60] [c02ef198] schedule_timeout+0x90/0xc0
[cb72dfa0] [c015dde4] xfssyncd+0x40/0x184
[cb72dfd0] [c0027ea0] kthread+0x48/0x84
[cb72dff0] [c000461c] kernel_thread+0x44/0x60
kjournald     S 00000000     0  1403      2
Call Trace:
[cb72ff20] [c0009214] __switch_to+0x5c/0x74
[cb72ff40] [c02eeca4] schedule+0x2e8/0x320
[cb72ff70] [c00beb54] kjournald+0x144/0x1b8
[cb72ffd0] [c0027ea0] kthread+0x48/0x84
[cb72fff0] [c000461c] kernel_thread+0x44/0x60
dhcpcd_reques S 0ff0a224     0  1440      1
Call Trace:
[ca4c3d80] [c38ed7fc] 0xc38ed7fc (unreliable)
[ca4c3e40] [c0009214] __switch_to+0x5c/0x74
[ca4c3e60] [c02eeca4] schedule+0x2e8/0x320
[ca4c3e90] [c02ef7a4] do_nanosleep+0x60/0xa0
[ca4c3eb0] [c002b75c] hrtimer_nanosleep+0x48/0x120
[ca4c3f10] [c002b8f0] sys_nanosleep+0xbc/0x158
[ca4c3f40] [c00022e0] ret_from_syscall+0x0/0x3c
dhcpcd_reques S 0ff0a224     0  1456      1
Call Trace:
[ca57bd80] [ca7027f8] 0xca7027f8 (unreliable)
[ca57be40] [c0009214] __switch_to+0x5c/0x74
[ca57be60] [c02eeca4] schedule+0x2e8/0x320
[ca57be90] [c02ef7a4] do_nanosleep+0x60/0xa0
[ca57beb0] [c002b75c] hrtimer_nanosleep+0x48/0x120
[ca57bf10] [c002b8f0] sys_nanosleep+0xbc/0x158
[ca57bf40] [c00022e0] ret_from_syscall+0x0/0x3c
dhcpcd        S 0ff3e224     0  1479      1
Call Trace:
[cb71fd80] [cb41fb80] 0xcb41fb80 (unreliable)
[cb71fe40] [c0009214] __switch_to+0x5c/0x74
[cb71fe60] [c02eeca4] schedule+0x2e8/0x320
[cb71fe90] [c02ef7a4] do_nanosleep+0x60/0xa0
[cb71feb0] [c002b75c] hrtimer_nanosleep+0x48/0x120
[cb71ff10] [c002b8f0] sys_nanosleep+0xbc/0x158
[cb71ff40] [c00022e0] ret_from_syscall+0x0/0x3c
nmbd          S 0fb639f0     0  1520      1
Call Trace:
[ca5ab980] [cfaf1ac0] 0xcfaf1ac0 (unreliable)
[ca5aba40] [c0009214] __switch_to+0x5c/0x74
[ca5aba60] [c02eeca4] schedule+0x2e8/0x320
[ca5aba90] [c02ef198] schedule_timeout+0x90/0xc0
[ca5abad0] [c006a40c] do_select+0x38c/0x3e4
[ca5abd90] [c006a710] core_sys_select+0x2ac/0x360
[ca5abef0] [c006a8e0] sys_select+0x11c/0x234
[ca5abf30] [c0006c40] ppc_select+0x8c/0x9c
[ca5abf40] [c00022e0] ret_from_syscall+0x0/0x3c
pure-ftpd     S 0f6e99f0     0  1542      1
Call Trace:
[cb76d980] [c000e850] enqueue_task_fair+0x70/0xa8 (unreliable)
[cb76da40] [c0009214] __switch_to+0x5c/0x74
[cb76da60] [c02eeca4] schedule+0x2e8/0x320
[cb76da90] [c02ef134] schedule_timeout+0x2c/0xc0
[cb76dad0] [c006a40c] do_select+0x38c/0x3e4
[cb76dd90] [c006a710] core_sys_select+0x2ac/0x360
[cb76def0] [c006a8e0] sys_select+0x11c/0x234
[cb76df30] [c0006c40] ppc_select+0x8c/0x9c
[cb76df40] [c00022e0] ret_from_syscall+0x0/0x3c
mdadm         S 0ff679f0     0  1770      1
Call Trace:
[cb4f7980] [c000e4a4] sched_slice+0x20/0x9c (unreliable)
[cb4f7a40] [c0009214] __switch_to+0x5c/0x74
[cb4f7a60] [c02eeca4] schedule+0x2e8/0x320
[cb4f7a90] [c02ef198] schedule_timeout+0x90/0xc0
[cb4f7ad0] [c006a40c] do_select+0x38c/0x3e4
[cb4f7d90] [c006a710] core_sys_select+0x2ac/0x360
[cb4f7ef0] [c006a8e0] sys_select+0x11c/0x234
[cb4f7f30] [c0006c40] ppc_select+0x8c/0x9c
[cb4f7f40] [c00022e0] ret_from_syscall+0x0/0x3c
myhotplug     S 0ff3e224     0  1823      1
Call Trace:
[ca5ffd80] [100a0000] 0x100a0000 (unreliable)
[ca5ffe40] [c0009214] __switch_to+0x5c/0x74
[ca5ffe60] [c02eeca4] schedule+0x2e8/0x320
[ca5ffe90] [c02ef7a4] do_nanosleep+0x60/0xa0
[ca5ffeb0] [c002b75c] hrtimer_nanosleep+0x48/0x120
[ca5fff10] [c002b8f0] sys_nanosleep+0xbc/0x158
[ca5fff40] [c00022e0] ret_from_syscall+0x0/0x3c
smbd          S 0fa739f0     0  1902      1
Call Trace:
[ca5db980] [cf426000] 0xcf426000 (unreliable)
[ca5dba40] [c0009214] __switch_to+0x5c/0x74
[ca5dba60] [c02eeca4] schedule+0x2e8/0x320
[ca5dba90] [c02ef134] schedule_timeout+0x2c/0xc0
[ca5dbad0] [c006a40c] do_select+0x38c/0x3e4
[ca5dbd90] [c006a710] core_sys_select+0x2ac/0x360
[ca5dbef0] [c006a8e0] sys_select+0x11c/0x234
[ca5dbf30] [c0006c40] ppc_select+0x8c/0x9c
[ca5dbf40] [c00022e0] ret_from_syscall+0x0/0x3c
do_btncpy     S 0ff3e224     0  1905      1
Call Trace:
[cb631d80] [c0007324] do_softirq+0x3c/0x54 (unreliable)
[cb631e40] [c0009214] __switch_to+0x5c/0x74
[cb631e60] [c02eeca4] schedule+0x2e8/0x320
[cb631e90] [c02ef7a4] do_nanosleep+0x60/0xa0
[cb631eb0] [c002b75c] hrtimer_nanosleep+0x48/0x120
[cb631f10] [c002b8f0] sys_nanosleep+0xbc/0x158
[cb631f40] [c00022e0] ret_from_syscall+0x0/0x3c
crond         S 0fef1224     0  1908      1
Call Trace:
[cb62fd80] [cf421c40] 0xcf421c40 (unreliable)
[cb62fe40] [c0009214] __switch_to+0x5c/0x74
[cb62fe60] [c02eeca4] schedule+0x2e8/0x320
[cb62fe90] [c02ef7a4] do_nanosleep+0x60/0xa0
[cb62feb0] [c002b75c] hrtimer_nanosleep+0x48/0x120
[cb62ff10] [c002b8f0] sys_nanosleep+0xbc/0x158
[cb62ff40] [c00022e0] ret_from_syscall+0x0/0x3c
diagnosed     S 0fe5d1b8     0  1912      1
Call Trace:
[ca5c7e20] [c004145c] put_page+0x128/0x138 (unreliable)
[ca5c7ee0] [c0009214] __switch_to+0x5c/0x74
[ca5c7f00] [c02eeca4] schedule+0x2e8/0x320
[ca5c7f30] [c0020ae8] sys_pause+0x18/0x2c
[ca5c7f40] [c00022e0] ret_from_syscall+0x0/0x3c
smbd          S 0fa4a1b8     0  1921   1902
Call Trace:
[cb65fee0] [c0009214] __switch_to+0x5c/0x74
[cb65ff00] [c02eeca4] schedule+0x2e8/0x320
[cb65ff30] [c0020ae8] sys_pause+0x18/0x2c
[cb65ff40] [c00022e0] ret_from_syscall+0x0/0x3c
httpd         S 0fb569f0     0  1922      1
Call Trace:
[cb463980] [c000d018] openpic_get_irq+0x14/0x78 (unreliable)
[cb463a40] [c0009214] __switch_to+0x5c/0x74
[cb463a60] [c02eeca4] schedule+0x2e8/0x320
[cb463a90] [c02ef198] schedule_timeout+0x90/0xc0
[cb463ad0] [c006a40c] do_select+0x38c/0x3e4
[cb463d90] [c006a710] core_sys_select+0x2ac/0x360
[cb463ef0] [c006a8e0] sys_select+0x11c/0x234
[cb463f30] [c0006c40] ppc_select+0x8c/0x9c
[cb463f40] [c00022e0] ret_from_syscall+0x0/0x3c
nduagent      S 0fbe368c     0  1925      1
Call Trace:
[cb469b10] [c000f198] enqueue_task+0x24/0x40 (unreliable)
[cb469bd0] [c0009214] __switch_to+0x5c/0x74
[cb469bf0] [c02eeca4] schedule+0x2e8/0x320
[cb469c20] [c02ef134] schedule_timeout+0x2c/0xc0
[cb469c60] [c02815dc] skb_recv_datagram+0x19c/0x20c
[cb469cc0] [c02be2dc] udp_recvmsg+0x6c/0x218
[cb469d00] [c027d264] sock_common_recvmsg+0x3c/0x60
[cb469d20] [c0278ec8] sock_recvmsg+0xb8/0xe8
[cb469e20] [c027a288] sys_recvfrom+0x9c/0x108
[cb469f00] [c027aa60] sys_socketcall+0x178/0x1dc
[cb469f40] [c00022e0] ret_from_syscall+0x0/0x3c
httpd         S 0fb540bc     0  2034   1922
Call Trace:
[cb52fa30] [0023033f] 0x23033f (unreliable)
[cb52faf0] [c0009214] __switch_to+0x5c/0x74
[cb52fb10] [c02eeca4] schedule+0x2e8/0x320
[cb52fb40] [c02ef134] schedule_timeout+0x2c/0xc0
[cb52fb80] [c006b058] do_sys_poll+0x2d4/0x3a8
[cb52ff20] [c006b208] sys_poll+0x58/0xa4
[cb52ff40] [c00022e0] ret_from_syscall+0x0/0x3c
httpd         S 0fb5f140     0  2035   1922
Call Trace:
[cb53dbd0] [cb53dcc0] 0xcb53dcc0 (unreliable)
[cb53dc90] [c0009214] __switch_to+0x5c/0x74
[cb53dcb0] [c02eeca4] schedule+0x2e8/0x320
[cb53dce0] [c01625f8] sys_semtimedop+0x53c/0x628
[cb53df00] [c00067e0] sys_ipc+0x7c/0x25c
[cb53df40] [c00022e0] ret_from_syscall+0x0/0x3c
zysync        S 0fd329f0     0  2192      1
Call Trace:
[cb66b980] [c0002934] ret_from_except+0x0/0x18 (unreliable)
[cb66ba40] [c0009214] __switch_to+0x5c/0x74
[cb66ba60] [c02eeca4] schedule+0x2e8/0x320
[cb66ba90] [c02ef134] schedule_timeout+0x2c/0xc0
[cb66bad0] [c006a40c] do_select+0x38c/0x3e4
[cb66bd90] [c006a710] core_sys_select+0x2ac/0x360
[cb66bef0] [c006a8e0] sys_select+0x11c/0x234
[cb66bf30] [c0006c40] ppc_select+0x8c/0x9c
[cb66bf40] [c00022e0] ret_from_syscall+0x0/0x3c
stunnel       S 0ff29c14     0  2220      1
Call Trace:
[cb46bae0] [cb4d0000] 0xcb4d0000 (unreliable)
[cb46bba0] [c0009214] __switch_to+0x5c/0x74
[cb46bbc0] [c02eeca4] schedule+0x2e8/0x320
[cb46bbf0] [c02ef134] schedule_timeout+0x2c/0xc0
[cb46bc30] [c02d475c] unix_stream_recvmsg+0x1d0/0x438
[cb46bcd0] [c0278ec8] sock_recvmsg+0xb8/0xe8
[cb46bdd0] [c027a834] sys_recvmsg+0x138/0x1ec
[cb46bf00] [c027aaa8] sys_socketcall+0x1c0/0x1dc
[cb46bf40] [c00022e0] ret_from_syscall+0x0/0x3c
stunnel       S 0ff29c14     0  2221      1
Call Trace:
[cb521ae0] [cb4d0450] 0xcb4d0450 (unreliable)
[cb521ba0] [c0009214] __switch_to+0x5c/0x74
[cb521bc0] [c02eeca4] schedule+0x2e8/0x320
[cb521bf0] [c02ef134] schedule_timeout+0x2c/0xc0
[cb521c30] [c02d475c] unix_stream_recvmsg+0x1d0/0x438
[cb521cd0] [c0278ec8] sock_recvmsg+0xb8/0xe8
[cb521dd0] [c027a834] sys_recvmsg+0x138/0x1ec
[cb521f00] [c027aaa8] sys_socketcall+0x1c0/0x1dc
[cb521f40] [c00022e0] ret_from_syscall+0x0/0x3c
stunnel       S 0ff29c14     0  2222      1
Call Trace:
[cb52dae0] [c0018aa8] tasklet_action+0x7c/0xb8 (unreliable)
[cb52dba0] [c0009214] __switch_to+0x5c/0x74
[cb52dbc0] [c02eeca4] schedule+0x2e8/0x320
[cb52dbf0] [c02ef134] schedule_timeout+0x2c/0xc0
[cb52dc30] [c02d475c] unix_stream_recvmsg+0x1d0/0x438
[cb52dcd0] [c0278ec8] sock_recvmsg+0xb8/0xe8
[cb52ddd0] [c027a834] sys_recvmsg+0x138/0x1ec
[cb52df00] [c027aaa8] sys_socketcall+0x1c0/0x1dc
[cb52df40] [c00022e0] ret_from_syscall+0x0/0x3c
stunnel       S 0ff29c14     0  2223      1
Call Trace:
[cb727ae0] [5a6ad144] 0x5a6ad144 (unreliable)
[cb727ba0] [c0009214] __switch_to+0x5c/0x74
[cb727bc0] [c02eeca4] schedule+0x2e8/0x320
[cb727bf0] [c02ef134] schedule_timeout+0x2c/0xc0
[cb727c30] [c02d475c] unix_stream_recvmsg+0x1d0/0x438
[cb727cd0] [c0278ec8] sock_recvmsg+0xb8/0xe8
[cb727dd0] [c027a834] sys_recvmsg+0x138/0x1ec
[cb727f00] [c027aaa8] sys_socketcall+0x1c0/0x1dc
[cb727f40] [c00022e0] ret_from_syscall+0x0/0x3c
stunnel       S 0ff29c14     0  2224      1
Call Trace:
[cb529ae0] [c017f318] get_io_context+0x10/0x38 (unreliable)
[cb529ba0] [c0009214] __switch_to+0x5c/0x74
[cb529bc0] [c02eeca4] schedule+0x2e8/0x320
[cb529bf0] [c02ef134] schedule_timeout+0x2c/0xc0
[cb529c30] [c02d475c] unix_stream_recvmsg+0x1d0/0x438
[cb529cd0] [c0278ec8] sock_recvmsg+0xb8/0xe8
[cb529dd0] [c027a834] sys_recvmsg+0x138/0x1ec
[cb529f00] [c027aaa8] sys_socketcall+0x1c0/0x1dc
[cb529f40] [c00022e0] ret_from_syscall+0x0/0x3c
stunnel       S 0fc060bc     0  2225      1
Call Trace:
[cb52ba30] [4c0d93a1] 0x4c0d93a1 (unreliable)
[cb52baf0] [c0009214] __switch_to+0x5c/0x74
[cb52bb10] [c02eeca4] schedule+0x2e8/0x320
[cb52bb40] [c02ef134] schedule_timeout+0x2c/0xc0
[cb52bb80] [c006b058] do_sys_poll+0x2d4/0x3a8
[cb52bf20] [c006b208] sys_poll+0x58/0xa4
[cb52bf40] [c00022e0] ret_from_syscall+0x0/0x3c
app_wd        S 0fe5d224     0  2229      1
Call Trace:
[cb4d3d80] [00000014] 0x14 (unreliable)
[cb4d3e40] [c0009214] __switch_to+0x5c/0x74
[cb4d3e60] [c02eeca4] schedule+0x2e8/0x320
[cb4d3e90] [c02ef7a4] do_nanosleep+0x60/0xa0
[cb4d3eb0] [c002b75c] hrtimer_nanosleep+0x48/0x120
[cb4d3f10] [c002b8f0] sys_nanosleep+0xbc/0x158
[cb4d3f40] [c00022e0] ret_from_syscall+0x0/0x3c
schedule_cont S 0ff6f6b8     0  2238      1
Call Trace:
[ca5f5b10] [0000000a] 0xa (unreliable)
[ca5f5bd0] [c0009214] __switch_to+0x5c/0x74
[ca5f5bf0] [c02eeca4] schedule+0x2e8/0x320
[ca5f5c20] [c02ef134] schedule_timeout+0x2c/0xc0
[ca5f5c60] [c02815dc] skb_recv_datagram+0x19c/0x20c
[ca5f5cc0] [c02be2dc] udp_recvmsg+0x6c/0x218
[ca5f5d00] [c027d264] sock_common_recvmsg+0x3c/0x60
[ca5f5d20] [c0278ec8] sock_recvmsg+0xb8/0xe8
[ca5f5e20] [c027a288] sys_recvfrom+0x9c/0x108
[ca5f5f00] [c027aa60] sys_socketcall+0x178/0x1dc
[ca5f5f40] [c00022e0] ret_from_syscall+0x0/0x3c
timer_source  S 0ff679f0     0  2240      1
Call Trace:
[cb635980] [c0028350] autoremove_wake_function+0x18/0x50 (unreliable)
[cb635a40] [c0009214] __switch_to+0x5c/0x74
[cb635a60] [c02eeca4] schedule+0x2e8/0x320
[cb635a90] [c02ef198] schedule_timeout+0x90/0xc0
[cb635ad0] [c006a40c] do_select+0x38c/0x3e4
[cb635d90] [c006a710] core_sys_select+0x2ac/0x360
[cb635ef0] [c006a8e0] sys_select+0x11c/0x234
[cb635f30] [c0006c40] ppc_select+0x8c/0x9c
[cb635f40] [c00022e0] ret_from_syscall+0x0/0x3c
sh            S 0fef0c78     0  2244      1
Call Trace:
[ca523dc0] [cb6cb7f8] 0xcb6cb7f8 (unreliable)
[ca523e80] [c0009214] __switch_to+0x5c/0x74
[ca523ea0] [c02eeca4] schedule+0x2e8/0x320
[ca523ed0] [c0016f24] do_wait+0xcb0/0xe48
[ca523f40] [c00022e0] ret_from_syscall+0x0/0x3c
ksnaphd       S 00000000     0  2293      2
Call Trace:
[cb537e80] [c03e9948] 0xc03e9948 (unreliable)
[cb537f40] [c0009214] __switch_to+0x5c/0x74
[cb537f60] [c02eeca4] schedule+0x2e8/0x320
[cb537f90] [c0023ec0] worker_thread+0x80/0xbc
[cb537fd0] [c0027ea0] kthread+0x48/0x84
[cb537ff0] [c000461c] kernel_thread+0x44/0x60
kcopyd        D 00000000     0  2295      2
Call Trace:
[ca5b5d30] [c00072d0] do_IRQ+0x88/0xa0 (unreliable)
[ca5b5df0] [c0009214] __switch_to+0x5c/0x74
[ca5b5e10] [c02eeca4] schedule+0x2e8/0x320
[ca5b5e40] [c017e004] generic_make_request+0x184/0x3a4
[ca5b5eb0] [c026f774] flush_bios+0x28/0x48
[ca5b5ed0] [c02713c8] persistent_commit+0x1c4/0x1e8
[ca5b5f10] [c026fac4] copy_callback+0x6c/0x7c
[ca5b5f20] [c026c8d0] run_complete_job+0x58/0xa0
[ca5b5f50] [c026cb30] process_jobs+0x28/0x100
[ca5b5f70] [c026cc2c] do_work+0x24/0x5c
[ca5b5f80] [c0023dc0] run_workqueue+0xbc/0x13c
[ca5b5f90] [c0023ee0] worker_thread+0xa0/0xbc
[ca5b5fd0] [c0027ea0] kthread+0x48/0x84
[ca5b5ff0] [c000461c] kernel_thread+0x44/0x60
cp            D 0ff12f0c     0  2312   2244
Call Trace:
[ca41d820] [ca4c5b60] 0xca4c5b60 (unreliable)
[ca41d8e0] [c0009214] __switch_to+0x5c/0x74
[ca41d900] [c02eeca4] schedule+0x2e8/0x320
[ca41d930] [c017e004] generic_make_request+0x184/0x3a4
[ca41d9a0] [c017e30c] submit_bio+0xe8/0xfc
[ca41d9e0] [c0153a08] xfs_submit_ioend_bio+0x40/0x5c
[ca41da00] [c0153c68] xfs_submit_ioend+0xf8/0x124
[ca41da20] [c0154df8] xfs_page_state_convert+0x684/0x6e8
[ca41dad0] [c0154f2c] xfs_vm_writepage+0xd0/0x128
[ca41db00] [c003fafc] __writepage+0x24/0x68
[ca41db10] [c0040360] write_cache_pages+0x1ec/0x33c
[ca41dbb0] [c0154fa8] xfs_vm_writepages+0x24/0x34
[ca41dbc0] [c0040520] do_writepages+0x48/0x78
[ca41dbd0] [c007bd28] __writeback_single_inode+0x12c/0x29c
[ca41dc20] [c007c2b8] sync_sb_inodes+0x1e8/0x284
[ca41dc50] [c007c594] writeback_inodes+0x70/0xb4
[ca41dc70] [c003f528] balance_dirty_pages_ratelimited_nr+0x118/0x250
[ca41dce0] [c003ac18] generic_file_buffered_write+0x1bc/0x5d4
[ca41dd80] [c015cf80] xfs_write+0x4f4/0x748
[ca41de20] [c0158238] xfs_file_aio_write+0x6c/0x7c
[ca41de30] [c005c6c0] do_sync_write+0xb8/0x10c
[ca41def0] [c005c7dc] vfs_write+0xc8/0x15c
[ca41df10] [c005c948] sys_write+0x4c/0x8c
[ca41df40] [c00022e0] ret_from_syscall+0x0/0x3c

I find many threads blocked by "generic_make_request", but I am not sure this 
was
caused by zumastor's patch.  Then, I rebooted the system and the system was 
hanged by
"mount" process.  The trace info was what I posted before. 

Besides, when I traced the hang problem for "mount" process, I added printk in 
__generic_make_request and bio_endio to see the values for need and available.  
I
find zumastor's patch in the function, __generic_make_request, would consume 
and ran
out of the available value.  However, no one called bio_endio to return the 
available
value and wake up the "mount" process.  This is why the "mount" process would be
hanged.  But I do not know why there is no one to do this.  If I increased the
default throttle value to 10240, the available value would be returned but the
"mount" process was still in hang.  I think this may be the race condition issue
mentioned by FIXME in __generic_make_request.  
The CPU is Freescale 8543 and ram size is 256MB.  Kernel version is 
Linux-2.6.24.2.

Original comment by ej3Georg...@gmail.com on 27 May 2008 at 2:17

GoogleCodeExporter commented 9 years ago 
I tried to reproduce your problem on my testing machine and have managed to get
another hanging problem. It is similar to your first test, but that is lvremove 
that
was hung in my case. The test script and the sysrq-t trace are attached. I will 
do
more tests to see if the problem is caused by the bio_throttling patch and if 
'ddsnap
snapshot' has the same problem.

Original comment by jiayin...@gmail.com on 27 May 2008 at 10:24

Attachments:

GoogleCodeExporter commented 9 years ago 
There are some problems while using xfs_freeze before creating snapshot.  
http://wiki.centos.org/TipsAndTricks/LvmSnapshotsOnXFS

Original comment by ej3Georg...@gmail.com on 28 May 2008 at 1:58

GoogleCodeExporter commented 9 years ago 
[deleted comment]
GoogleCodeExporter commented 9 years ago 
Thanks for the link!

I haven't seen any problem with the above test if the bio_throttling patch is 
not
applied. With the bio_throttling patch applied, I did not see any problem 
either when
running xfs with ddsnap snapshots. However, sync hangs when I run xfs on top of
ddsnap on top of software raid.

Original comment by jiayin...@gmail.com on 28 May 2008 at 2:07

GoogleCodeExporter commented 9 years ago 
Please reference the attached script file.

Original comment by ej3Georg...@gmail.com on 28 May 2008 at 3:05

Attachments:

GoogleCodeExporter commented 9 years ago 
In the attachment is a patch that turns on/off bio throttling with a sysctl 
variable.
Could you apply the patch and try your test with bio_throttle turned off:
'cat 0 > /proc/sys/dev/dm/bio_throttle'? Your test hanged on my machine when the
bio_throttle is turned on, but passed when bio_throttle is off. We will look at 
our
bio_throttling patch more carefully and let you know once we find where the 
problem is.

Thanks a lot for your info!

Original comment by jiayin...@gmail.com on 28 May 2008 at 9:44

Attachments:

GoogleCodeExporter commented 9 years ago 
We will add a sysctl variable to enable/disable bio_throttling in 0.9 release. 
After
that, we can re-target the issue for a later release that will have 
bio_throttling
bug fixed.

Original comment by jiahotc...@gmail.com on 5 Jun 2008 at 5:15

GoogleCodeExporter commented 9 years ago

Original comment by jiahotc...@gmail.com on 5 Jun 2008 at 5:17

GoogleCodeExporter commented 9 years ago 
Patch of adding a sysctl variable to enable/disable bio_throttling was 
committed as
revision 1689. We currently disable bio_throttling by default. Defer the issue 
to
0.10 for the real fix of the bio_throttling patch.

Original comment by jiahotc...@gmail.com on 6 Jun 2008 at 6:53