CendioOssman
commented
5 years ago This is something we've seen a few times before (#185 and #233). It seems like there is still something more lurking.
Can you reproduce this or did it just happen once?
Closed balwierz closed 4 years ago
CendioOssman
commented
5 years ago This is something we've seen a few times before (#185 and #233). It seems like there is still something more lurking.
Can you reproduce this or did it just happen once?
balwierz
commented
5 years ago I have seen both reports, but these were for the old versions and both are closed.
It happened once and I did not manage to reproduce it.
CendioOssman
commented
5 years ago The stack dump unfortunately doesn't tell us where the issue is. So please try to reproduce this. We'll need to add some debug code once we have a decent way of testing things.
CendioOssman
commented
4 years ago This has been quiet for quite some time. Closing.
stefvanvlierberghe
commented
4 years ago Hello Pierre,
I'm having this issue frequently on Red Hat Enterprise Linux Server release 7.7 (Maipo) running on vmware. Tried both 1.8.0 and 1.9.0.
VNCSConnST: Server default pixel format depth 24 (32bpp) little-endian rgb888
VNCSConnST: Client pixel format depth 24 (32bpp) little-endian rgb888
(EE)
(EE) Backtrace:
(EE) 0: /usr/bin/Xvnc (xorg_backtrace+0x55) [0x5c31b5]
(EE) 1: /usr/bin/Xvnc (0x400000+0x1c6b19) [0x5c6b19]
(EE) 2: /usr/lib64/libpthread.so.0 (0x7ffff741a000+0xf630) [0x7ffff7429630]
(EE) 3: /usr/bin/Xvnc (_ZNK3rfb6Region9get_rectsEPSt6vectorINS_4RectESaIS2_EEbbi+0x16d) [0x5383ad]
(EE) 4: /usr/bin/Xvnc (_ZN3rfb22ComparingUpdateTracker7compareEv+0x196) [0x53c1b6]
(EE) 5: /usr/bin/Xvnc (_ZN3rfb11VNCServerST11writeUpdateEv+0x208) [0x53a188]
(EE) 6: /usr/bin/Xvnc (_ZN3rfb11VNCServerST13handleTimeoutEPNS_5TimerE+0x3e) [0x53a39e]
(EE) 7: /usr/bin/Xvnc (_ZN3rfb5Timer13checkTimeoutsEv+0x97) [0x5470f7]
(EE) 8: /usr/bin/Xvnc (_ZN3rfb11VNCServerST13checkTimeoutsEv+0x1d) [0x538cbd]
(EE) 9: /usr/bin/Xvnc (_ZN14XserverDesktop12blockHandlerEPi+0x1fd) [0x52a63d]
(EE) 10: /usr/bin/Xvnc (vncCallBlockHandlers+0x28) [0x520848]
(EE) 11: /usr/bin/Xvnc (BlockHandler+0x56) [0x578396]
(EE) 12: /usr/bin/Xvnc (WaitForSomething+0x90) [0x5c1040]
(EE) 13: /usr/bin/Xvnc (Dispatch+0xac) [0x57382c]
(EE) 14: /usr/bin/Xvnc (dix_main+0x39a) [0x57799a]
(EE) 15: /usr/lib64/libc.so.6 (__libc_start_main+0xf5) [0x7ffff4f58545]
(EE) 16: /usr/bin/Xvnc (0x400000+0x5630e) [0x45630e]
(EE)
(EE) Floating point exception at address 0x5383ad
(EE)
Fatal server error:
(EE) Caught signal 8 (Floating point exception). Server aborting
(EE) Was already mentioned in 2015 :
https://bugzilla.redhat.com/show_bug.cgi?id=1282360
I tried to understand by attaching a gdb to 1.9.0 and it looks like a division by zero, you find below the gdb output, stack trace, and a disass showing an idiv %ebx with rbx zero.
Program received signal SIGFPE, Arithmetic exception.
0x0000000000631134 in rfb::Region::get_rects(std::vector<rfb::Rect, std::allocator<rfb::Rect> >*, bool, bool, int) const ()
(gdb) info args
No symbol table info available.
(gdb) up
#1 0x0000000000636026 in rfb::ComparingUpdateTracker::compare() ()
(gdb) bt
#0 0x0000000000631134 in rfb::Region::get_rects(std::vector<rfb::Rect, std::allocator<rfb::Rect> >*, bool, bool, int) const ()
#1 0x0000000000636026 in rfb::ComparingUpdateTracker::compare() ()
#2 0x0000000000633ed4 in rfb::VNCServerST::writeUpdate() ()
#3 0x0000000000634152 in rfb::VNCServerST::handleTimeout(rfb::Timer*) ()
#4 0x00000000006318bd in rfb::Timer::checkTimeouts() ()
#5 0x000000000063202d in rfb::VNCServerST::checkTimeouts() ()
#6 0x0000000000623087 in XserverDesktop::blockHandler(int*) ()
#7 0x0000000000617907 in vncCallBlockHandlers ()
#8 0x000000000062065d in ?? ()
#9 0x00000000005bd925 in BlockHandler ()
#10 0x0000000000605b2d in WaitForSomething ()
#11 0x00000000005b945d in Dispatch ()
#12 0x00000000004cd4fe in main ()
(gdb) do
#0 0x0000000000631134 in rfb::Region::get_rects(std::vector<rfb::Rect, std::allocator<rfb::Rect> >*, bool, bool, int) const ()
(gdb) do
Bottom (innermost) frame selected; you cannot go down.
(gdb) disass
Dump of assembler code for function _ZNK3rfb6Region9get_rectsEPSt6vectorINS_4RectESaIS2_EEbbi:
0x0000000000630ff0 <+0>: push %r15
0x0000000000630ff2 <+2>: push %r14
0x0000000000630ff4 <+4>: mov %rdi,%r14
0x0000000000630ff7 <+7>: push %r13
0x0000000000630ff9 <+9>: push %r12
0x0000000000630ffb <+11>: mov %rsi,%r12
0x0000000000630ffe <+14>: push %rbp
0x0000000000630fff <+15>: push %rbx
0x0000000000631000 <+16>: sub $0x48,%rsp
0x0000000000631004 <+20>: mov (%rdi),%rax
0x0000000000631007 <+23>: cmp $0x1,%dl
0x000000000063100a <+26>: mov %r8d,0xc(%rsp)
0x000000000063100f <+31>: mov 0x8(%rax),%r13
0x0000000000631013 <+35>: sbb %eax,%eax
0x0000000000631015 <+37>: or $0x1,%eax
0x0000000000631018 <+40>: test %cl,%cl
0x000000000063101a <+42>: mov %eax,0x24(%rsp)
0x000000000063101e <+46>: mov %r13d,%r9d
0x0000000000631021 <+49>: jne 0x631226 <_ZNK3rfb6Region9get_rectsEPSt6vectorINS_4RectESaIS2_EEbbi+566>
0x0000000000631027 <+55>: lea -0x1(%r13),%eax
0x000000000063102b <+59>: movl $0xffffffff,0x18(%rsp)
0x0000000000631033 <+67>: mov %eax,0x1c(%rsp)
0x0000000000631037 <+71>: mov (%r12),%rdi
0x000000000063103b <+75>: movslq %r13d,%rbp
0x000000000063103e <+78>: movabs $0xfffffffffffffff,%rax
0x0000000000631048 <+88>: cmp %rax,%rbp
0x000000000063104b <+91>: mov %rdi,0x8(%r12)
0x0000000000631050 <+96>: ja 0x6312ac <_ZNK3rfb6Region9get_rectsEPSt6vectorINS_4RectESaIS2_EEbbi+700>
0x0000000000631056 <+102>: mov 0x10(%r12),%rax
0x000000000063105b <+107>: mov %rdi,%rsi
0x000000000063105e <+110>: sub %rdi,%rax
0x0000000000631061 <+113>: sar $0x4,%rax
0x0000000000631065 <+117>: cmp %rax,%rbp
0x0000000000631068 <+120>: ja 0x63123b <_ZNK3rfb6Region9get_rectsEPSt6vectorINS_4RectESaIS2_EEbbi+587>
0x000000000063106e <+126>: test %r13d,%r13d
0x0000000000631071 <+129>: jle 0x63128a <_ZNK3rfb6Region9get_rectsEPSt6vectorINS_4RectESaIS2_EEbbi+666>
0x0000000000631077 <+135>: mov (%r14),%rax
0x000000000063107a <+138>: mov 0x10(%rax),%r11
0x000000000063107e <+142>: movslq 0x18(%rsp),%rax
0x0000000000631083 <+147>: shl $0x3,%rax
0x0000000000631087 <+151>: mov %rax,0x28(%rsp)
0x000000000063108c <+156>: movslq 0x24(%rsp),%rax
0x0000000000631091 <+161>: shl $0x3,%rax
0x0000000000631095 <+165>: mov %rax,(%rsp)
0x0000000000631099 <+169>: lea 0x30(%rsp),%rax
0x000000000063109e <+174>: mov %rax,0x10(%rsp)
0x00000000006310a3 <+179>: mov %r14,%rax
0x00000000006310a6 <+182>: mov %r9d,%r14d
0x00000000006310a9 <+185>: mov %rax,%r9
0x00000000006310ac <+188>: mov 0x1c(%rsp),%ecx
0x00000000006310b0 <+192>: mov 0x18(%rsp),%edi
0x00000000006310b4 <+196>: xor %r13d,%r13d
0x00000000006310b7 <+199>: mov 0x28(%rsp),%r8
0x00000000006310bc <+204>: movslq %ecx,%rax
0x00000000006310bf <+207>: lea (%r11,%rax,8),%rax
0x00000000006310c3 <+211>: movzwl 0x4(%rax),%edx
0x00000000006310c7 <+215>: nopw 0x0(%rax,%rax,1)
0x00000000006310d0 <+224>: cmp %dx,0x4(%rax)
0x00000000006310d4 <+228>: jne 0x631203 <_ZNK3rfb6Region9get_rectsEPSt6vectorINS_4RectESaIS2_EEbbi+531>
0x00000000006310da <+234>: add %edi,%ecx
0x00000000006310dc <+236>: add $0x1,%r13d
0x00000000006310e0 <+240>: add %r8,%rax
0x00000000006310e3 <+243>: sub $0x1,%r14d
0x00000000006310e7 <+247>: jne 0x6310d0 <_ZNK3rfb6Region9get_rectsEPSt6vectorINS_4RectESaIS2_EEbbi+224>
0x00000000006310e9 <+249>: mov 0x18(%rsp),%edi
0x00000000006310ed <+253>: cmp %edi,0x24(%rsp)
0x00000000006310f1 <+257>: jne 0x631211 <_ZNK3rfb6Region9get_rectsEPSt6vectorINS_4RectESaIS2_EEbbi+545>
0x00000000006310f7 <+263>: mov 0x1c(%rsp),%eax
0x00000000006310fb <+267>: mov %ecx,0x1c(%rsp)
0x00000000006310ff <+271>: cltq
0x0000000000631101 <+273>: mov %r14d,0x20(%rsp)
0x0000000000631106 <+278>: mov %r9,%r14
0x0000000000631109 <+281>: lea 0x0(,%rax,8),%r15
0x0000000000631111 <+289>: nopl 0x0(%rax)
0x0000000000631118 <+296>: lea (%r11,%r15,1),%rdi
0x000000000063111c <+300>: mov 0xc(%rsp),%eax
0x0000000000631120 <+304>: movswl 0x2(%rdi),%r8d
0x0000000000631125 <+309>: movswl (%rdi),%r10d
0x0000000000631129 <+313>: cltd
0x000000000063112a <+314>: movswl 0x4(%rdi),%ecx
0x000000000063112e <+318>: mov %r8d,%ebx
0x0000000000631131 <+321>: sub %r10d,%ebx
=> 0x0000000000631134 <+324>: idiv %ebx
0x0000000000631136 <+326>: mov %eax,%ebx
0x0000000000631138 <+328>: movswl 0x6(%rdi),%eax
0x000000000063113c <+332>: test %ebx,%ebx
0x000000000063113e <+334>: jne 0x63115b <_ZNK3rfb6Region9get_rectsEPSt6vectorINS_4RectESaIS2_EEbbi+363>
0x0000000000631140 <+336>: mov %eax,%ebx
0x0000000000631142 <+338>: sub %ecx,%ebx
0x0000000000631144 <+340>: jmp 0x63115b <_ZNK3rfb6Region9get_rectsEPSt6vectorINS_4RectESaIS2_EEbbi+363>
0x0000000000631146 <+342>: nopw %cs:0x0(%rax,%rax,1)
0x0000000000631150 <+352>: movswl 0x2(%rdi),%r8d
0x0000000000631155 <+357>: movswl (%rdi),%r10d
0x0000000000631159 <+361>: mov %ebp,%ecx
0x000000000063115b <+363>: mov %eax,%edx
0x000000000063115d <+365>: mov %r10d,0x30(%rsp)
0x0000000000631162 <+370>: mov %ecx,0x34(%rsp)
0x0000000000631166 <+374>: sub %ecx,%edx
0x0000000000631168 <+376>: mov %r8d,0x38(%rsp)
0x000000000063116d <+381>: cmp %edx,%ebx
0x000000000063116f <+383>: cmovg %edx,%ebx
0x0000000000631172 <+386>: cmp %rsi,0x10(%r12)
0x0000000000631177 <+391>: lea (%rcx,%rbx,1),%ebp
0x000000000063117a <+394>: mov %ebp,0x3c(%rsp)
0x000000000063117e <+398>: je 0x6311e0 <_ZNK3rfb6Region9get_rectsEPSt6vectorINS_4RectESaIS2_EEbbi+496>
0x0000000000631180 <+400>: test %rsi,%rsi
0x0000000000631183 <+403>: je 0x631196 <_ZNK3rfb6Region9get_rectsEPSt6vectorINS_4RectESaIS2_EEbbi+422>
0x0000000000631185 <+405>: mov 0x30(%rsp),%r9
0x000000000063118a <+410>: mov 0x38(%rsp),%r10
0x000000000063118f <+415>: mov %r9,(%rsi)
0x0000000000631192 <+418>: mov %r10,0x8(%rsi)
0x0000000000631196 <+422>: add $0x10,%rsi
0x000000000063119a <+426>: mov %rsi,0x8(%r12)
0x000000000063119f <+431>: cmp %eax,%ebp
0x00000000006311a1 <+433>: jl 0x631150 <_ZNK3rfb6Region9get_rectsEPSt6vectorINS_4RectESaIS2_EEbbi+352>
0x00000000006311a3 <+435>: add (%rsp),%r15
0x00000000006311a7 <+439>: sub $0x1,%r13d
0x00000000006311ab <+443>: jne 0x631118 <_ZNK3rfb6Region9get_rectsEPSt6vectorINS_4RectESaIS2_EEbbi+296>
0x00000000006311b1 <+449>: mov %r14,%r9
0x00000000006311b4 <+452>: mov 0x20(%rsp),%r14d
0x00000000006311b9 <+457>: test %r14d,%r14d
0x00000000006311bc <+460>: jne 0x6310ac <_ZNK3rfb6Region9get_rectsEPSt6vectorINS_4RectESaIS2_EEbbi+188>
0x00000000006311c2 <+466>: cmp %rsi,(%r12)
0x00000000006311c6 <+470>: setne %al
0x00000000006311c9 <+473>: add $0x48,%rsp
0x00000000006311cd <+477>: pop %rbx
0x00000000006311ce <+478>: pop %rbp
0x00000000006311cf <+479>: pop %r12
0x00000000006311d1 <+481>: pop %r13
0x00000000006311d3 <+483>: pop %r14
0x00000000006311d5 <+485>: pop %r15
0x00000000006311d7 <+487>: retq
0x00000000006311d8 <+488>: nopl 0x0(%rax,%rax,1)
0x00000000006311e0 <+496>: mov 0x10(%rsp),%rdx
0x00000000006311e5 <+501>: mov %r12,%rdi
0x00000000006311e8 <+504>: callq 0x6312c0 <_ZNSt6vectorIN3rfb4RectESaIS1_EE13_M_insert_auxEN9__gnu_cxx17__normal_iteratorIPS1_S3_EERKS1_>
0x00000000006311ed <+509>: mov (%r14),%rax
0x00000000006311f0 <+512>: mov 0x8(%r12),%rsi
0x00000000006311f5 <+517>: mov 0x10(%rax),%r11
0x00000000006311f9 <+521>: lea (%r11,%r15,1),%rdi
0x00000000006311fd <+525>: movswl 0x6(%rdi),%eax
0x0000000000631201 <+529>: jmp 0x63119f <_ZNK3rfb6Region9get_rectsEPSt6vectorINS_4RectESaIS2_EEbbi+431>
0x0000000000631203 <+531>: mov 0x18(%rsp),%edi
0x0000000000631207 <+535>: cmp %edi,0x24(%rsp)
0x000000000063120b <+539>: je 0x63129f <_ZNK3rfb6Region9get_rectsEPSt6vectorINS_4RectESaIS2_EEbbi+687>
0x0000000000631211 <+545>: mov %ecx,%eax
0x0000000000631213 <+547>: sub 0x18(%rsp),%eax
0x0000000000631217 <+551>: mov %ecx,0x1c(%rsp)
0x000000000063121b <+555>: test %r13d,%r13d
0x000000000063121e <+558>: jne 0x6310ff <_ZNK3rfb6Region9get_rectsEPSt6vectorINS_4RectESaIS2_EEbbi+271>
0x0000000000631224 <+564>: jmp 0x6311b9 <_ZNK3rfb6Region9get_rectsEPSt6vectorINS_4RectESaIS2_EEbbi+457>
0x0000000000631226 <+566>: movl $0x1,0x18(%rsp)
0x000000000063122e <+574>: movl $0x0,0x1c(%rsp)
0x0000000000631236 <+582>: jmpq 0x631037 <_ZNK3rfb6Region9get_rectsEPSt6vectorINS_4RectESaIS2_EEbbi+71>
0x000000000063123b <+587>: test %rbp,%rbp
0x000000000063123e <+590>: je 0x63129b <_ZNK3rfb6Region9get_rectsEPSt6vectorINS_4RectESaIS2_EEbbi+683>
0x0000000000631240 <+592>: shl $0x4,%rbp
0x0000000000631244 <+596>: mov %r9d,(%rsp)
0x0000000000631248 <+600>: mov %rbp,%rdi
0x000000000063124b <+603>: callq 0x8c44f0 <_Znwm>
0x0000000000631250 <+608>: mov (%r12),%rdi
0x0000000000631254 <+612>: mov (%rsp),%r9d
0x0000000000631258 <+616>: mov %rax,%rbx
0x000000000063125b <+619>: test %rdi,%rdi
0x000000000063125e <+622>: je 0x63126d <_ZNK3rfb6Region9get_rectsEPSt6vectorINS_4RectESaIS2_EEbbi+637>
0x0000000000631260 <+624>: mov %r9d,(%rsp)
0x0000000000631264 <+628>: callq 0x8c4550 <_ZdlPv>
0x0000000000631269 <+633>: mov (%rsp),%r9d
0x000000000063126d <+637>: add %rbx,%rbp
0x0000000000631270 <+640>: test %r13d,%r13d
0x0000000000631273 <+643>: mov %rbx,(%r12)
0x0000000000631277 <+647>: mov %rbx,0x8(%r12)
0x000000000063127c <+652>: mov %rbp,0x10(%r12)
0x0000000000631281 <+657>: mov %rbx,%rsi
0x0000000000631284 <+660>: jg 0x631077 <_ZNK3rfb6Region9get_rectsEPSt6vectorINS_4RectESaIS2_EEbbi+135>
0x000000000063128a <+666>: add $0x48,%rsp
0x000000000063128e <+670>: xor %eax,%eax
0x0000000000631290 <+672>: pop %rbx
0x0000000000631291 <+673>: pop %rbp
0x0000000000631292 <+674>: pop %r12
0x0000000000631294 <+676>: pop %r13
0x0000000000631296 <+678>: pop %r14
0x0000000000631298 <+680>: pop %r15
0x000000000063129a <+682>: retq
0x000000000063129b <+683>: xor %ebx,%ebx
0x000000000063129d <+685>: jmp 0x63125b <_ZNK3rfb6Region9get_rectsEPSt6vectorINS_4RectESaIS2_EEbbi+619>
0x000000000063129f <+687>: mov 0x1c(%rsp),%eax
0x00000000006312a3 <+691>: mov %ecx,0x1c(%rsp)
0x00000000006312a7 <+695>: jmpq 0x63121b <_ZNK3rfb6Region9get_rectsEPSt6vectorINS_4RectESaIS2_EEbbi+555>
0x00000000006312ac <+700>: lea 0x33b6d0(%rip),%rdi # 0x96c983
0x00000000006312b3 <+707>: callq 0x8c7720 <_ZSt20__throw_length_errorPKc>
End of assembler dump.
(gdb) info reg
rax 0x0 0
rbx 0x0 0
rcx 0x21 33
rdx 0x0 0
rsi 0x1d32eb0 30617264
rdi 0x1d35c60 30628960
rbp 0x1d32ec0 0x1d32ec0
rsp 0x7fffffff5d50 0x7fffffff5d50
r8 0x7 7
r9 0xda9a98 14326424
r10 0x7 7
r11 0x1d35c60 30628960
r12 0x7fffffff5df0 140737488313840
r13 0x1 1
r14 0xda9a98 14326424
r15 0x0 0
rip 0x631134 0x631134 <rfb::Region::get_rects(std::vector<rfb::Rect, std::allocator<rfb::Rect> >*, bool, bool, int) const+324>
eflags 0x10246 [ PF ZF IF RF ]
cs 0x33 51
ss 0x2b 43
ds 0x0 0
es 0x0 0
fs 0x0 0
gs 0x0 0I can produce it by opening a server, inside the server open a viewer to another server, inside that nested viewer I use an emacs and select some text.
Maybe you have an idea or you can suggest further experiments ?
stefvanvlierberghe
commented
4 years ago The failing insn can be found above : => 0x0000000000631134 <+324>: idiv %ebx
stefvanvlierberghe
commented
4 years ago vmware is not involved, just had the issue again on a non-virtualized workstation
stefvanvlierberghe
commented
4 years ago Getting closer with debuginfo installed. The problem is an empty rectangle in a vector of rectangles associated with an Xregion in Region.cxx, where this construct assumes the x-dimension of the rectangle is non-zero :
while (nRectsInBand > 0) {
int y = xrgn->rects[i].y1;
int h = maxArea / (xrgn->rects[i].x2 - xrgn->rects[i].x1);gdb shows this is not the case
(gdb) p this->xrgn->rects[0]
$9 = {
x1 = 7,
x2 = 7,
y1 = 24,
y2 = 24
}One frame up this is a copy construct that seems to have noticed the empty rectangle:
(gdb) info frame
Stack level 1, frame at 0x7fffffff66e0:
rip = 0x53c1b6 in rfb::ComparingUpdateTracker::compare (/usr/src/debug/tigervnc-1.8.0/common/rfb/ComparingUpdateTracker.cxx:73);
saved rip = 0x53a188
called by frame at 0x7fffffff67b0, caller of frame at 0x7fffffff6660
source language c++.
Arglist at 0x7fffffff6658, args: this=0x8a9af0
Locals at 0x7fffffff6658, Previous frame's sp is 0x7fffffff66e0
Saved registers:
rbx at 0x7fffffff66a8, rbp at 0x7fffffff66b0, r12 at 0x7fffffff66b8, r13 at 0x7fffffff66c0, r14 at 0x7fffffff66c8, r15 at 0x7fffffff66d0,
rip at 0x7fffffff66d8
(gdb) p this->copy_delta
$12 = {
x = 0,
y = 0
}but it called changed.get_rects anyway:
copied.get_rects(&rects, copy_delta.x<=0, copy_delta.y<=0);
for (i = rects.begin(); i != rects.end(); i++)
oldFb.copyRect(*i, copy_delta);
changed.get_rects(&rects);Right, the system assumes it won't be fed empty rects. So the bug is in vncHooks.c. If you can put a check in add_changed() and add_copied() there you should be able to figure out where it is coming from.
stefvanvlierberghe
commented
4 years ago Thanks, Pierre.
I’m on a breakpoint now in add_changed adding a zero x-dimension rect :
void vncAddChanged(int scrIdx, const struct UpdateRect extents, int nRects, const struct UpdateRect rects) { Region reg;
reg.setExtentsAndOrderedRects((const ShortRect)extents, nRects, (const ShortRect)rects); desktop[scrIdx]->add_changed(reg); }
Three non-singular rects go in :
(gdb) p nRects $28 = 3 … (gdb) p rects[0] $31 = { x1 = 41, y1 = 24, x2 = 1287, y2 = 48 } (gdb) p rects[1] $32 = { x1 = 41, y1 = 48, x2 = 48, y2 = 1185 } (gdb) p rects[2] $33 = { x1 = 1280, y1 = 48, x2 = 1287, y2 = 1185 }
And the resulting reg.xrgn.extents is x-singular: (gdb) p *reg.xrgn $36 = { size = 3, numRects = 3, rects = 0xe4dff0, extents = { x1 = 1287, x2 = 1287, y1 = 24, y2 = 1185 } }
This message and any files transmitted with it are legally privileged and intended for the sole use of the individual(s) or entity to whom they are addressed. If you are not the intended recipient, please notify the sender by reply and delete the message and any attachments from your system. Any unauthorised use or disclosure of the content of this message is strictly prohibited and may be unlawful.
Nothing in this e-mail message amounts to a contractual or legal commitment on the part of EUROCONTROL, unless it is confirmed by appropriately signed hard copy.
Any views expressed in this message are those of the sender.
CendioOssman
commented
4 years ago We're mostly interested in where it comes from. Can you do a backtrace where that hits?
balwierz
commented
4 years ago Hi. I am the person who opened this issue. Since original posting I didn't reproduce this bug. I can try if you can narrow down events which may result in zero-size rectangles being copied.
stefvanvlierberghe
commented
4 years ago Sorry, was busy for a while, this is the backtrace where add_changed was called with extents.x1=extents.x2 :
(gdb) p *region.xrgn $3 = ( size => 3, numRects => 3, rects => 0xe4dff0, extents => ( x1 => 1287, x2 => 1287, y1 => 24, y2 => 1185 ) )
(gdb) bt
at vncHooks.c:929at /usr/src/debug/tigervnc-1.8.0/common/rfb/VNCSConnectionST.cxx:643at /usr/src/debug/tigervnc-1.8.0/common/rfb/VNCSConnectionST.cxx:168write=write@entry=false) at XserverDesktop.cc:459at XserverDesktop.cc:408rtld_fini=<optimized out>, stack_end=0x7fffffff5cc8) at ../csu/libc-start.c:266From: Pierre Ossman (Work account) notifications@github.com Sent: 14 February 2020 15:32 To: TigerVNC/tigervnc tigervnc@noreply.github.com Cc: VAN VLIERBERGHE Stef stef.van-vlierberghe@eurocontrol.int; Comment comment@noreply.github.com Subject: Re: [TigerVNC/tigervnc] Floating point exception while resizing a window on Debian Buster (#846)
We're mostly interested in where it comes from. Can you do a backtrace where that hits?
— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/TigerVNC/tigervnc/issues/846?email_source=notifications&email_token=AH5WRHHOTTG26AACAJVD7STRC2TMRA5CNFSM4HYRMENKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOELZGYGY#issuecomment-586312731, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AH5WRHGDEEE7227BJHIYZNDRC2TMRANCNFSM4HYRMENA.
This message and any files transmitted with it are legally privileged and intended for the sole use of the individual(s) or entity to whom they are addressed. If you are not the intended recipient, please notify the sender by reply and delete the message and any attachments from your system. Any unauthorised use or disclosure of the content of this message is strictly prohibited and may be unlawful.
Nothing in this e-mail message amounts to a contractual or legal commitment on the part of EUROCONTROL, unless it is confirmed by appropriately signed hard copy.
Any views expressed in this message are those of the sender.
stefvanvlierberghe
commented
4 years ago Another difficulty is that I still cannot systematically reproduce the issue, sometimes it crashes after a few seconds sometimes it survives…
This message and any files transmitted with it are legally privileged and intended for the sole use of the individual(s) or entity to whom they are addressed. If you are not the intended recipient, please notify the sender by reply and delete the message and any attachments from your system. Any unauthorised use or disclosure of the content of this message is strictly prohibited and may be unlawful.
Nothing in this e-mail message amounts to a contractual or legal commitment on the part of EUROCONTROL, unless it is confirmed by appropriately signed hard copy.
Any views expressed in this message are those of the sender.
stefvanvlierberghe
commented
4 years ago Hi Pierre,
Can’t make it crash any longer, but I can easily get in gdb on a breakpoint which shows wrong extents :
(gdb) p *reg
$109 = {
extents = {
x1 = 1332,
y1 = 36,
x2 = 1332,
y2 = 36
},
data = 0x8561c0
Maybe adding some self-checking code could help identify the origins of such singular rectangles ?
Otherwise, if you can’t figure out where such values originate, an extra condition to skip singular values could avoid the crashes. This is not ideal, I suppose it would be nice and more efficient to never create such rectangles, but it is surely better than the divide by zero…
(gdb) bt
xoff=xoff@entry=0, yoff=yoff@entry=0, pClipPict=0xc99c80) at mirect.c:77rtld_fini=<optimized out>, stack_end=0x7fffffff5cc8) at ../csu/libc-start.c:266From: Pierre Ossman (Work account) notifications@github.com Sent: 14 February 2020 15:32 To: TigerVNC/tigervnc tigervnc@noreply.github.com Cc: VAN VLIERBERGHE Stef stef.van-vlierberghe@eurocontrol.int; Comment comment@noreply.github.com Subject: Re: [TigerVNC/tigervnc] Floating point exception while resizing a window on Debian Buster (#846)
We're mostly interested in where it comes from. Can you do a backtrace where that hits?
— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/TigerVNC/tigervnc/issues/846?email_source=notifications&email_token=AH5WRHHOTTG26AACAJVD7STRC2TMRA5CNFSM4HYRMENKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOELZGYGY#issuecomment-586312731, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AH5WRHGDEEE7227BJHIYZNDRC2TMRANCNFSM4HYRMENA.
This message and any files transmitted with it are legally privileged and intended for the sole use of the individual(s) or entity to whom they are addressed. If you are not the intended recipient, please notify the sender by reply and delete the message and any attachments from your system. Any unauthorised use or disclosure of the content of this message is strictly prohibited and may be unlawful.
Nothing in this e-mail message amounts to a contractual or legal commitment on the part of EUROCONTROL, unless it is confirmed by appropriately signed hard copy.
Any views expressed in this message are those of the sender.
stefvanvlierberghe
commented
4 years ago Hi Pierre,
FYI, we are also getting feedback from RedHat support about this issue :
The coredumps match exactly the issue reported at
https://bugzilla.redhat.com/show_bug.cgi?id=1753158 and
https://bugzilla.redhat.com/show_bug.cgi?id=1670342
(tigervnc crashes whenever xfreerdp client is closed (faults in DamageUnregister unless backingstore is disabled))
A fix should be available soon, and requires updating to
xorg-x11-server-1.20.4-9.el7 (released) and tigervnc-1.8.0-18.el7 (not yet released).
Until the new tigervnc is released, the know workarounds are to downgrade to tigervnc-1.8.0-5.el7 or use the -bs option, for example:
This message and any files transmitted with it are legally privileged and intended for the sole use of the individual(s) or entity to whom they are addressed. If you are not the intended recipient, please notify the sender by reply and delete the message and any attachments from your system. Any unauthorised use or disclosure of the content of this message is strictly prohibited and may be unlawful.
Nothing in this e-mail message amounts to a contractual or legal commitment on the part of EUROCONTROL, unless it is confirmed by appropriately signed hard copy.
Any views expressed in this message are those of the sender.
bphinz
commented
4 years ago (tigervnc crashes whenever xfreerdp client > is closed (faults in DamageUnregister unless backingstore is disabled))
That’s interesting because it sounds like #368, which we’ve never been able to pin down the exact cause of.
-brian
On Fri, Feb 14, 2020 at 4:00 PM stefvanvlierberghe notifications@github.com wrote:
Hi Pierre,
FYI, we are also getting feedback from RedHat support about this issue :
The coredumps match exactly the issue reported at
https://bugzilla.redhat.com/show_bug.cgi?id=1753158 and
https://bugzilla.redhat.com/show_bug.cgi?id=1670342
(tigervnc crashes whenever xfreerdp client is closed (faults in DamageUnregister unless backingstore is disabled))
A fix should be available soon, and requires updating to
xorg-x11-server-1.20.4-9.el7 (released) and tigervnc-1.8.0-18.el7 (not yet released).
Until the new tigervnc is released, the know workarounds are to downgrade to tigervnc-1.8.0-5.el7 or use the -bs option, for example:
vncserver :5 -bs
This message and any files transmitted with it are legally privileged and intended for the sole use of the individual(s) or entity to whom they are addressed. If you are not the intended recipient, please notify the sender by reply and delete the message and any attachments from your system. Any unauthorised use or disclosure of the content of this message is strictly prohibited and may be unlawful.
Nothing in this e-mail message amounts to a contractual or legal commitment on the part of EUROCONTROL, unless it is confirmed by appropriately signed hard copy.
Any views expressed in this message are those of the sender.
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/TigerVNC/tigervnc/issues/846?email_source=notifications&email_token=AB45M3J2TN5VM7TS6OZJNELRC4A7TA5CNFSM4HYRMENKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEL2NKGI#issuecomment-586470681, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB45M3IQHHGXYAWEOMJLUQLRC4A7TANCNFSM4HYRMENA .
-- Sent from Gmail Mobile
CendioOssman
commented
4 years ago Does this patch help:
diff --git a/unix/xserver/hw/vnc/vncHooks.c b/unix/xserver/hw/vnc/vncHooks.c
index 5cf2c0d1..d3428788 100644
--- a/unix/xserver/hw/vnc/vncHooks.c
+++ b/unix/xserver/hw/vnc/vncHooks.c
@@ -388,6 +388,8 @@ static inline void add_changed(ScreenPtr pScreen, RegionPtr reg)
vncHooksScreenPtr vncHooksScreen = vncHooksScreenPrivate(pScreen);
if (vncHooksScreen->ignoreHooks)
return;
+ if (REGION_NIL(reg))
+ return;
vncAddChanged(pScreen->myNum,
(const struct UpdateRect*)REGION_EXTENTS(pScreen, reg),
REGION_NUM_RECTS(reg),
@@ -400,6 +402,8 @@ static inline void add_copied(ScreenPtr pScreen, RegionPtr dst,
vncHooksScreenPtr vncHooksScreen = vncHooksScreenPrivate(pScreen);
if (vncHooksScreen->ignoreHooks)
return;
+ if (REGION_NIL(dst))
+ return;
vncAddCopied(pScreen->myNum,
(const struct UpdateRect*)REGION_EXTENTS(pScreen, dst),
REGION_NUM_RECTS(dst),
@@ -562,8 +566,7 @@ static void vncHooksCopyWindow(WindowPtr pWin, DDXPointRec ptOldOrg,
(*pScreen->CopyWindow) (pWin, ptOldOrg, pOldRegion);
- if (REGION_NOTEMPTY(pScreen, &copied))
- add_copied(pScreen, &copied, dx, dy);
+ add_copied(pScreen, &copied, dx, dy);
REGION_UNINIT(pScreen, &copied);
REGION_UNINIT(pScreen, &screen_rgn);
@@ -807,8 +810,7 @@ static void vncHooksComposite(CARD8 op, PicturePtr pSrc, PicturePtr pMask,
(*ps->Composite)(op, pSrc, pMask, pDst, xSrc, ySrc,
xMask, yMask, xDst, yDst, width, height);
- if (REGION_NOTEMPTY(pScreen, &changed))
- add_changed(pScreen, &changed);
+ add_changed(pScreen, &changed);
REGION_UNINIT(pScreen, &changed);
@@ -910,8 +912,7 @@ static void vncHooksGlyphs(CARD8 op, PicturePtr pSrc, PicturePtr pDst,
(*ps->Glyphs)(op, pSrc, pDst, maskFormat, xSrc, ySrc, nlists, lists, glyphs);
- if (REGION_NOTEMPTY(pScreen, changed))
- add_changed(pScreen, changed);
+ add_changed(pScreen, changed);
REGION_DESTROY(pScreen, changed);
@@ -933,8 +934,7 @@ static void vncHooksCompositeRects(CARD8 op, PicturePtr pDst,
(*ps->CompositeRects)(op, pDst, color, nRect, rects);
- if (REGION_NOTEMPTY(pScreen, changed))
- add_changed(pScreen, changed);
+ add_changed(pScreen, changed);
REGION_DESTROY(pScreen, changed);
@@ -1001,8 +1001,7 @@ static void vncHooksTrapezoids(CARD8 op, PicturePtr pSrc, PicturePtr pDst,
(*ps->Trapezoids)(op, pSrc, pDst, maskFormat, xSrc, ySrc, ntrap, traps);
- if (REGION_NOTEMPTY(pScreen, &changed))
- add_changed(pScreen, &changed);
+ add_changed(pScreen, &changed);
REGION_UNINIT(pScreen, &changed);
@@ -1067,8 +1066,7 @@ static void vncHooksTriangles(CARD8 op, PicturePtr pSrc, PicturePtr pDst,
(*ps->Triangles)(op, pSrc, pDst, maskFormat, xSrc, ySrc, ntri, tris);
- if (REGION_NOTEMPTY(pScreen, &changed))
- add_changed(pScreen, &changed);
+ add_changed(pScreen, &changed);
REGION_UNINIT(pScreen, &changed);
@@ -1128,8 +1126,7 @@ static void vncHooksTriStrip(CARD8 op, PicturePtr pSrc, PicturePtr pDst,
(*ps->TriStrip)(op, pSrc, pDst, maskFormat, xSrc, ySrc, npoint, points);
- if (REGION_NOTEMPTY(pScreen, &changed))
- add_changed(pScreen, &changed);
+ add_changed(pScreen, &changed);
REGION_UNINIT(pScreen, &changed);
@@ -1187,8 +1184,7 @@ static void vncHooksTriFan(CARD8 op, PicturePtr pSrc, PicturePtr pDst,
(*ps->TriFan)(op, pSrc, pDst, maskFormat, xSrc, ySrc, npoint, points);
- if (REGION_NOTEMPTY(pScreen, &changed))
- add_changed(pScreen, &changed);
+ add_changed(pScreen, &changed);
REGION_UNINIT(pScreen, &changed);
@@ -1509,13 +1505,11 @@ static RegionPtr vncHooksCopyArea(DrawablePtr pSrc, DrawablePtr pDst,
ret = (*pGC->ops->CopyArea) (pSrc, pDst, pGC, srcx, srcy, w, h, dstx, dsty);
- if (REGION_NOTEMPTY(pScreen, &dst))
- add_copied(pGC->pScreen, &dst,
- dstx + pDst->x - srcx - pSrc->x,
- dsty + pDst->y - srcy - pSrc->y);
+ add_copied(pGC->pScreen, &dst,
+ dstx + pDst->x - srcx - pSrc->x,
+ dsty + pDst->y - srcy - pSrc->y);
- if (REGION_NOTEMPTY(pScreen, &changed))
- add_changed(pGC->pScreen, &changed);
+ add_changed(pGC->pScreen, &changed);
REGION_UNINIT(pGC->pScreen, &dst);
REGION_UNINIT(pGC->pScreen, &src);It should cover the simple cases more consistently at least.
stefvanvlierberghe
commented
4 years ago Hi Pierre,
Thanks for your help, but unfortunately I have not been able any more to reproduce the issue, it went away as mysteriously as it came… I also failed to recompile Xvnc from sources, I made a request to get the rhel7 X sources installed which will hopefully make this possible. Will try your patch if/when I get the issue back.
P.S. I still think It would be good practice to check that x2 != x1 before dividing by (x2-x1), maybe you can think of some “central” place where you could assert this invariant.
All the best,
Stef
This message and any files transmitted with it are legally privileged and intended for the sole use of the individual(s) or entity to whom they are addressed. If you are not the intended recipient, please notify the sender by reply and delete the message and any attachments from your system. Any unauthorised use or disclosure of the content of this message is strictly prohibited and may be unlawful.
Nothing in this e-mail message amounts to a contractual or legal commitment on the part of EUROCONTROL, unless it is confirmed by appropriately signed hard copy.
Any views expressed in this message are those of the sender.
bphinz
commented
4 years ago I will try to test it with RTL Compiler next week since that was a very reproducible crash.
On Thu, Feb 27, 2020 at 1:35 PM stefvanvlierberghe notifications@github.com wrote:
Hi Pierre,
Thanks for your help, but unfortunately I have not been able any more to reproduce the issue, it went away as mysteriously as it came… I also failed to recompile Xvnc from sources, I made a request to get the rhel7 X sources installed which will hopefully make this possible. Will try your patch if/when I get the issue back.
P.S. I still think It would be good practice to check that x2 != x1 before dividing by (x2-x1), maybe you can think of some “central” place where you could assert this invariant.
All the best,
Stef
This message and any files transmitted with it are legally privileged and intended for the sole use of the individual(s) or entity to whom they are addressed. If you are not the intended recipient, please notify the sender by reply and delete the message and any attachments from your system. Any unauthorised use or disclosure of the content of this message is strictly prohibited and may be unlawful.
Nothing in this e-mail message amounts to a contractual or legal commitment on the part of EUROCONTROL, unless it is confirmed by appropriately signed hard copy.
Any views expressed in this message are those of the sender.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/TigerVNC/tigervnc/issues/846?email_source=notifications&email_token=AB45M3LRLEHNEZG2CXORJ4LRFABVJA5CNFSM4HYRMENKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOENFOKRY#issuecomment-592110919, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB45M3JQNNVS6ZPBXACST6DRFABVJANCNFSM4HYRMENA .
stefvanvlierberghe
commented
4 years ago Finally got this patch built from source on rhel7.7, took me 7 hours, if you are not familiar with building X from source then this is not an easy job. I ended up with the ugly contraption below, but at least I can use it now. We are not yet capable of reproducing the issue but one team member if getting it frequently, so hopefully we will find that this patched Xvnc no longer suffers from the SIGSEGV.
We hope you and Redhat can work together to produce a fixed rpm, because the below is really a bit of a mess.
~vvl/tigervnc/PAD is the patch you provided above.
Any advise about how to get a cleaner build procedure or official RedHat rpm is welcome.
mkdir vnc
cd vnc
mkdir prefix
prefix_dir=$(/bin/pwd)/prefix
mkdir exec_prefix
exec_prefix_dir=$(/bin/pwd)/exec_prefix
unzip ~vvl/tigervnc/tigervnc-master.20200413.zip
cd tigervnc-master
build_dir=$(/bin/pwd)
cmake -G "Unix Makefiles" ${build_dir}
make
cp -R /usr/share/xorg-x11-server-source/* unix/xserver/
patch -p1 < ~vvl/tigervnc/PAD
cd unix/xserver
patch -p1 < ${build_dir}/unix/xserver120.patch
autoreconf -fiv
./configure \
--prefix=${prefix_dir} --exec-prefix=${exec_prefix_dir} \
--disable-xorg --disable-xnest --disable-xvfb --disable-dmx \
--disable-xwin --disable-xephyr --disable-kdrive --with-pic \
--disable-static --disable-xwayland \
--with-default-font-path="/etc/X11/fontpath.d,built-ins" \
--with-fontdir=/usr/share/X11/fonts \
--with-xkb-output=${prefix_dir}/xkb \
--enable-install-libxf86config \
--enable-glx --disable-dri --enable-dri2 --disable-dri3 \
--disable-unit-tests \
--disable-config-hal \
--disable-config-udev \
--with-dri-driver-path=/usr/lib64/dri \
--without-dtrace \
--disable-devel-docs \
--enable-listen-tcp
make TIGERVNC_SRCDIR=${build_dir}
make install
cp -r /usr/share/X11 ${prefix_dir}/share/X11
ln -s /usr/bin/xkbcomp ${exec_prefix_dir}/bin
ln -s /usr/bin/vncpasswd ${exec_prefix_dir}/bin
cp /bin/vncserver ${exec_prefix_dir}/bin/vncserver
sed -i -e "s@\$exedir = \"\";@\$exedir = \"${exec_prefix_dir}/bin/\";@" ${exec_prefix_dir}/bin/vncserver
Great. Let us know how testing goes.
stefvanvlierberghe
commented
4 years ago Hi Pierre,
So far no more issues with the zero dimension divide by zero.
We did hit another issue (EE) (EE) Backtrace: (EE) 0: /auto/local_build/dhws028/disk1/vnc/exec_prefix/bin/Xvnc (OsSigHandler+0x29) [0x5c3169] (EE) 1: /lib64/libpthread.so.0 (_L_unlock_13+0x34) [0x7ffff7763663] (EE) 2: /auto/local_build/dhws028/disk1/vnc/exec_prefix/bin/Xvnc (DamageUnregister+0x10) [0x4d27f0] (EE) 3: /auto/local_build/dhws028/disk1/vnc/exec_prefix/bin/Xvnc (compSetParentPixmap+0x37) [0x517117] (EE) 4: /auto/local_build/dhws028/disk1/vnc/exec_prefix/bin/Xvnc (compFreeClientWindow+0x201) [0x517381] (EE) 5: /auto/local_build/dhws028/disk1/vnc/exec_prefix/bin/Xvnc (FreeCompositeClientWindow+0x9) [0x5120a9] (EE) 6: /auto/local_build/dhws028/disk1/vnc/exec_prefix/bin/Xvnc (doFreeResource+0x62) [0x595172] (EE) 7: /auto/local_build/dhws028/disk1/vnc/exec_prefix/bin/Xvnc (FreeResource+0xde) [0x595cae] (EE) 8: /auto/local_build/dhws028/disk1/vnc/exec_prefix/bin/Xvnc (compUnredirectWindow+0xb1) [0x5167b1] (EE) 9: /auto/local_build/dhws028/disk1/vnc/exec_prefix/bin/Xvnc (compChangeWindowAttributes+0x193) [0x513233] (EE) 10: /auto/local_build/dhws028/disk1/vnc/exec_prefix/bin/Xvnc (compDestroyWindow+0x179) [0x514c99] (EE) 11: /auto/local_build/dhws028/disk1/vnc/exec_prefix/bin/Xvnc (damageDestroyWindow+0x9e) [0x4d2a7e] (EE) 12: /auto/local_build/dhws028/disk1/vnc/exec_prefix/bin/Xvnc (DbeDestroyWindow+0xd0) [0x487670] (EE) 13: /auto/local_build/dhws028/disk1/vnc/exec_prefix/bin/Xvnc (present_destroy_window+0x24e) [0x4ccc0e] (EE) 14: /auto/local_build/dhws028/disk1/vnc/exec_prefix/bin/Xvnc (FreeWindowResources+0x114) [0x5999e4] (EE) 15: /auto/local_build/dhws028/disk1/vnc/exec_prefix/bin/Xvnc (DeleteWindow+0x236) [0x59c686] (EE) 16: /auto/local_build/dhws028/disk1/vnc/exec_prefix/bin/Xvnc (doFreeResource+0x62) [0x595172] (EE) 17: /auto/local_build/dhws028/disk1/vnc/exec_prefix/bin/Xvnc (FreeResource+0xde) [0x595cae] (EE) 18: /auto/local_build/dhws028/disk1/vnc/exec_prefix/bin/Xvnc (ProcDestroyWindow+0x77) [0x56cd77] (EE) 19: /auto/local_build/dhws028/disk1/vnc/exec_prefix/bin/Xvnc (Dispatch+0x31d) [0x57221d] (EE) 20: /auto/local_build/dhws028/disk1/vnc/exec_prefix/bin/Xvnc (dix_main+0x38a) [0x575f2a] (EE) 21: /lib64/libc.so.6 (__libc_start_main+0xf5) [0x7ffff4e19545] (EE) 22: /auto/local_build/dhws028/disk1/vnc/exec_prefix/bin/Xvnc (_start+0x29) [0x4568ce]
We also noticed that the above was frequently triggered when using other viewers running on Mac, but also triggered (just once) using tiger vncviewer.
The above seems to match : see : https://bugzilla.redhat.com/show_bug.cgi?id=1438898
So I suspect redhat has a patch in their tigervnc for that which was not applied in their x server sources. I also managed to rebuild their rpm from source with your patch (with a few less of the factorized/now obsolete checks before the calls to the 2 central checking procedures be removed). This also worked ok and did not yet reproduce the issue above, so rebuilding vnc from source on redhat should also include the redhat patches (which can be extracted from their src.rpm).
After having seen your patch I can’t imagine how it could do harm, so looks safe to commit and hope it finds its way to the redhat rpm.
Thanks for your help, and good health to you.
This message and any files transmitted with it are legally privileged and intended for the sole use of the individual(s) or entity to whom they are addressed. If you are not the intended recipient, please notify the sender by reply and delete the message and any attachments from your system. Any unauthorised use or disclosure of the content of this message is strictly prohibited and may be unlawful.
Nothing in this e-mail message amounts to a contractual or legal commitment on the part of EUROCONTROL, unless it is confirmed by appropriately signed hard copy.
Any views expressed in this message are those of the sender.
CendioOssman
commented
4 years ago That looks like #939, so let's please discuss that issue there. It is currently stalled waiting for a test case.
Thank you for testing the patch for this issue. I've committed it as f59e9649b06ce7cfa33ed217c9309cfa02b4853e so this should be fixed once we roll out a new release.
stefvanvlierberghe
commented
4 years ago Hi Pierre,
That other issue wasn’t produced again since we used a rebuild of the tigervnc-1.8.0-19.el7.src.rpm with your patch added.
Also we do have libjpeg-turbo installed, so don’t see the link with #939https://github.com/TigerVNC/tigervnc/issues/939 rpm -ql libjpeg-turbo-1.2.90-8.el7.x86_64 /usr/lib64/libjpeg.so.62 /usr/lib64/libjpeg.so.62.1.0
All the best, Stef
From: Pierre Ossman (Work account) notifications@github.com Sent: 20 April 2020 08:58 To: TigerVNC/tigervnc tigervnc@noreply.github.com Cc: VAN VLIERBERGHE Stef stef.van-vlierberghe@eurocontrol.int; Comment comment@noreply.github.com Subject: Re: [TigerVNC/tigervnc] Floating point exception while resizing a window on Debian Buster (#846)
That looks like #939https://github.com/TigerVNC/tigervnc/issues/939, so let's please discuss that issue there. It is currently stalled waiting for a test case.
Thank you for testing the patch for this issue. I've committed it as f59e964https://github.com/TigerVNC/tigervnc/commit/f59e9649b06ce7cfa33ed217c9309cfa02b4853e so this should be fixed once we roll out a new release.
— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/TigerVNC/tigervnc/issues/846#issuecomment-616350062, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AH5WRHFB5ZCOAO6SA4ZFTGTRNPW7VANCNFSM4HYRMENA.
This message and any files transmitted with it are legally privileged and intended for the sole use of the individual(s) or entity to whom they are addressed. If you are not the intended recipient, please notify the sender by reply and delete the message and any attachments from your system. Any unauthorised use or disclosure of the content of this message is strictly prohibited and may be unlawful.
Nothing in this e-mail message amounts to a contractual or legal commitment on the part of EUROCONTROL, unless it is confirmed by appropriately signed hard copy.
Any views expressed in this message are those of the sender.
CendioOssman
commented
4 years ago Sorry, wrong issue. It was supposed to be #979.
stefvanvlierberghe
commented
4 years ago Sorry to bring bad news but the fix of Feb 27th above does not suffice, I just got the issue again with the patch included (I double checked).
This is what I see in gdb :
stefvanvlierberghe
commented
4 years ago (gdb) fr 7
#7 0x00000000005387dd in rfb::Region::get_rects (this=this@entry=0x8aaab8, rects=rects@entry=0x7fffffff3e00, left2right=left2right@entry=true, topdown=topdown@entry=true, maxArea=maxArea@entry=0)
at /auto/local_build/dhws029/disk1/vncrpm.19/BUILD/tigervnc-1.8.0/common/rfb/Region.cxx:214
(gdb) p *(xrgn->rects)
$1 = {
x1 = 265,
x2 = 265,
y1 = 50,
y2 = 50
}
(gdb) up
#8 0x000000000053c5e6 in rfb::ComparingUpdateTracker::compare (this=0x8aaab0) at /auto/local_build/dhws029/disk1/vncrpm.19/BUILD/tigervnc-1.8.0/common/rfb/ComparingUpdateTracker.cxx:73
(gdb) p *(*changed.xrgn).rects
$2 = {
x1 = 265,
x2 = 265,
y1 = 50,
y2 = 50
}
(gdb) So singular in both dimensions and this .changed was definitely not added by the patched function:
static inline void add_changed(ScreenPtr pScreen, RegionPtr reg)
{
vncHooksScreenPtr vncHooksScreen = vncHooksScreenPrivate(pScreen);
if (vncHooksScreen->ignoreHooks)
return;
if (REGION_NIL(reg))
return;
vncAddChanged(pScreen->myNum,
(const struct UpdateRect*)REGION_EXTENTS(pScreen, reg),
REGION_NUM_RECTS(reg),
(const struct UpdateRect*)REGION_RECTS(reg));
}The core dump happened when I was typing a mail in Outlook running in an xfreerdp session which was itself running in the tigervnc (which I've done a lot for years).
What also might have had an influence is that I was using the option -compareFB=1 after somebody suggested that when teleworking over a WAN it was better to spend some more CPU in the server than to waste bandwidth by changes that could have been optimized out.
I looked around a bit in the code (not understanding most of it) but possibly these singular rectangles are produced by this bit of code that looks to implement this compareFB:
Region newChanged;
for (i = rects.begin(); i != rects.end(); i++)
compareRect(*i, &newChanged);
changed.get_rects(&rects);
for (i = rects.begin(); i != rects.end(); i++)
totalPixels += i->area();
newChanged.get_rects(&rects);
for (i = rects.begin(); i != rects.end(); i++)
missedPixels += i->area();
if (changed.equals(newChanged))
return false;
changed = newChanged;That last assignment to changed did not explicitly check for singular rectangles, so could that be another source of the same divide by zero ?
Was also wondering if any methods inherited from SimpleUpdateTracker could directly access the changes member and produce singular rectanges (I'm an Ada developer with limited C++ knowledge).
Also, could you think of a way to avoid the divide by zero as long as there is no proof of the absence of singular rectangles (which looks very far away to me) ? It seems to me that a simple protection if (xrgn->rects[i].x2 - xrgn->rects[i].x1) around the construct below would act as if the singular rectangle xrgn->rects[i] was not present 👍
int h = maxArea / (xrgn->rects[i].x2 - xrgn->rects[i].x1);
if (!h) h = xrgn->rects[i].y2 - y;
do {
if (h > xrgn->rects[i].y2 - y)
h = xrgn->rects[i].y2 - y;
Rect r(xrgn->rects[i].x1, y, xrgn->rects[i].x2, y+h);
rects->push_back(r);
y += h;
} while (y < xrgn->rects[i].y2);This protection would destroy the "feedback loop" which uses the reported crashes as a mechanism to detect the presence of singular rectangles, but this is a lot of trouble for the user providing marginal benefit for the development. Producing a warning in the log (once) would be an alternative (although I understand people will not be motivated to report such warnings). For me robustness is far more important than the desire to get a solution that gets as close to perfection as possible based on crash reports.
If you want I can upload build and core dump (but the core is 1.2 Gb).
Awaiting advice I will stop using -compareFB=1
stefvanvlierberghe
commented
4 years ago Alas, just had another divide by zero similar to the above.
My boss also had the same crash with the compareFB=1 active and he remarked that the issues always seem to occur when there is another layer of pixel buffering present. My most recent crash was also triggered by using a vncviewer inside the vncviewer and many earlier failures happened using vncviewer or xfreerdp inside the vncviewer...
Maybe that means something to you ?
stefvanvlierberghe
commented
4 years ago I tried to implement a detector/repearer combination in Region.cxx, runs ok so far. But I would like to call xorg_backtrace to show where the singular rectangle originates, this compiles but I get an unresolved reference in the linker. Advice about where I would need to add which lib would be welcome.
Below the patch I'm testing now :
*** tigervnc-1.8.0/common/rfb/Region.h.org Sun Apr 26 02:09:07 2020
--- tigervnc-1.8.0/common/rfb/Region.h Sun Apr 26 02:09:12 2020
***************
*** 73,78 ****
--- 73,79 ----
Rect get_bounding_rect() const;
void debug_print(const char *prefix) const;
+ void check_for_singular_rectangles();
protected:
*** tigervnc-1.8.0/common/rfb/ComparingUpdateTracker.cxx.org Sun Apr 26 02:20:13 2020
--- tigervnc-1.8.0/common/rfb/ComparingUpdateTracker.cxx Sun Apr 26 02:27:42 2020
***************
*** 1,15 ****
/* Copyright (C) 2002-2005 RealVNC Ltd. All Rights Reserved.
! *
* This is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
! *
* This software is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
! *
* You should have received a copy of the GNU General Public License
* along with this software; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307,
--- 1,15 ----
/* Copyright (C) 2002-2005 RealVNC Ltd. All Rights Reserved.
! *
* This is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
! *
* This software is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
! *
* You should have received a copy of the GNU General Public License
* along with this software; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307,
***************
*** 32,37 ****
--- 32,38 ----
enabled(true), totalPixels(0), missedPixels(0)
{
changed.assign_union(fb->getRect());
+ changed.check_for_singular_rectangles();
}
ComparingUpdateTracker::~ComparingUpdateTracker()
***************
*** 70,81 ****
--- 71,84 ----
for (i = rects.begin(); i != rects.end(); i++)
oldFb.copyRect(*i, copy_delta);
+ changed.check_for_singular_rectangles();
changed.get_rects(&rects);
Region newChanged;
for (i = rects.begin(); i != rects.end(); i++)
compareRect(*i, &newChanged);
+ changed.check_for_singular_rectangles();
changed.get_rects(&rects);
for (i = rects.begin(); i != rects.end(); i++)
totalPixels += i->area();
***************
*** 87,92 ****
--- 90,96 ----
return false;
changed = newChanged;
+ changed.check_for_singular_rectangles();
return true;
}
*** tigervnc-1.8.0/common/rfb/Region.cxx.org Sun Apr 26 02:07:18 2020
--- tigervnc-1.8.0/common/rfb/Region.cxx Sun Apr 26 02:32:35 2020
***************
*** 145,162 ****
--- 145,166 ----
void rfb::Region::copyFrom(const rfb::Region& r) {
XUnionRegion(r.xrgn, r.xrgn, xrgn);
+ check_for_singular_rectangles();
}
void rfb::Region::assign_intersect(const rfb::Region& r) {
XIntersectRegion(xrgn, r.xrgn, xrgn);
+ check_for_singular_rectangles();
}
void rfb::Region::assign_union(const rfb::Region& r) {
XUnionRegion(xrgn, r.xrgn, xrgn);
+ check_for_singular_rectangles();
}
void rfb::Region::assign_subtract(const rfb::Region& r) {
XSubtractRegion(xrgn, r.xrgn, xrgn);
+ check_for_singular_rectangles();
}
rfb::Region rfb::Region::intersect(const rfb::Region& r) const {
***************
*** 250,252 ****
--- 254,290 ----
xrgn->rects[i].y2-xrgn->rects[i].y1);
}
}
+
+
+ extern void xorg_backtrace(void);
+ //?? Not sure how to include tigervnc-master/unix/xserver/include/os.h
+ //?? Not sure how to call a backtrace here, make fails Linking CXX executable x0vncserver
+ //? Region.cxx:282: undefined reference to `xorg_backtrace()'
+
+ void rfb::Region::check_for_singular_rectangles() {
+ int Number_Of_Singular = 0;
+ for (int i = 0; i < xrgn->numRects; i++) {
+ if (( xrgn->rects[i].x1 >= xrgn->rects[i].x2 ) || ( xrgn->rects[i].y1 >= xrgn->rects[i].y2 ))
+ { Number_Of_Singular++;
+ // This rectangle is singular, remove it
+ fprintf (stderr,
+ "Region::check_for_singular_rectangles skipping x1=%d, x2=%d, y1=%d, y2=%d\n",
+ xrgn->rects[i].x1,
+ xrgn->rects[i].x2,
+ xrgn->rects[i].y1,
+ xrgn->rects[i].y2);
+ }
+ else if ( Number_Of_Singular > 0 )
+ { // This is a non-sigular rectangle preceded by singular ones, needs to be copied to an earlier component
+ xrgn->rects[i-Number_Of_Singular].x1 = xrgn->rects[i].x1;
+ xrgn->rects[i-Number_Of_Singular].x2 = xrgn->rects[i].x2;
+ xrgn->rects[i-Number_Of_Singular].y1 = xrgn->rects[i].y1;
+ xrgn->rects[i-Number_Of_Singular].y2 = xrgn->rects[i].y2;
+ }
+ }
+ if ( Number_Of_Singular > 0 )
+ {
+ xrgn->numRects = xrgn->numRects - Number_Of_Singular;
+ // xorg_backtrace();
+ }
+ }
*** tigervnc-1.8.0/common/rfb/UpdateTracker.cxx.org Sun Apr 26 02:30:40 2020
--- tigervnc-1.8.0/common/rfb/UpdateTracker.cxx Sun Apr 26 02:33:59 2020
***************
*** 1,15 ****
/* Copyright (C) 2002-2005 RealVNC Ltd. All Rights Reserved.
! *
* This is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
! *
* This software is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
! *
* You should have received a copy of the GNU General Public License
* along with this software; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307,
--- 1,15 ----
/* Copyright (C) 2002-2005 RealVNC Ltd. All Rights Reserved.
! *
* This is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
! *
* This software is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
! *
* You should have received a copy of the GNU General Public License
* along with this software; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307,
***************
*** 77,82 ****
--- 77,83 ----
void SimpleUpdateTracker::add_changed(const Region ®ion) {
changed.assign_union(region);
+ changed.check_for_singular_rectangles();
}
void SimpleUpdateTracker::add_copied(const Region &dest, const Point &delta) {
***************
*** 121,127 ****
Region invalid_src = overlap.intersect(changed);
invalid_src.translate(delta);
changed.assign_union(invalid_src);
!
overlap.translate(delta);
Region nonoverlapped_copied = dest.union_(copied).subtract(overlap);
--- 122,128 ----
Region invalid_src = overlap.intersect(changed);
invalid_src.translate(delta);
changed.assign_union(invalid_src);
!
overlap.translate(delta);
Region nonoverlapped_copied = dest.union_(copied).subtract(overlap);
***************
*** 142,147 ****
--- 143,150 ----
{
copied.assign_subtract(changed);
info->changed = changed.intersect(clip);
+ info->changed.check_for_singular_rectangles();
+
info->copied = copied.intersect(clip);
info->copy_delta = copy_delta;
}I've put a construct in the detect and repair function check_for_singular_rectangles to dump core and continue, which is a bit heavy but provides a lot more info than just a backtrace.
One of my Eurocontrol collegues had a core dump with this modification to the patch above, and now I understand better what was the root cause of this issue and why the patch did not protect us.
First the root cause (they may be others):
(gdb) bt
#0 0x00007ffff4bb24b9 in __libc_waitpid (pid=48240, stat_loc=stat_loc@entry=0x7fffffff9af0, options=options@entry=0)
at ../sysdeps/unix/sysv/linux/waitpid.c:40
#1 0x00007ffff4b2ff62 in do_system (line=line@entry=0x7fffffff9c90 "gcore -o core.mea.1588145193 17054") at ../sysdeps/posix/system.c:148
#2 0x00007ffff4b30311 in __libc_system (line=0x7fffffff9c90 "gcore -o core.mea.1588145193 17054") at ../sysdeps/posix/system.c:189
#3 0x0000000000538811 in rfb::Region::check_for_singular_rectangles (this=this@entry=0x8aaab8)
at /auto/local_build/dhws029/disk1/tigervnc.rpm.dir.19/BUILD/tigervnc-1.8.0/common/rfb/Region.cxx:295
#4 0x00000000005388bb in rfb::Region::assign_union (this=this@entry=0x8aaab8, r=...)
at /auto/local_build/dhws029/disk1/tigervnc.rpm.dir.19/BUILD/tigervnc-1.8.0/common/rfb/Region.cxx:162
#5 0x0000000000547cfd in rfb::SimpleUpdateTracker::add_changed (this=0x8aaab0, region=...)
at /auto/local_build/dhws029/disk1/tigervnc.rpm.dir.19/BUILD/tigervnc-1.8.0/common/rfb/UpdateTracker.cxx:79
#6 0x0000000000539a16 in rfb::VNCServerST::add_changed (this=0x8aa8d0, region=...)
at /auto/local_build/dhws029/disk1/tigervnc.rpm.dir.19/BUILD/tigervnc-1.8.0/common/rfb/VNCServerST.cxx:420
#7 0x000000000052a54e in XserverDesktop::add_changed (this=<optimized out>, region=...) at XserverDesktop.cc:383
#8 0x0000000000521227 in vncAddChanged (scrIdx=0, extents=extents@entry=0x7fffffffa180, nRects=1, rects=0x7fffffffa180) at vncExtInit.cc:368
#9 0x0000000000523c84 in add_changed (reg=0x7fffffffa180, pScreen=0x89ec40) at vncHooks.c:373
#10 vncHooksComposite (op=<optimized out>, pSrc=0xfd1740, pMask=0x0, pDst=<optimized out>, xSrc=<optimized out>, ySrc=<optimized out>, xMask=0,
yMask=0, xDst=0, yDst=0, width=0, height=0) at vncHooks.c:790
#11 0x00000000004d9b68 in damageComposite (op=<optimized out>, pSrc=0xfd1740, pMask=0x0, pDst=0x17588e0, xSrc=<optimized out>,
ySrc=<optimized out>, xMask=0, yMask=0, xDst=0, yDst=0, width=0, height=0) at damage.c:513
#12 0x00000000004cce4a in ProcRenderComposite (client=0x3eed070) at render.c:695
#13 0x00000000005741ad in Dispatch () at dispatch.c:478
#14 0x00000000005780aa in dix_main (argc=32, argv=0x7fffffffa4a8, envp=<optimized out>) at main.c:276
#15 0x00007ffff4b0f555 in __libc_start_main (main=0x455100 <main>, argc=32, argv=0x7fffffffa4a8, init=<optimized out>, fini=<optimized out>,
rtld_fini=<optimized out>, stack_end=0x7fffffffa498) at ../csu/libc-start.c:266
#16 0x00000000004565ee in _start ()
(gdb) When the Dispatch calls ProcRenderComposite the function receives a rather bizarre render request:
(gdb) p *stuff
$17 = {
reqType = 139 '\213',
renderReqType = 8 '\b',
length = 9,
op = 1 '\001',
pad1 = 0 '\000',
pad2 = 0,
src = 77595548,
mask = 0,
dst = 77595856,
xSrc = 0,
ySrc = 0,
xMask = 0,
yMask = 0,
xDst = 0,
yDst = 0,
width = 0,
height = 0
}The zero width and height is not being tested and this leads eventually to the singular rectangle resulting to the divide by zero in the official version, and to the repair and core dump in my version.
Below a gdb sesstion inspecting frames 4 to 10 (where, as usual, plenty of parameters are optimized out).
In frame 4 we can see the singular single rectangle region :
(gdb) up
#4 0x00000000005388bb in rfb::Region::assign_union (this=this@entry=0x8aaab8, r=...)
at /auto/local_build/dhws029/disk1/tigervnc.rpm.dir.19/BUILD/tigervnc-1.8.0/common/rfb/Region.cxx:162
(gdb) p *this
$1 = {
xrgn = 0x89fd70
}
(gdb) p this->xrgn
$2 = (_XRegion *) 0x89fd70
(gdb) p *(this->xrgn)
$3 = {
size = 23,
numRects = 0,
rects = 0x1691680,
extents = {
x1 = 576,
x2 = 576,
y1 = 138,
y2 = 138
}
}
(gdb) p *(this->xrgn).rects
$4 = {
x1 = 576,
x2 = 576,
y1 = 138,
y2 = 138
}
(gdb) p r
$5 = <optimized out>Then all is optimized out until we see the region again in frame 8 vncAddChanged
(gdb) up
#5 0x0000000000547cfd in rfb::SimpleUpdateTracker::add_changed (this=0x8aaab0, region=...)
at /auto/local_build/dhws029/disk1/tigervnc.rpm.dir.19/BUILD/tigervnc-1.8.0/common/rfb/UpdateTracker.cxx:79
(gdb) p region
$6 = <optimized out>
(gdb) up
#6 0x0000000000539a16 in rfb::VNCServerST::add_changed (this=0x8aa8d0, region=...)
at /auto/local_build/dhws029/disk1/tigervnc.rpm.dir.19/BUILD/tigervnc-1.8.0/common/rfb/VNCServerST.cxx:420
(gdb) p region
$7 = <optimized out>
(gdb) up
#7 0x000000000052a54e in XserverDesktop::add_changed (this=<optimized out>, region=...) at XserverDesktop.cc:383
(gdb) p region
$8 = <optimized out>
(gdb) up
#8 0x0000000000521227 in vncAddChanged (scrIdx=0, extents=extents@entry=0x7fffffffa180, nRects=1, rects=0x7fffffffa180) at vncExtInit.cc:368
(gdb) p reg
$9 = {
xrgn = 0x15fc7f0
}
(gdb) p reg.xrgn
$10 = (_XRegion *) 0x15fc7f0
(gdb) p *(reg.xrgn)
$11 = {
size = 1,
numRects = 1,
rects = 0xfd0940,
extents = {
x1 = 576,
x2 = 576,
y1 = 138,
y2 = 138
}
}
(gdb) p *rects
$12 = {
x1 = 576,
y1 = 138,
x2 = 576,
y2 = 138
}Here is gets interesting because in frame 9 we see the protection you added:
if (REGION_NIL(reg))
return;So this code clearly did not consider this region to be NIL
(gdb) up
#9 0x0000000000523c84 in add_changed (reg=0x7fffffffa180, pScreen=0x89ec40) at vncHooks.c:373
(gdb) p reg
$13 = (RegionPtr) 0x7fffffffa180
(gdb) p *reg
$14 = {
extents = {
x1 = 576,
y1 = 138,
x2 = 576,
y2 = 138
},
data = 0x0
}I guess the function calling the the fix attempt came from regionstr.h:
RegionNil(RegionPtr reg)
{
return ((reg)->data && !(reg)->data->numRects);
}
#define REGION_NIL RegionNilAnd clearly this code is not checking for singular rectangles, it just assumes that if the numRects is larger than zero (1 in our case) then all is well.
This also shows the challenge for this approach to fixing the issue : A region could contain any number of rectangles and some of these could be singular, in that case a protection simply discarding the entire region would lead to a partial rendering, while one accepting the entire region would lead to the divide by zero.
So I believe my detect and repair is a better approach as it discards only the singular rectangles.
(gdb) up
#10 vncHooksComposite (op=<optimized out>, pSrc=0xfd1740, pMask=0x0, pDst=<optimized out>, xSrc=<optimized out>, ySrc=<optimized out>, xMask=0,
yMask=0, xDst=0, yDst=0, width=0, height=0) at vncHooks.c:790I will proceed with building and testing another patch with:
stefvanvlierberghe
commented
4 years ago Hi Pierre, can you open the #846 again ?
I’ve added comments to explain what the root cause was, why the patch failed, how I repaired (a?) root cause and how to recover from possible other sources.
Will comment with a new patch that should fix everything (but as I said, I’m not a C++ programmer so hopefully you can review the changes and build a better patch).
All the best and good health, Stef
From: Pierre Ossman (Work account) notifications@github.com Sent: 20 April 2020 08:58 To: TigerVNC/tigervnc tigervnc@noreply.github.com Cc: VAN VLIERBERGHE Stef stef.van-vlierberghe@eurocontrol.int; Comment comment@noreply.github.com Subject: Re: [TigerVNC/tigervnc] Floating point exception while resizing a window on Debian Buster (#846)
Closed #846https://github.com/TigerVNC/tigervnc/issues/846.
— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/TigerVNC/tigervnc/issues/846#event-3250341980, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AH5WRHFEKPBA3BKV4VPBAN3RNPW7XANCNFSM4HYRMENA.
This message and any files transmitted with it are legally privileged and intended for the sole use of the individual(s) or entity to whom they are addressed. If you are not the intended recipient, please notify the sender by reply and delete the message and any attachments from your system. Any unauthorised use or disclosure of the content of this message is strictly prohibited and may be unlawful.
Nothing in this e-mail message amounts to a contractual or legal commitment on the part of EUROCONTROL, unless it is confirmed by appropriately signed hard copy.
Any views expressed in this message are those of the sender.
stefvanvlierberghe
commented
4 years ago The new patch as described above:
*** tigervnc-1.8.0/common/rfb/Region.h.org Sun Apr 26 02:09:07 2020
--- tigervnc-1.8.0/common/rfb/Region.h Sun Apr 26 02:09:12 2020
***************
*** 73,78 ****
--- 73,79 ----
Rect get_bounding_rect() const;
void debug_print(const char *prefix) const;
+ void check_for_singular_rectangles();
protected:
*** tigervnc-1.8.0/common/rfb/Region.cxx.org Sun Apr 26 02:07:18 2020
--- tigervnc-1.8.0/common/rfb/Region.cxx Thu Apr 30 00:19:43 2020
***************
*** 26,31 ****
--- 26,35 ----
#include <rfb/Region.h>
#include <assert.h>
#include <stdio.h>
+ #include <stdlib.h>
+ #include <time.h>
+ #include <unistd.h>
+
extern "C" {
#include <Xregion/Xlibint.h>
***************
*** 145,162 ****
--- 149,170 ----
void rfb::Region::copyFrom(const rfb::Region& r) {
XUnionRegion(r.xrgn, r.xrgn, xrgn);
+ check_for_singular_rectangles();
}
void rfb::Region::assign_intersect(const rfb::Region& r) {
XIntersectRegion(xrgn, r.xrgn, xrgn);
+ check_for_singular_rectangles();
}
void rfb::Region::assign_union(const rfb::Region& r) {
XUnionRegion(xrgn, r.xrgn, xrgn);
+ check_for_singular_rectangles();
}
void rfb::Region::assign_subtract(const rfb::Region& r) {
XSubtractRegion(xrgn, r.xrgn, xrgn);
+ check_for_singular_rectangles();
}
rfb::Region rfb::Region::intersect(const rfb::Region& r) const {
***************
*** 250,252 ****
--- 258,308 ----
xrgn->rects[i].y2-xrgn->rects[i].y1);
}
}
+
+
+ volatile int core_dump_taken = 0;
+ // Will be set 1 when a core dump is taken, to avoid multiple core dumps
+ //?? Maybe std::call_once() is a better implementation.
+
+ // extern void xorg_backtrace(void);
+ //?? Not sure how to include tigervnc-master/unix/xserver/include/os.h
+ //?? Not sure how to call a backtrace here, make fails Linking CXX executable x0vncserver
+ //?? Region.cxx:282: undefined reference to `xorg_backtrace()'
+ //?? For the time being using a core dump as this is better for analyzing the issue.
+
+ void rfb::Region::check_for_singular_rectangles() {
+ int Number_Of_Singular = 0;
+ for (int i = 0; i < xrgn->numRects; i++) {
+ if (( xrgn->rects[i].x1 >= xrgn->rects[i].x2 ) || ( xrgn->rects[i].y1 >= xrgn->rects[i].y2 ))
+ { Number_Of_Singular++;
+ // This rectangle is singular, remove it
+ fprintf (stderr,
+ "Region::check_for_singular_rectangles skipping x1=%d, x2=%d, y1=%d, y2=%d\n",
+ xrgn->rects[i].x1,
+ xrgn->rects[i].x2,
+ xrgn->rects[i].y1,
+ xrgn->rects[i].y2);
+ }
+ else if ( Number_Of_Singular > 0 )
+ { // This is a non-sigular rectangle preceded by singular ones, needs to be copied to an earlier component
+ xrgn->rects[i-Number_Of_Singular].x1 = xrgn->rects[i].x1;
+ xrgn->rects[i-Number_Of_Singular].x2 = xrgn->rects[i].x2;
+ xrgn->rects[i-Number_Of_Singular].y1 = xrgn->rects[i].y1;
+ xrgn->rects[i-Number_Of_Singular].y2 = xrgn->rects[i].y2;
+ }
+ }
+ if ( Number_Of_Singular > 0 )
+ { xrgn->numRects = xrgn->numRects - Number_Of_Singular; // Repair
+ if (! core_dump_taken)
+ { core_dump_taken = 1; // Too many core dumps could annoy the user, so we set this to 1 asap to minimize the risk other threads would core dump in parallel
+ { char dump_cmd[1000]; // Report by calling gcore to produce core.${LOGNAME}.<unix_timestamp>
+ char *logname = getenv ("LOGNAME");
+ sprintf (dump_cmd, "gcore -o core.%s.%d %d", getenv ("LOGNAME"), time(0), getpid());
+ fprintf (stderr, "Calling %s\n", dump_cmd);
+ { int result = system (dump_cmd);
+ fprintf (stderr, "Call to %s returned %d\n", dump_cmd, result);
+ }
+ }
+ }
+ }
+ }
*** tigervnc-1.8.0/common/rfb/ComparingUpdateTracker.cxx.org Sun Apr 26 02:20:13 2020
--- tigervnc-1.8.0/common/rfb/ComparingUpdateTracker.cxx Sun Apr 26 02:27:42 2020
***************
*** 32,37 ****
--- 32,38 ----
enabled(true), totalPixels(0), missedPixels(0)
{
changed.assign_union(fb->getRect());
+ changed.check_for_singular_rectangles();
}
ComparingUpdateTracker::~ComparingUpdateTracker()
***************
*** 70,81 ****
--- 71,84 ----
for (i = rects.begin(); i != rects.end(); i++)
oldFb.copyRect(*i, copy_delta);
+ changed.check_for_singular_rectangles();
changed.get_rects(&rects);
Region newChanged;
for (i = rects.begin(); i != rects.end(); i++)
compareRect(*i, &newChanged);
+ changed.check_for_singular_rectangles();
changed.get_rects(&rects);
for (i = rects.begin(); i != rects.end(); i++)
totalPixels += i->area();
***************
*** 87,92 ****
--- 90,96 ----
return false;
changed = newChanged;
+ changed.check_for_singular_rectangles();
return true;
}
*** tigervnc-1.8.0/common/rfb/UpdateTracker.cxx.org Sun Apr 26 02:30:40 2020
--- tigervnc-1.8.0/common/rfb/UpdateTracker.cxx Sun Apr 26 02:33:59 2020
***************
*** 77,82 ****
--- 77,83 ----
void SimpleUpdateTracker::add_changed(const Region ®ion) {
changed.assign_union(region);
+ changed.check_for_singular_rectangles();
}
void SimpleUpdateTracker::add_copied(const Region &dest, const Point &delta) {
***************
*** 121,127 ****
Region invalid_src = overlap.intersect(changed);
invalid_src.translate(delta);
changed.assign_union(invalid_src);
!
overlap.translate(delta);
Region nonoverlapped_copied = dest.union_(copied).subtract(overlap);
--- 122,128 ----
Region invalid_src = overlap.intersect(changed);
invalid_src.translate(delta);
changed.assign_union(invalid_src);
!
overlap.translate(delta);
Region nonoverlapped_copied = dest.union_(copied).subtract(overlap);
***************
*** 142,147 ****
--- 143,150 ----
{
copied.assign_subtract(changed);
info->changed = changed.intersect(clip);
+ info->changed.check_for_singular_rectangles();
+
info->copied = copied.intersect(clip);
info->copy_delta = copy_delta;
}
*** tigervnc-1.8.0/unix/xserver/miext/damage/damage.c.org Wed Apr 29 23:45:24 2020
--- tigervnc-1.8.0/unix/xserver/miext/damage/damage.c Thu Apr 30 00:33:10 2020
***************
*** 483,489 ****
INT16 xMask,
INT16 yMask,
INT16 xDst, INT16 yDst, CARD16 width, CARD16 height)
! {
ScreenPtr pScreen = pDst->pDrawable->pScreen;
PictureScreenPtr ps = GetPictureScreen(pScreen);
--- 483,491 ----
INT16 xMask,
INT16 yMask,
INT16 xDst, INT16 yDst, CARD16 width, CARD16 height)
! { if ( width == 0 || height == 0 ) return;
! /* see github.com-tigervnc-issues-846 some callers request rendering singular rectangles */
! {
ScreenPtr pScreen = pDst->pDrawable->pScreen;
PictureScreenPtr ps = GetPictureScreen(pScreen);
***************
*** 517,522 ****
--- 519,525 ----
xSrc, ySrc, xMask, yMask, xDst, yDst, width, height);
damageRegionProcessPending(pDst->pDrawable);
wrap(pScrPriv, ps, Composite, damageComposite);
+ }
}
static voidThanks for your hard work. It's a shame the first fix did not solve things.
It looks like REGION_INIT() is creating an invalid Region if fed an empty box. But I'm also surprised that REGION_INTERSECT() isn't culling those empty rects. Will have to dig more.
CendioOssman
commented
4 years ago Yeah, REGION_INIT() has no safety checks. Oddly enough is one of few things that haven't been delegated to pixman's region handling. And pixman_region_init_rect() does the right thing.
I think the best method is simply avoiding REGION_INIT() and using the newer RegionInitBoxes() instead.
stefvanvlierberghe
commented
4 years ago Hi Pierre,
Thanks for reopening #846.
Yes, the REGION_NIL(reg) checks didn’t protect, I would recommend to rollback that fix attempt.
I don’t see where this REGION_INIT() happens, for me the “main” fix is the first line in damageComposite :
damageComposite(CARD8 op, PicturePtr pSrc, PicturePtr pMask, PicturePtr pDst, INT16 xSrc, INT16 ySrc, INT16 xMask, INT16 yMask, INT16 xDst, INT16 yDst, CARD16 width, CARD16 height) { if ( width > 0 && height > 0 ) <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<MAIN FIX
If this were an Ada program I would have changed the type CARD16 to Positive to indicate that a caller passing zero for width or height must be doing something wrong.
But as far as I understand this is a message decoded from the X protocol, and therefore such a stricter convention would need to be enforced in the software writing these messages, which is unfortunately not a single implementation shared by all tools.
So it seemed to me that this protection would stop such “nonsensical” singleton rectangles are early as possible.
We have not had any new core dumps since the MAIN FIX above, and I don’t see how it could harm.
But I’d like to also keep the detect and repair code in case there would be yet another source of singular rectangles (possibly the system(gcore) can only be preserved in Linux implementations, or possibly there is a central implementation to do a core dump and survive), because from a user’s perspective losing the Xvnc and everything that was running in the desktop is a real nightmare.
Would that be acceptable for you ?
All the best, Stef
— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/TigerVNC/tigervnc/issues/846#issuecomment-622385015, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AH5WRHAQQ6MSVQ7NTNG66T3RPLEBJANCNFSM4HYRMENA.
This message and any files transmitted with it are legally privileged and intended for the sole use of the individual(s) or entity to whom they are addressed. If you are not the intended recipient, please notify the sender by reply and delete the message and any attachments from your system. Any unauthorised use or disclosure of the content of this message is strictly prohibited and may be unlawful.
Nothing in this e-mail message amounts to a contractual or legal commitment on the part of EUROCONTROL, unless it is confirmed by appropriately signed hard copy.
Any views expressed in this message are those of the sender.
CendioOssman
commented
4 years ago But I’d like to also keep the detect and repair code in case there would be yet another source of singular rectangles (possibly the system(gcore) can only be preserved in Linux implementations, or possibly there is a central implementation to do a core dump and survive), because from a user’s perspective losing the Xvnc and everything that was running in the desktop is a real nightmare. Would that be acceptable for you ?
The problem is that everything in Region is coded with the assumption that it doesn't have empty rects in it, so it is a bit like putting a band aid on a severed arm.
Please have a look at #1009. It should fix this issue more properly. It's a rather large diff though, so if it's too complex then hold off until it's been merged and you can try a nightly build.
DocMAX
commented
3 years ago I have that too. It happens when i open LibreOffice Draw
(EE) Backtrace:
(EE) 0: /usr/bin/Xvnc (xorg_backtrace+0x5b) [0x564b0885907b]
(EE) 1: /usr/bin/Xvnc (0x564b08687000+0x1d5a15) [0x564b0885ca15]
(EE) 2: /lib64/libpthread.so.0 (0x7f6d419a5000+0x121f0) [0x7f6d419b71f0]
(EE) 3: /usr/bin/Xvnc (_ZNK3rfb6Region9get_rectsEPSt6vectorINS_4RectESaIS2_EEbbi+0x146) [0x564b087cdad6]
(EE) 4: /usr/bin/Xvnc (_ZN3rfb22ComparingUpdateTracker7compareEv+0x1b5) [0x564b087d2275]
(EE) 5: /usr/bin/Xvnc (_ZN3rfb11VNCServerST11writeUpdateEv+0x1fd) [0x564b087d036d]
(EE) 6: /usr/bin/Xvnc (_ZN3rfb11VNCServerST13handleTimeoutEPNS_5TimerE+0x61) [0x564b087d0571]
(EE) 7: /usr/bin/Xvnc (_ZN3rfb5Timer13checkTimeoutsEv+0x8c) [0x564b087ce35c]
(EE) 8: /usr/bin/Xvnc (_ZN3rfb11VNCServerST13checkTimeoutsEv+0x17) [0x564b087cea47]
(EE) 9: /usr/bin/Xvnc (_ZN14XserverDesktop12blockHandlerEPi+0x253) [0x564b087c2733]
(EE) 10: /usr/bin/Xvnc (vncCallBlockHandlers+0x28) [0x564b087b6518]
(EE) 11: /usr/bin/Xvnc (BlockHandler+0x3e) [0x564b0880d1ee]
(EE) 12: /usr/bin/Xvnc (WaitForSomething+0x116) [0x564b08856d86]
(EE) 13: /usr/bin/Xvnc (Dispatch+0xa7) [0x564b088086d7]
(EE) 14: /usr/bin/Xvnc (dix_main+0x374) [0x564b0880c894]
(EE) 15: /lib64/libc.so.6 (__libc_start_main+0xcd) [0x7f6d410447ed]
(EE) 16: /usr/bin/Xvnc (_start+0x2a) [0x564b086f092a]
(EE)
(EE) Floating point exception at address 0x564b087cdad6
(EE)
Fatal server error:
(EE) Caught signal 8 (Floating point exception). Server aborting
It would be wise to report such issues together with the version of tigervnc you are using. The issue is fixed in the latest release so try using that one. All the best, Stef
From: DocMAX @.> Sent: 12 June 2021 15:45 To: TigerVNC/tigervnc @.> Cc: VAN VLIERBERGHE Stef @.>; Comment @.> Subject: Re: [TigerVNC/tigervnc] Floating point exception while resizing a window on Debian Buster (#846)
I have that too. It happens when i open LibreOffice Draw
(EE) Backtrace:
(EE) 0: /usr/bin/Xvnc (xorg_backtrace+0x5b) [0x564b0885907b]
(EE) 1: /usr/bin/Xvnc (0x564b08687000+0x1d5a15) [0x564b0885ca15]
(EE) 2: /lib64/libpthread.so.0 (0x7f6d419a5000+0x121f0) [0x7f6d419b71f0]
(EE) 3: /usr/bin/Xvnc (_ZNK3rfb6Region9get_rectsEPSt6vectorINS_4RectESaIS2_EEbbi+0x146) [0x564b087cdad6]
(EE) 4: /usr/bin/Xvnc (_ZN3rfb22ComparingUpdateTracker7compareEv+0x1b5) [0x564b087d2275]
(EE) 5: /usr/bin/Xvnc (_ZN3rfb11VNCServerST11writeUpdateEv+0x1fd) [0x564b087d036d]
(EE) 6: /usr/bin/Xvnc (_ZN3rfb11VNCServerST13handleTimeoutEPNS_5TimerE+0x61) [0x564b087d0571]
(EE) 7: /usr/bin/Xvnc (_ZN3rfb5Timer13checkTimeoutsEv+0x8c) [0x564b087ce35c]
(EE) 8: /usr/bin/Xvnc (_ZN3rfb11VNCServerST13checkTimeoutsEv+0x17) [0x564b087cea47]
(EE) 9: /usr/bin/Xvnc (_ZN14XserverDesktop12blockHandlerEPi+0x253) [0x564b087c2733]
(EE) 10: /usr/bin/Xvnc (vncCallBlockHandlers+0x28) [0x564b087b6518]
(EE) 11: /usr/bin/Xvnc (BlockHandler+0x3e) [0x564b0880d1ee]
(EE) 12: /usr/bin/Xvnc (WaitForSomething+0x116) [0x564b08856d86]
(EE) 13: /usr/bin/Xvnc (Dispatch+0xa7) [0x564b088086d7]
(EE) 14: /usr/bin/Xvnc (dix_main+0x374) [0x564b0880c894]
(EE) 15: /lib64/libc.so.6 (__libc_start_main+0xcd) [0x7f6d410447ed]
(EE) 16: /usr/bin/Xvnc (_start+0x2a) [0x564b086f092a]
(EE)
(EE) Floating point exception at address 0x564b087cdad6
(EE)
Fatal server error:
(EE) Caught signal 8 (Floating point exception). Server aborting
— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/TigerVNC/tigervnc/issues/846#issuecomment-860055551, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AH5WRHESYB24EIO5DNG36LDTSNQEBANCNFSM4HYRMENA.
This message and any files transmitted with it are legally privileged and intended for the sole use of the individual(s) or entity to whom they are addressed. If you are not the intended recipient, please notify the sender by reply and delete the message and any attachments from your system. Any unauthorised use or disclosure of the content of this message is strictly prohibited and may be unlawful.
Nothing in this e-mail message amounts to a contractual or legal commitment on the part of EUROCONTROL, unless it is confirmed by appropriately signed hard copy.
Any views expressed in this message are those of the sender.
I had a vncviewer connection (on machine A) to a vncserver (on machine B), which in turn has a long-running vncviewer session to vncserver (on machine C). vncserver on machine B crashed while I was resizing vncviewer window to machine C. I don't know if it has anything to do that the resized window was itself an instance of vncviewer, but such crash has happened before to me.
OS: A: Arch Linux, B & C: Debian Buster Version 1.9 everywhere Desktop Manager on B: LxQt