search

cyrusimap / cyrus-imapd

Cyrus IMAP is an email, contacts and calendar server
http://cyrusimap.org
Other
531 stars 146 forks source link

sieve regex bug ? #1731

Open brong opened 8 years ago

brong commented 8 years ago

From: Ronny Bugzilla-Id: 3904 Version: 2.5.x (next) Owner: Bron Gondwana

brong commented 8 years ago

From: Ronny

Cyrus-Imapd-2.5.6 (Gentoo ebuild)

Hello,

i tried to used a regex in my sieve script and maybe triggered a bug/issue

My sieve script:

require ["imap4flags","regex"];

rule:[IPv6Test]

if header :regex :comparator "i;ascii-casemap" "received" "(from).+[IPv6:.([[:space:]].)(by mx1.domain.tld)" { addflag ["$label6"]; } rule:[TLSTest] if header :regex "received" "(from).+[[:space:]]+\(using TLS.+([[:space:]].)*(by mx1\.domain\.tld)" { addflag ["$label7"]; }

brong commented 8 years ago

From: elliefm

I can't reproduce this

If I put the script exactly as provided through sievec, I get sensible parse errors about the first regex (unmatched brackets) and the " rule:[TLSTest]" line (which is missing '#' at start). This makes sense. I don't get any sort of crash or corrupted double-linked list error.

If I double-backslash the '[' before 'IPv6' in the first regex (c.f. 'mx1\.domain' in the second regex), then the unmatched brackets error goes away. I'm not fluent in sieve, but I'm assuming here that with only a single backslash, [ gets expanded to [ by the quoted string handling, and so the regex engine sees an unmatched [. And that using the double backslash means the quoted string handling passes through [ to the regex engine, and so it gets treated correctly.

If I comment the " rule:[TLSTest]" line, the other error goes away.

brong commented 8 years ago

From: Ronny

Thanks for looking into this. The missing "#" in one line was a copy/paste error from me ,sorry for that.

I checked this again with the following sieve file (i have fixed the missing "#" and the not correct escapped "[" so this should now be ok)

======================================================================== require ["fileinto","reject","vacation","imap4flags","regex"];

rule:[IPv6Test]

if header :regex :comparator "i;ascii-casemap" "received" "(from).+\[IPv6:.([[:space:]].)(by mx1.domain.tld)" { addflag ["$label6"]; }

rule:[TLSTest]

if header :regex "received" "(from).+[[:space:]]+\(using TLS.+([[:space:]].)*(by mx1\.domain\.tld)" { addflag ["$label7"]; }

And now i get with sievec test test.sieve the following output: (runs as cyrus user:# ~/sievec test test.sieve)

======================================================================== Error in `/usr/lib/cyrus/sievec': free(): invalid next size (fast): 0x0000000001ee8440 ======= Backtrace: ========= /lib64/libc.so.6(+0x71f4b)[0x7f4451046f4b] /lib64/libc.so.6(+0x7740e)[0x7f445104c40e] /lib64/libc.so.6(+0x77bfb)[0x7f445104cbfb] /usr/lib64/libcyrus_sieve.so.0(+0x12b88)[0x7f44517d8b88] /usr/lib64/libcyrus_sieve.so.0(+0x12c09)[0x7f44517d8c09] /usr/lib64/libcyrus_sieve.so.0(+0x168d4)[0x7f44517dc8d4] /usr/lib64/libcyrus_sieve.so.0(sieve_script_parse+0xc0)[0x7f44517d4f20] /usr/lib/cyrus/sievec[0x401532] /lib64/libc.so.6(__libc_start_main+0xf0)[0x7f4450ff57b0] /usr/lib/cyrus/sievec[0x401729] ======= Memory map: ======== 00400000-00402000 r-xp 00000000 08:13 3955652 /usr/lib64/cyrus/sievec 00601000-00602000 r--p 00001000 08:13 3955652 /usr/lib64/cyrus/sievec 00602000-00603000 rw-p 00002000 08:13 3955652 /usr/lib64/cyrus/sievec 01ee0000-01f01000 rw-p 00000000 00:00 0 [heap] 7f444da0d000-7f444da23000 r-xp 00000000 08:13 4228471 /usr/lib64/gcc/x86_64-pc-linux-gnu/4.9.3/libgcc_s.so.1 7f444da23000-7f444dc22000 ---p 00016000 08:13 4228471 /usr/lib64/gcc/x86_64-pc-linux-gnu/4.9.3/libgcc_s.so.1 7f444dc22000-7f444dc23000 r--p 00015000 08:13 4228471 /usr/lib64/gcc/x86_64-pc-linux-gnu/4.9.3/libgcc_s.so.1 7f444dc23000-7f444dc24000 rw-p 00016000 08:13 4228471 /usr/lib64/gcc/x86_64-pc-linux-gnu/4.9.3/libgcc_s.so.1 7f444dc24000-7f444dc49000 r-xp 00000000 08:13 7111183 /lib64/liblzma.so.5.2.2 7f444dc49000-7f444de48000 ---p 00025000 08:13 7111183 /lib64/liblzma.so.5.2.2 7f444de48000-7f444de49000 r--p 00024000 08:13 7111183 /lib64/liblzma.so.5.2.2 7f444de49000-7f444de4a000 rw-p 00025000 08:13 7111183 /lib64/liblzma.so.5.2.2 7f444de4a000-7f444deb9000 r-xp 00000000 08:13 3424434 /lib64/libpcre.so.1.2.6 7f444deb9000-7f444e0b8000 ---p 0006f000 08:13 3424434 /lib64/libpcre.so.1.2.6 7f444e0b8000-7f444e0b9000 r--p 0006e000 08:13 3424434 /lib64/libpcre.so.1.2.6 7f444e0b9000-7f444e0ba000 rw-p 0006f000 08:13 3424434 /lib64/libpcre.so.1.2.6 7f444e0ba000-7f444e0bc000 r-xp 00000000 08:13 7489165 /lib64/libdl-2.21.so 7f444e0bc000-7f444e2bc000 ---p 00002000 08:13 7489165 /lib64/libdl-2.21.so 7f444e2bc000-7f444e2bd000 r--p 00002000 08:13 7489165 /lib64/libdl-2.21.so 7f444e2bd000-7f444e2be000 rw-p 00003000 08:13 7489165 /lib64/libdl-2.21.so 7f444e2be000-7f444e3b9000 r-xp 00000000 08:13 7222965 /usr/lib64/libsqlite3.so.0.8.6 7f444e3b9000-7f444e5b8000 ---p 000fb000 08:13 7222965 /usr/lib64/libsqlite3.so.0.8.6 7f444e5b8000-7f444e5bb000 r--p 000fa000 08:13 7222965 /usr/lib64/libsqlite3.so.0.8.6 7f444e5bb000-7f444e5be000 rw-p 000fd000 08:13 7222965 /usr/lib64/libsqlite3.so.0.8.6 7f444e5be000-7f444e5bf000 rw-p 00000000 00:00 0 7f444e5bf000-7f444e62b000 r-xp 00000000 08:13 7352871 /usr/lib64/libssl.so.1.0.0 7f444e62b000-7f444e82a000 ---p 0006c000 08:13 7352871 /usr/lib64/libssl.so.1.0.0 7f444e82a000-7f444e82f000 r--p 0006b000 08:13 7352871 /usr/lib64/libssl.so.1.0.0 7f444e82f000-7f444e836000 rw-p 00070000 08:13 7352871 /usr/lib64/libssl.so.1.0.0 7f444e836000-7f444eb1e000 r-xp 00000000 08:13 7365322 /usr/lib64/libmysqlclient.so.18.0.0 7f444eb1e000-7f444ed1e000 ---p 002e8000 08:13 7365322 /usr/lib64/libmysqlclient.so.18.0.0 7f444ed1e000-7f444ed25000 r--p 002e8000 08:13 7365322 /usr/lib64/libmysqlclient.so.18.0.0 7f444ed25000-7f444ed8b000 rw-p 002ef000 08:13 7365322 /usr/lib64/libmysqlclient.so.18.0.0 7f444ed8b000-7f444ed95000 rw-p 00000000 00:00 0 7f444ed95000-7f444ef10000 r-xp 00000000 08:13 3538958 /usr/lib64/libdb-4.8.so 7f444ef10000-7f444f10f000 ---p 0017b000 08:13 3538958 /usr/lib64/libdb-4.8.so 7f444f10f000-7f444f112000 r--p 0017a000 08:13 3538958 /usr/lib64/libdb-4.8.so 7f444f112000-7f444f115000 rw-p 0017d000 08:13 3538958 /usr/lib64/libdb-4.8.so 7f444f115000-7f444f131000 r-xp 00000000 08:13 3953367 /usr/lib64/libsasl2.so.3.0.0 7f444f131000-7f444f331000 ---p 0001c000 08:13 3953367 /usr/lib64/libsasl2.so.3.0.0 7f444f331000-7f444f332000 r--p 0001c000 08:13 3953367 /usr/lib64/libsasl2.so.3.0.0 7f444f332000-7f444f333000 rw-p 0001d000 08:13 3953367 /usr/lib64/libsasl2.so.3.0.0 7f444f333000-7f444f348000 r-xp 00000000 08:13 3282385 /lib64/libz.so.1.2.8 7f444f348000-7f444f547000 ---p 00015000 08:13 3282385 /lib64/libz.so.1.2.8 7f444f547000-7f444f548000 r--p 00014000 08:13 3282385 /lib64/libz.so.1.2.8 7f444f548000-7f444f549000 rw-p 00015000 08:13 3282385 /lib64/libz.so.1.2.8 7f444f549000-7f444f64b000 r-xp 00000000 08:13 7488115 /lib64/libm-2.21.so 7f444f64b000-7f444f84a000 ---p 00102000 08:13 7488115 /lib64/libm-2.21.so 7f444f84a000-7f444f84b000 r--p 00101000 08:13 7488115 /lib64/libm-2.21.so 7f444f84b000-7f444f84c000 rw-p 00102000 08:13 7488115 /lib64/libm-2.21.so 7f444f84c000-7f444f858000 r-xp 00000000 08:13 3163210 /usr/lib64/libjansson.so.4.7.0 7f444f858000-7f444fa57000 ---p 0000c000 08:13 3163210 /usr/lib64/libjansson.so.4.7.0 7f444fa57000-7f444fa58000 r--p 0000b000 08:13 3163210 /usr/lib64/libjansson.so.4.7.0 7f444fa58000-7f444fa59000 rw-p 0000c000 08:13 3163210 /usr/lib64/libjansson.so.4.7.0 7f444fa59000-7f444fa69000 r-xp 00000000 08:13 3677591 /usr/lib64/libicalvcal.so.1.0.1 7f444fa69000-7f444fc69000 ---p 00010000 08:13 3677591 /usr/lib64/libicalvcal.so.1.0.1 7f444fc69000-7f444fc6c000 r--p 00010000 08:13 3677591 /usr/lib64/libicalvcal.so.1.0.1 7f444fc6c000-7f444fc6e000 rw-p 00013000 08:13 3677591 /usr/lib64/libicalvcal.so.1.0.1 7f444fc6e000-7f444fc6f000 rw-p 00000000 00:00 0 7f444fc6f000-7f444fc84000 r-xp 00000000 08:13 3677835 /usr/lib64/libicalss.so.1.0.1 7f444fc84000-7f444fe84000 ---p 00015000 08:13 3677835 /usr/lib64/libicalss.so.1.0.1 7f444fe84000-7f444fe85000 r--p 00015000 08:13 3677835 /usr/lib64/libicalss.so.1.0.1 7f444fe85000-7f444fe86000 rw-p 00016000 08:13 3677835 /usr/lib64/libicalss.so.1.0.1 7f444fe86000-7f444fe88000 rw-p 00000000 00:00 0 7f444fe88000-7f444fedf000 r-xp 00000000 08:13 3677834 /usr/lib64/libical.so.1.0.1 7f444fedf000-7f44500de000 ---p 00057000 08:13 3677834 /usr/lib64/libical.so.1.0.1 7f44500de000-7f44500eb000 r--p 00056000 08:13 3677834 /usr/lib64/libical.so.1.0.1 7f44500eb000-7f44500ed000 rw-p 00063000 08:13 3677834 /usr/lib64/libical.so.1.0.1 7f44500ed000-7f44500ee000 rw-p 00000000 00:00 0 7f44500ee000-7f445024b000 r-xp 00000000 08:13 7231504 /usr/lib64/libxml2.so.2.9.3 7f445024b000-7f445044a000 ---p 0015d000 08:13 7231504 /usr/lib64/libxml2.so.2.9.3 7f445044a000-7f4450452000 r--p 0015c000 08:13 7231504 /usr/lib64/libxml2.so.2.9.3 7f4450452000-7f4450454000 rw-p 00164000 08:13 7231504 /usr/lib64/libxml2.so.2.9.3 7f4450454000-7f4450456000 rw-p 00000000 00:00 0 7f4450456000-7f4450458000 r-xp 00000000 08:13 3424484 /usr/lib64/libpcreposix.so.0.0.3 7f4450458000-7f4450657000 ---p 00002000 08:13 3424484 /usr/lib64/libpcreposix.so.0.0.3 7f4450657000-7f4450658000 r--p 00001000 08:13 3424484 /usr/lib64/libpcreposix.so.0.0.3 7f4450658000-7f4450659000 rw-p 00002000 08:13 3424484 /usr/lib64/libpcreposix.so.0.0.3 7f4450659000-7f445086d000 r-xp 00000000 08:13 7351199 /usr/lib64/libcrypto.so.1.0.0 7f445086d000-7f4450a6c000 ---p 00214000 08:13 7351199 /usr/lib64/libcrypto.so.1.0.0 7f4450a6c000-7f4450a8a000 r--p 00213000 08:13 7351199 /usr/lib64/libcrypto.so.1.0.0 7f4450a8a000-7f4450a96000 rw-p 00231000 08:13 7351199 /usr/lib64/libcrypto.so.1.0.0 7f4450a96000-7f4450a9a000 rw-p 00000000 00:00 0 7f4450a9a000-7f4450bcf000 r-xp 00000000 08:13 3956619 /usr/lib64/libcyrus.so.0.0.0 7f4450bcf000-7f4450dce000 ---p 00135000 08:13 3956619 /usr/lib64/libcyrus.so.0.0.0 7f4450dce000-7f4450dcf000 r--p 00134000 08:13 3956619 /usr/lib64/libcyrus.so.0.0.0 7f4450dcf000-7f4450dd1000 rw-p 00135000 08:13 3956619 /usr/lib64/libcyrus.so.0.0.0 7f4450dd1000-7f4450dd2000 rw-p 00000000 00:00 0 7f4450dd2000-7f4450dd4000 r-xp 00000000 08:13 3955578 /usr/lib64/libcyrus_com_err.so.0.0.0 7f4450dd4000-7f4450fd3000 ---p 00002000 08:13 3955578 /usr/lib64/libcyrus_com_err.so.0.0.0 7f4450fd3000-7f4450fd4000 r--p 00001000 08:13 3955578 /usr/lib64/libcyrus_com_err.so.0.0.0 7f4450fd4000-7f4450fd5000 rw-p 00002000 08:13 3955578 /usr/lib64/libcyrus_com_err.so.0.0.0 7f4450fd5000-7f4451167000 r-xp 00000000 08:13 7488248 /lib64/libc-2.21.so 7f4451167000-7f4451366000 ---p 00192000 08:13 7488248 /lib64/libc-2.21.so 7f4451366000-7f445136a000 r--p 00191000 08:13 7488248 /lib64/libc-2.21.so 7f445136a000-7f445136c000 rw-p 00195000 08:13 7488248 /lib64/libc-2.21.so 7f445136c000-7f4451370000 rw-p 00000000 00:00 0 7f4451370000-7f4451387000 r-xp 00000000 08:13 7488237 /lib64/libpthread-2.21.so 7f4451387000-7f4451586000 ---p 00017000 08:13 7488237 /lib64/libpthread-2.21.so 7f4451586000-7f4451587000 r--p 00016000 08:13 7488237 /lib64/libpthread-2.21.so 7f4451587000-7f4451588000 rw-p 00017000 08:13 7488237 /lib64/libpthread-2.21.so 7f4451588000-7f445158c000 rw-p 00000000 00:00 0 7f445158c000-7f44515a3000 r-xp 00000000 08:13 3956618 /usr/lib64/libcyrus_min.so.0.0.0 7f44515a3000-7f44517a2000 ---p 00017000 08:13 3956618 /usr/lib64/libcyrus_min.so.0.0.0 7f44517a2000-7f44517a3000 r--p 00016000 08:13 3956618 /usr/lib64/libcyrus_min.so.0.0.0 7f44517a3000-7f44517c6000 rw-p 00017000 08:13 3956618 /usr/lib64/libcyrus_min.so.0.0.0 7f44517c6000-7f44517e6000 r-xp 00000000 08:13 3955605 /usr/lib64/libcyrus_sieve.so.0.0.0 7f44517e6000-7f44519e5000 ---p 00020000 08:13 3955605 /usr/lib64/libcyrus_sieve.so.0.0.0 7f44519e5000-7f44519e6000 r--p 0001f000 08:13 3955605 /usr/lib64/libcyrus_sieve.so.0.0.0 7f44519e6000-7f44519e7000 rw-p 00020000 08:13 3955605 /usr/lib64/libcyrus_sieve.so.0.0.0 7f44519e7000-7f4451a09000 r-xp 00000000 08:13 7487553 /lib64/ld-2.21.so 7f4451bd6000-7f4451be2000 rw-p 00000000 00:00 0 7f4451c05000-7f4451c08000 rw-p 00000000 00:00 0 7f4451c08000-7f4451c09000 r--p 00021000 08:13 7487553 /lib64/ld-2.21.so 7f4451c09000-7f4451c0a000 rw-p 00022000 08:13 7487553 /lib64/ld-2.21.so 7f4451c0a000-7f4451c0b000 rw-p 00000000 00:00 0 7fff76e2e000-7fff76e4f000 rw-p 00000000 00:00 0 [stack] 7fff76eaa000-7fff76eac000 r--p 00000000 00:00 0 [vvar] 7fff76eac000-7fff76eae000 r-xp 00000000 00:00 0 [vdso] ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] Abgebrochen

I am running now "cyrus-imapd 2.5.7" (aka Gentoo: net-mail/cyrus-imapd-2.5.7) (x86_64 now, while opening the bug, it was and older system running in 32bits only (if that makes any difference))

Perhaps, you can provide some test sieve cases with regex, which i can then verify with sievec ? If i can help you to track this further or you need any information, please ask, thanks Ronny

brong commented 8 years ago

From: elliefm

Thanks for the update. It's interesting that it's still happening on a newer system (though I notice the exact error has changed, though it's similar in spirit)

I've tried again with your updated test sieve and I still can't reproduce this myself (cyrus-imapd-2.5.7, debian testing)

The backtrace you've provided isn't very useful, looks like your binaries are stripped or were compiled without debug symbols (CFLAGS=-g) enabled. Are you able to reproduce this in a build that has been compiled with debug symbols enabled, and provide the backtrace from that? (Sorry I can't provide more specific directions than this, I don't know how Gentoo packages work.)

At this point, I suspect there's probably some kind of memory mishandling happening, and for whatever reason Gentoo complains about it where Debian doesn't. If this is the case, it'd be good to find it and fix it if we can.

The other thing that might be informative is running sievec under valgrind. I've just done this myself and it reports a couple of leaks, but doesn't detect anything else. It'd be interesting to see what valgrind on Gentoo reports.

dekeonus commented 7 years ago

My google-fu leads me to believe the issue to be https://bugs.exim.org/show_bug.cgi?id=1830 for which Debian has a patch https://sources.debian.net/patches/pcre3/2:8.39-2/pcreposix.patch/ This was once assigned Debian bug 22525 but seems the report is no longer available; Comment 10 in pcre's bug tracker suggests Debian has been applying this patch for 18 years. This patch is not applied on gentoo's libpcre

Back trace from cyrus-imapd-2.5.10 on gentoo-x86

*** Error in `/usr/lib/cyrus/sievec': corrupted double-linked list: 0x0804d8e8 ***
======= Backtrace: =========
/lib/libc.so.6(+0x687a6)[0xb7e1f7a6]
/lib/libc.so.6(+0x6e65d)[0xb7e2565d]
/lib/libc.so.6(+0x70a66)[0xb7e27a66]
/lib/libc.so.6(__libc_malloc+0x5f)[0xb7e28c37]
/lib/libc.so.6(+0xcb33a)[0xb7e8233a]
/lib/libc.so.6(regcomp+0xb4)[0xb7e83297]
/usr/lib/libcyrus_sieve.so.0(+0xd354)[0xb7fac354]
/usr/lib/libcyrus_sieve.so.0(+0xd8e4)[0xb7fac8e4]
/usr/lib/libcyrus_sieve.so.0(+0x11155)[0xb7fb0155]
/usr/lib/libcyrus_sieve.so.0(sieve_script_parse+0xc8)[0xb7fab381]
/usr/lib/cyrus/sievec[0x8049081]
/lib/libc.so.6(__libc_start_main+0xf4)[0xb7dcf49b]
/usr/lib/cyrus/sievec[0x80492a6]
======= Memory map: ========
08048000-0804a000 r-xp 00000000 08:05 1242493    /usr/lib/cyrus/sievec
0804a000-0804b000 r--p 00001000 08:05 1242493    /usr/lib/cyrus/sievec
0804b000-0804c000 rw-p 00002000 08:05 1242493    /usr/lib/cyrus/sievec
0804c000-0806d000 rw-p 00000000 00:00 0          [heap]
b5000000-b5021000 rw-p 00000000 00:00 0
b5021000-b5100000 ---p 00000000 00:00 0
b51de000-b51e3000 rw-p 00000000 00:00 0
b51e3000-b530e000 r-xp 00000000 08:05 1349500    /usr/lib/gcc/i686-pc-linux-gnu/4.9.4/libstdc++.so.6.0.20
b530e000-b5313000 r--p 0012a000 08:05 1349500    /usr/lib/gcc/i686-pc-linux-gnu/4.9.4/libstdc++.so.6.0.20
b5313000-b5316000 rw-p 0012f000 08:05 1349500    /usr/lib/gcc/i686-pc-linux-gnu/4.9.4/libstdc++.so.6.0.20
b5316000-b531c000 rw-p 00000000 00:00 0
b531c000-b6c1c000 r-xp 00000000 08:05 944955     /usr/lib/libicudata.so.58.1
b6c1c000-b6c1d000 r--p 018ff000 08:05 944955     /usr/lib/libicudata.so.58.1
b6c1d000-b6c1e000 rw-p 01900000 08:05 944955     /usr/lib/libicudata.so.58.1
b6c1e000-b6e88000 r-xp 00000000 08:05 944972     /usr/lib/libicui18n.so.58.1
b6e88000-b6e90000 r--p 00269000 08:05 944972     /usr/lib/libicui18n.so.58.1
b6e90000-b6e91000 rw-p 00271000 08:05 944972     /usr/lib/libicui18n.so.58.1
b6e91000-b6e92000 rw-p 00000000 00:00 0
b6e92000-b6eab000 r-xp 00000000 08:05 1349540    /usr/lib/gcc/i686-pc-linux-gnu/4.9.4/libgcc_s.so.1
b6eab000-b6eac000 r--p 00018000 08:05 1349540    /usr/lib/gcc/i686-pc-linux-gnu/4.9.4/libgcc_s.so.1
b6eac000-b6ead000 rw-p 00019000 08:05 1349540    /usr/lib/gcc/i686-pc-linux-gnu/4.9.4/libgcc_s.so.1
b6ead000-b6eae000 rw-p 00000000 00:00 0
b6eae000-b6ec0000 r-xp 00000000 08:05 1953769    /lib/libresolv-2.22.so
b6ec0000-b6ec1000 ---p 00012000 08:05 1953769    /lib/libresolv-2.22.so
b6ec1000-b6ec2000 r--p 00012000 08:05 1953769    /lib/libresolv-2.22.so
b6ec2000-b6ec3000 rw-p 00013000 08:05 1953769    /lib/libresolv-2.22.so
b6ec3000-b6ec5000 rw-p 00000000 00:00 0
b6ec5000-b6ec8000 r-xp 00000000 08:05 1953567    /lib/libkeyutils.so.1.5
b6ec8000-b6ec9000 r--p 00002000 08:05 1953567    /lib/libkeyutils.so.1.5
b6ec9000-b6eca000 rw-p 00003000 08:05 1953567    /lib/libkeyutils.so.1.5
b6eca000-b6ed4000 r-xp 00000000 08:05 948083     /usr/lib/libkrb5support.so.0.1
b6ed4000-b6ed5000 r--p 00009000 08:05 948083     /usr/lib/libkrb5support.so.0.1
b6ed5000-b6ed6000 rw-p 0000a000 08:05 948083     /usr/lib/libkrb5support.so.0.1
b6ed6000-b6f06000 r-xp 00000000 08:05 948180     /usr/lib/libk5crypto.so.3.1
b6f06000-b6f07000 ---p 00030000 08:05 948180     /usr/lib/libk5crypto.so.3.1
b6f07000-b6f08000 r--p 00030000 08:05 948180     /usr/lib/libk5crypto.so.3.1
b6f08000-b6f09000 rw-p 00031000 08:05 948180     /usr/lib/libk5crypto.so.3.1
b6f09000-b6f0a000 rw-p 00000000 00:00 0
b6f0a000-b6f32000 r-xp 00000000 08:05 1953878    /lib/liblzma.so.5.2.2
b6f32000-b6f33000 r--p 00027000 08:05 1953878    /lib/liblzma.so.5.2.2
b6f33000-b6f34000 rw-p 00028000 08:05 1953878    /lib/liblzma.so.5.2.2
b6f34000-b6f35000 rw-p 00000000 00:00 0
b6f35000-b70c1000 r-xp 00000000 08:05 944950     /usr/lib/libicuuc.so.58.1
b70c1000-b70c2000 ---p 0018c000 08:05 944950     /usr/lib/libicuuc.so.58.1
b70c2000-b70cd000 r--p 0018c000 08:05 944950     /usr/lib/libicuuc.so.58.1
b70cd000-b70ce000 rw-p 00197000 08:05 944950     /usr/lib/libicuuc.so.58.1
b70ce000-b70cf000 rw-p 00000000 00:00 0
b70cf000-b7142000 r-xp 00000000 08:05 1953564    /lib/libpcre.so.1.2.7
b7142000-b7143000 r--p 00072000 08:05 1953564    /lib/libpcre.so.1.2.7
b7143000-b7144000 rw-p 00073000 08:05 1953564    /lib/libpcre.so.1.2.7
b7144000-b7146000 r-xp 00000000 08:05 1955187    /lib/libdl-2.22.so
b7146000-b7147000 ---p 00002000 08:05 1955187    /lib/libdl-2.22.so
b7147000-b7148000 r--p 00002000 08:05 1955187    /lib/libdl-2.22.so
b7148000-b7149000 rw-p 00003000 08:05 1955187    /lib/libdl-2.22.so
b7149000-b724f000 r-xp 00000000 08:05 945321     /usr/lib/libsqlite3.so.0.8.6
b724f000-b7251000 r--p 00105000 08:05 945321     /usr/lib/libsqlite3.so.0.8.6
b7251000-b7253000 rw-p 00107000 08:05 945321     /usr/lib/libsqlite3.so.0.8.6
b7253000-b72b8000 r-xp 00000000 08:05 946471     /usr/lib/libssl.so.1.0.0
b72b8000-b72bb000 r--p 00064000 08:05 946471     /usr/lib/libssl.so.1.0.0
b72bb000-b72bf000 rw-p 00067000 08:05 946471     /usr/lib/libssl.so.1.0.0
b72bf000-b72c0000 rw-p 00000000 00:00 0
b72c0000-b759b000 r-xp 00000000 08:05 946280     /usr/lib/libmysqlclient.so.18.0.0
b759b000-b759f000 r--p 002da000 08:05 946280     /usr/lib/libmysqlclient.so.18.0.0
b759f000-b75f5000 rw-p 002de000 08:05 946280     /usr/lib/libmysqlclient.so.18.0.0
b75f5000-b75fc000 rw-p 00000000 00:00 0
b75fc000-b7773000 r-xp 00000000 08:05 946646     /usr/lib/libdb-4.8.so
b7773000-b7775000 r--p 00176000 08:05 946646     /usr/lib/libdb-4.8.so
b7775000-b7777000 rw-p 00178000 08:05 946646     /usr/lib/libdb-4.8.so
b7777000-b7834000 r-xp 00000000 08:05 948177     /usr/lib/libkrb5.so.3.3
b7834000-b783a000 r--p 000bc000 08:05 948177     /usr/lib/libkrb5.so.3.3
b783a000-b783c000 rw-p 000c2000 08:05 948177     /usr/lib/libkrb5.so.3.3
b783c000-b7857000 r-xp 00000000 08:05 947566     /usr/lib/libsasl2.so.3.0.0
b7857000-b7858000 r--p 0001a000 08:05 947566     /usr/lib/libsasl2.so.3.0.0
b7858000-b7859000 rw-p 0001b000 08:05 947566     /usr/lib/libsasl2.so.3.0.0
b7859000-b786e000 r-xp 00000000 08:05 1953700    /lib/libz.so.1.2.8
b786e000-b786f000 r--p 00014000 08:05 1953700    /lib/libz.so.1.2.8
b786f000-b7870000 rw-p 00015000 08:05 1953700    /lib/libz.so.1.2.8
b7870000-b78b4000 r-xp 00000000 08:05 1955190    /lib/libm-2.22.so
b78b4000-b78b5000 r--p 00043000 08:05 1955190    /lib/libm-2.22.so
b78b5000-b78b6000 rw-p 00044000 08:05 1955190    /lib/libm-2.22.so
b78b6000-b78b7000 rw-p 00000000 00:00 0
b78b7000-b78c3000 r-xp 00000000 08:05 945050     /usr/lib/libjansson.so.4.9.0
b78c3000-b78c4000 r--p 0000b000 08:05 945050     /usr/lib/libjansson.so.4.9.0
b78c4000-b78c5000 rw-p 0000c000 08:05 945050     /usr/lib/libjansson.so.4.9.0
b78c5000-b78d2000 r-xp 00000000 08:05 947398     /usr/lib/libicalvcal.so.1.0.1
b78d2000-b78d4000 r--p 0000c000 08:05 947398     /usr/lib/libicalvcal.so.1.0.1
b78d4000-b78d5000 rw-p 0000e000 08:05 947398     /usr/lib/libicalvcal.so.1.0.1
b78d5000-b78d6000 rw-p 00000000 00:00 0
b78d6000-b78e9000 r-xp 00000000 08:05 947400     /usr/lib/libicalss.so.1.0.1
b78e9000-b78ea000 r--p 00012000 08:05 947400     /usr/lib/libicalss.so.1.0.1
b78ea000-b78eb000 rw-p 00013000 08:05 947400     /usr/lib/libicalss.so.1.0.1
b78eb000-b78ed000 rw-p 00000000 00:00 0
b78ed000-b793a000 r-xp 00000000 08:05 947399     /usr/lib/libical.so.1.0.1
b793a000-b7944000 r--p 0004d000 08:05 947399     /usr/lib/libical.so.1.0.1
b7944000-b7945000 rw-p 00057000 08:05 947399     /usr/lib/libical.so.1.0.1
b7945000-b7946000 rw-p 00000000 00:00 0
b7946000-b7a98000 r-xp 00000000 08:05 945130     /usr/lib/libxml2.so.2.9.4
b7a98000-b7a9d000 r--p 00151000 08:05 945130     /usr/lib/libxml2.so.2.9.4
b7a9d000-b7a9e000 rw-p 00156000 08:05 945130     /usr/lib/libxml2.so.2.9.4
b7a9e000-b7aa0000 rw-p 00000000 00:00 0
b7aa0000-b7aa2000 r-xp 00000000 08:05 945314     /usr/lib/libpcreposix.so.0.0.4
b7aa2000-b7aa3000 r--p 00001000 08:05 945314     /usr/lib/libpcreposix.so.0.0.4
b7aa3000-b7aa4000 rw-p 00002000 08:05 945314     /usr/lib/libpcreposix.so.0.0.4
b7aa4000-b7c60000 r-xp 00000000 08:05 945885     /usr/lib/libcrypto.so.1.0.0
b7c60000-b7c71000 r--p 001bb000 08:05 945885     /usr/lib/libcrypto.so.1.0.0
b7c71000-b7c78000 rw-p 001cc000 08:05 945885     /usr/lib/libcrypto.so.1.0.0
b7c78000-b7c7b000 rw-p 00000000 00:00 0
b7c7b000-b7db0000 r-xp 00000000 08:05 946530     /usr/lib/libcyrus.so.0.0.0
b7db0000-b7db1000 r--p 00134000 08:05 946530     /usr/lib/libcyrus.so.0.0.0
b7db1000-b7db2000 rw-p 00135000 08:05 946530     /usr/lib/libcyrus.so.0.0.0
b7db2000-b7db3000 rw-p 00000000 00:00 0
b7db3000-b7db4000 r-xp 00000000 08:05 946348     /usr/lib/libcyrus_com_err.so.0.0.0
b7db4000-b7db5000 ---p 00001000 08:05 946348     /usr/lib/libcyrus_com_err.so.0.0.0
b7db5000-b7db6000 r--p 00001000 08:05 946348     /usr/lib/libcyrus_com_err.so.0.0.0
b7db6000-b7db7000 rw-p 00002000 08:05 946348     /usr/lib/libcyrus_com_err.so.0.0.0
b7db7000-b7f50000 r-xp 00000000 08:05 1955176    /lib/libc-2.22.so
b7f50000-b7f53000 r--p 00198000 08:05 1955176    /lib/libc-2.22.so
b7f53000-b7f55000 rw-p 0019b000 08:05 1955176    /lib/libc-2.22.so
b7f55000-b7f58000 rw-p 00000000 00:00 0
b7f58000-b7f6f000 r-xp 00000000 08:05 1953881    /lib/libpthread-2.22.so
b7f6f000-b7f70000 r--p 00016000 08:05 1953881    /lib/libpthread-2.22.so
b7f70000-b7f71000 rw-p 00017000 08:05 1953881    /lib/libpthread-2.22.so
b7f71000-b7f73000 rw-p 00000000 00:00 0
b7f73000-b7f76000 r-xp 00000000 08:05 1953803    /lib/libcom_err.so.2.1
b7f76000-b7f77000 r--p 00002000 08:05 1953803    /lib/libcom_err.so.2.1
b7f77000-b7f78000 rw-p 00003000 08:05 1953803    /lib/libcom_err.so.2.1
b7f78000-b7f8c000 r-xp 00000000 08:05 945688     /usr/lib/libcyrus_min.so.0.0.0
b7f8c000-b7f8d000 r--p 00013000 08:05 945688     /usr/lib/libcyrus_min.so.0.0.0
b7f8d000-b7f9f000 rw-p 00014000 08:05 945688     /usr/lib/libcyrus_min.so.0.0.0
b7f9f000-b7fbd000 r-xp 00000000 08:05 946144     /usr/lib/libcyrus_sieve.so.0.0.0
b7fbd000-b7fbe000 r--p 0001d000 08:05 946144     /usr/lib/libcyrus_sieve.so.0.0.0
b7fbe000-b7fbf000 rw-p 0001e000 08:05 946144     /usr/lib/libcyrus_sieve.so.0.0.0
b7fd7000-b7fd9000 rw-p 00000000 00:00 0
b7fd9000-b7fdb000 r--p 00000000 00:00 0          [vvar]
b7fdb000-b7fdc000 r-xp 00000000 00:00 0          [vdso]
b7fdc000-b7ffd000 r-xp 00000000 08:05 1955193    /lib/ld-2.22.so
b7ffd000-b7ffe000 rw-p 00000000 00:00 0
b7ffe000-b7fff000 r--p 00021000 08:05 1955193    /lib/ld-2.22.so
b7fff000-b8000000 rw-p 00022000 08:05 1955193    /lib/ld-2.22.so
bffde000-c0000000 rw-p 00000000 00:00 0          [stack]

Program received signal SIGABRT, Aborted.
0xb7fdbbcd in __kernel_vsyscall ()
(gdb) bt
#0  0xb7fdbbcd in __kernel_vsyscall ()
#1  0xb7de2f38 in raise () from /lib/libc.so.6
#2  0xb7de4471 in abort () from /lib/libc.so.6
#3  0xb7e1f7ab in __libc_message () from /lib/libc.so.6
#4  0xb7e2565d in malloc_printerr () from /lib/libc.so.6
#5  0xb7e27a66 in _int_malloc () from /lib/libc.so.6
#6  0xb7e28c37 in malloc () from /lib/libc.so.6
#7  0xb7e8233a in re_compile_internal () from /lib/libc.so.6
#8  0xb7e83297 in regcomp () from /lib/libc.so.6
#9  0xb7fac354 in verify_regex (parse_script=parse_script@entry=0x804dcd8,
    s=0x8053b40 ".*some sieve regex I didn't want to share", cflags=97, cflags@entry=33) at sieve/sieve.y:1663
#10 0xb7fac8e4 in verify_regexs (parse_script=parse_script@entry=0x804dcd8, comp=<optimized out>, sa=<optimized out>,
    sa=<optimized out>) at sieve/sieve.y:1683
#11 0xb7fb0155 in sieveparse (parse_script=0x804dcd8) at sieve/sieve.y:568
#12 0xb7fab381 in sieve_script_parse (interp=0x804c7d8, script=script@entry=0x804c008,
    script_context=script_context@entry=0xbfffe9ec, ret=ret@entry=0xbfffe9e8) at sieve/script.c:224
#13 0x08049081 in is_script_parsable (ret=0xbfffe9e0, errstr=<synthetic pointer>, stream=0x804c008)
    at sieve/sievec.c:217
#14 main (argc=3, argv=0xbfffeab4) at sieve/sievec.c:105
brong commented 7 years ago

Wow, 18 years. That's nearly as bad as the bug where I've been patching "multipart/*" support out of apache mod_perl2 on our servers for years (it was added for a HTTP draft that never went anywhere, and has been buggy all that time).

When we upgraded to Debian Jessie we started seeing the same crashes because it was still broken upstream, and had to port our patch forwards and rebuild.

Open Source - sometimes it helps ...

elliefm commented 7 years ago

Oh, I had an email discussion with @anatoli26 about the pcreposix conflict a while ago.

Postfix in particular gets around this issue by only using the pcreposix-specific interfaces (not the backward compatible ones), and requiring pcreposix specifically as a dependency (rather than trying to support multiple regex backends).

This issue doesn't come up for most of us Cyrus developers, since we mostly use Debian, and they've been quietly patching away the problem underneath us all this time. But it'll affect anyone who builds pcreposix from source for whatever reason, and doesn't apply or know about Debian's patch. Or anyone using a distro that doesn't include Debian's patch or something similar.

My inclination would be to follow Postfix's lead here, but we do use regexes in a bunch of places and I'm not sure what the impact would be beyond sieve

anatoli26 commented 7 years ago

Yepp, we discovered with @elliefm that the PCRE lib was causing a problem when built from sources as the sources don't include the Debian patch:

Everything is clear now. What happens is that from-sources PCRE lib (particularly, the POSIX regex wrapper (pcreposix.h), which is a replacement for the POSIX regex in libc) defines the same function names as in libc and it can't be used with mere library includes jiggling. What should be done (and what Debian package does) is to patch the PCRE lib sources to redefine in the pcreposix.h the 3 colliding functions like this:

define regcomp pcreposix_regcomp

and also rename correspondingly the export definitions for the lib to build with the new func names. You can check the patch with: apt-get source libpcre3-dev && cat pcre3-8.31/debian/patches/pcreposix.patch.

In this case, when PCRE headers are included, the actual function that the code links to is pcreposix_regcomp and there's no name collision with the libc implementation.

Now, leaving it as-is for someone else with from-sources PCRE lib to face the same weird memory corruption problems doesn't sound like the best approach. Other packages that use PCRE lib manage it by themselves (e.g. Postfix uses PCRE native API and NginX uses the library source code directly).

I just analyzed all the uses the Cyrus IMAP gives to PCRE lib when it's available, and it is actually quite minuscule IMO: just the UTF8 regex comparison support and only inside Sieve (e.g. sieve/bc_eval.c:127). The glob.c code which gave the segfault, for example, takes no advantage of linking with PCRE lib.

So we were discussing if PCRE in Cyrus is needed at all:

I'm not sure it should be a MUST to have PCRE for Sieve just because it supports UTF-8. If Sieve would use PCRE's additional functionality, that would make sense, but as-is.. Also, it should be considered that UTF-8 is an optional feature for PCRE lib, it could be built without it, this is what configure checks when deciding on PCRE support, so a hard dependency would have to require this as well.

I'm inclined to reason that, if someone went to all the trouble to detect UTF-8 support in the PCRE detection without actually using PCRE's interfaces, then they probably wanted UTF-8 support specifically, and PCRE itself was just a means to that end.

The use of other regex libraries when a UTF-8 PCRE isn't available looks like a poorly-thought-out fallback, but I think the goal is for sieve to support UTF-8. At least as I'm interpreting it.

When talking about UTF, not supporting it doesn't mean Sieve wouldn't work with UTF8 strings at all, or not being able to match UTF8 chars. From the POSIX standard, IEEE Std 1003.1-2008 (http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap09.html#tag_09):

Matching shall be based on the bit pattern used for encoding the character, not on the graphic representation of the character. This means that if a character set contains two or more encodings for a graphic symbol, or if the strings searched contain text encoded in more than one codeset, no attempt is made to search for any other representation of the encoded symbol. If that is required, the user can specify equivalence classes containing all variations of the desired graphic symbol.

If I understand it correctly, if the string and the match are both in unicode, the POSIX version should work well, though may be someone with specific knowledge on UTF8 could clarify this point. If it actually works, then I believe there is no need for PCRE in Cyrus at all.

Anyway, if you decide to continue supporting it, PCRE has v10 for some years already, which should be used instead of v8, as the 8 branch is quite buggy and unmaintained for general code improvements (v10 is buggy as well, but not to the point of v8). In general, I see PCRE as a potential exploitation vector, as well as Sieve itself. Just include some non-standard header (that passes through the incoming MTA unmodified) with name/value triggering some unknown PCRE/Sieve bug, and here it is.

brong commented 7 years ago

We should mandate re2 and stop doing pcre at all

On Tue, 17 Jan 2017, at 20:30, Anatoli wrote:

Yepp, we discovered with @elliefm[1] that the PCRE lib was causing a problem when built from sources as the sources don't include the Debian patch:

Everything is clear now. What happens is that from-sources PCRE lib (particularly, the POSIX regex wrapper (pcreposix.h), which is a replacement for the POSIX regex in libc) defines the same function names as in libc and it can't be used with mere library includes jiggling. What should be done (and what Debian package does) is to patch the PCRE lib sources to redefine in the pcreposix.h the 3 colliding functions like this:

define regcomp pcreposix_regcomp

and also rename correspondingly the export definitions for the lib to build with the new func names. You can check the patch with: apt-get source libpcre3-dev && cat pcre3-8.31/debian/patches/pcreposix.patch. In this case, when PCRE headers are included, the actual function that the code links to is pcreposix_regcomp and there's no name collision with the libc implementation. Now, leaving it as-is for someone else with from-sources PCRE lib to face the same weird memory corruption problems doesn't sound like the best approach. Other packages that use PCRE lib manage it by themselves (e.g. Postfix uses PCRE native API and NginX uses the library source code directly). I just analyzed all the uses the Cyrus IMAP gives to PCRE lib when it's available, and it is actually quite minuscule IMO: just the UTF8 regex comparison support and only inside Sieve (e.g. sieve/bc_eval.c:127). The glob.c code which gave the segfault, for example, takes no advantage of linking with PCRE lib. So we were discussing if PCRE in Cyrus is needed at all:

I'm not sure it should be a MUST to have PCRE for Sieve just because it supports UTF-8. If Sieve would use PCRE's additional functionality, that would make sense, but as-is.. Alse it should be considered that UTF-8 is an optional feature for PCRE lib, it could be built without it, this is what configure checks when deciding on PCRE support, so a hard dependency would have to require this as well. I'm inclined to reason that, if someone went to all the trouble to detect UTF-8 support in the PCRE detection without actually using PCRE's interfaces, then they probably wanted UTF-8 support specifically, and PCRE itself was just a means to that end. The use of other regex libraries when a UTF-8 PCRE isn't available looks like a poorly-thought-out fallback, but I think the goal is for sieve to support UTF-8. At least as I'm interpreting it. When talking about UTF, not supporting it doesn't mean Sieve wouldn't work with UTF8 strings at all, or not being able to match UTF8 chars. From the POSIX standard, IEEE Std 1003.1-2008 (http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap09.html#tag_09) : Matching shall be based on the bit pattern used for encoding the character, not on the graphic representation of the character. This means that if a character set contains two or more encodings for a graphic symbol, or if the strings searched contain text encoded in more than one codeset, no attempt is made to search for any other representation of the encoded symbol. If that is required, the user can specify equivalence classes containing all variations of the desired graphic symbol. If I understand it correctly, if the string and the match are both in unicode, the POSIX version should work well, though may be someone with specific knowledge on UTF8 could clarify this point. If it actually works, then I believe there is no need for PCRE in Cyrus at all. Anyway, if you decide to continue supporting it, PCRE has v10 for some years already, which should be used instead of v8, as the 8 branch is quite buggy and unmaintained for general code improvements (v10 is buggy as well, but not to the point of v8). In general, I see PCRE as a potential exploitation vector, as well as Sieve itself. Just include some non-standard header (that passes through the incoming MTA unmodified) with name/value triggering some unknown PCRE/Sieve bug, and here it is. — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub[2], or mute the thread[3].

--

Bron Gondwana

brong@fastmail.fm

Links:

  1. https://github.com/elliefm
  2. https://github.com/cyrusimap/cyrus-imapd/issues/1731#issuecomment-273064554
  3. https://github.com/notifications/unsubscribe-auth/AABE7RL4_3eoSr4a-DXoItSTvMc4QJzlks5rTIosgaJpZM4LjuFv
anatoli26 commented 7 years ago

+ change Sieve to spawn a new process per each processing of untrusted data (e.g. per each incoming message) that would do pure seccomp before starting the data analysis (not even seccomp-bpf): just read a single message file per process invocation and write to a single file the results for the master process to take an action