search

roundcube / roundcubemail

The Roundcube Webmail suite
https://roundcube.net
GNU General Public License v3.0
5.67k stars 1.61k forks source link

Showing nothing in headers fall-down tab with large headers fields #3911

Closed rcubetrac closed 11 years ago

rcubetrac commented 11 years ago

Reported by mxm on 3 Sep 2012 06:23 UTC as Trac ticket #1488666

I found that nothing showing (just blank) in in headers fall-down tab when message contains large header field for example with Spamassassin report. See screenshot and sample of such message with big X-Spam_report: field.

Migrated-From: http://trac.roundcube.net/ticket/1488666

rcubetrac commented 11 years ago

Owner changed by @alecpl on 3 Sep 2012 07:03 UTC

=> alec

rcubetrac commented 11 years ago

Comment by @alecpl on 3 Sep 2012 07:42 UTC

Works for me with Chromium 18.0.1025.168 and Firefox. Enable imap_debug in Roundcube and check the log. Debug HTTP response, etc.

rcubetrac commented 11 years ago

Comment by mxm on 3 Sep 2012 07:55 UTC

Here is a IMAP log after clicking to "down" arrow on tab to show hidden headers (see attached log). All looks correct.

[10:46:29 +0300](03-Sep-2012): [S: * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE STARTTLS AUTH=CRAM-MD5 AUTH=PLAIN AUTH=LOGIN](0F52]) Dovecot ready!
[10:46:29 +0300](03-Sep-2012): [C: A0001 ID ("name" "Roundcube Webmail" "version" "0.8.1" "php" "5.4.6" "os" "FreeBSD" "command" "/?_task=mail&_action=check-recent&_mbox=INBOX&_list=1&_remote=1&_unlock=0&_=1346658385552")
[03-Sep-2012 10:46:29 +0300](0F52]): [S: * ID NIL
[03-Sep-2012 10:46:29 +0300](0F52]): [S: A0001 OK ID completed.
[03-Sep-2012 10:46:29 +0300](0F52]): [C: A0002 AUTHENTICATE CRAM-MD5
[03-Sep-2012 10:46:29 +0300](0F52]): [S: + PDUyODQ3Nzk1ODU2OTMzNzMuMTM0NjY1ODM4OUBiZXRhLnBlZWsubG9jYWw+
[03-Sep-2012 10:46:29 +0300](0F52]): [C: c2FsZXNAcGVlay5ydSBiYjk5NGQxOWE2NWEzZjFmY2UxYWU0NjRmMmY0YjIxMQ==
[03-Sep-2012 10:46:29 +0300](0F52]): [S: A0002 OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS SPECIAL-USE ACL RIGHTS=texk](0F52]) Logged in
[10:46:29 +0300](03-Sep-2012): [C: A0003 STATUS INBOX (MESSAGES UNSEEN)
[03-Sep-2012 10:46:29 +0300](0F52]): [S: * STATUS "INBOX" (MESSAGES 2931 UNSEEN 1)
[03-Sep-2012 10:46:29 +0300](0F52]): [S: A0003 OK Status completed.
[03-Sep-2012 10:46:29 +0300](0F52]): [C: A0004 SELECT INBOX
[03-Sep-2012 10:46:29 +0300](0F52]): [S: * FLAGS (\Answered \Flagged \Deleted \Seen \Draft $Forwarded $MDNSent Junk)
[03-Sep-2012 10:46:29 +0300](0F52]): [S: * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft $Forwarded $MDNSent Junk \*)](0F52]) Flags permitted.
[10:46:29 +0300](03-Sep-2012): [S: * 2931 EXISTS
[03-Sep-2012 10:46:29 +0300](0F52]): [S: * 0 RECENT
[03-Sep-2012 10:46:29 +0300](0F52]): [S: * OK [UNSEEN 2931](0F52]) First unseen.
[10:46:29 +0300](03-Sep-2012): [S: * OK [UIDVALIDITY 1292571622](0F52]) UIDs valid
[10:46:29 +0300](03-Sep-2012): [S: * OK [UIDNEXT 12883](0F52]) Predicted next UID
[10:46:29 +0300](03-Sep-2012): [S: * OK [NOMODSEQ](0F52]) No permanent modsequences
[10:46:29 +0300](03-Sep-2012): [S: A0004 OK [READ-WRITE](0F52]) Select completed.
[10:46:29 +0300](03-Sep-2012): [C: A0005 UID SEARCH 2931
[03-Sep-2012 10:46:29 +0300](0F52]): [S: * SEARCH 12882
[03-Sep-2012 10:46:29 +0300](0F52]): [S: A0005 OK Search completed (0.000 secs).
[03-Sep-2012 10:46:29 +0300](0F52]): [C: A0006 LOGOUT
[03-Sep-2012 10:46:29 +0300](0F52]): [S: * BYE Logging out
[03-Sep-2012 10:46:29 +0300](0F52]): [S: A0006 OK Logout completed.
[03-Sep-2012 10:46:36 +0300](0F52]): [S: * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE STARTTLS AUTH=CRAM-MD5 AUTH=PLAIN AUTH=LOGIN](C615]) Dovecot ready!
[10:46:36 +0300](03-Sep-2012): [C: A0001 ID ("name" "Roundcube Webmail" "version" "0.8.1" "php" "5.4.6" "os" "FreeBSD" "command" "/?_task=mail&_action=headers")
[03-Sep-2012 10:46:36 +0300](C615]): [S: * ID NIL
[03-Sep-2012 10:46:36 +0300](C615]): [S: A0001 OK ID completed.
[03-Sep-2012 10:46:36 +0300](C615]): [C: A0002 AUTHENTICATE CRAM-MD5
[03-Sep-2012 10:46:36 +0300](C615]): [S: + PDcyMDU4NTA3ODE5NjY5NTEuMTM0NjY1ODM5NkBiZXRhLnBlZWsubG9jYWw+
[03-Sep-2012 10:46:36 +0300](C615]): [C: c2FsZXNAcGVlay5ydSA1YzA2ZGZlOTM5MmYyMTgzNGFkYzgwYTliYzA5NWM5NQ==
[03-Sep-2012 10:46:36 +0300](C615]): [S: A0002 OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS SPECIAL-USE ACL RIGHTS=texk](C615]) Logged in
[10:46:36 +0300](03-Sep-2012): [C: A0003 SELECT Junk
[03-Sep-2012 10:46:36 +0300](C615]): [S: * FLAGS (\Answered \Flagged \Deleted \Seen \Draft $MDNSent $Forwarded Junk)
[03-Sep-2012 10:46:36 +0300](C615]): [S: * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft $MDNSent $Forwarded Junk \*)](C615]) Flags permitted.
[10:46:36 +0300](03-Sep-2012): [S: * 16 EXISTS
[03-Sep-2012 10:46:36 +0300](C615]): [S: * 0 RECENT
[03-Sep-2012 10:46:36 +0300](C615]): [S: * OK [UNSEEN 1](C615]) First unseen.
[10:46:36 +0300](03-Sep-2012): [S: * OK [UIDVALIDITY 1292571646](C615]) UIDs valid
[10:46:36 +0300](03-Sep-2012): [S: * OK [UIDNEXT 3515](C615]) Predicted next UID
[10:46:36 +0300](03-Sep-2012): [S: * OK [NOMODSEQ](C615]) No permanent modsequences
[10:46:36 +0300](03-Sep-2012): [S: A0003 OK [READ-WRITE](C615]) Select completed.
[10:46:36 +0300](03-Sep-2012): [C: A0004 UID FETCH 3514 (BODY.PEEK[HEADER](C615]))
[10:46:36 +0300](03-Sep-2012): [S: * 16 FETCH (UID 3514 BODY[HEADER](C615]) {3380}
[10:46:36 +0300](03-Sep-2012): [S: Return-path: <vat@setinfomail.ru>
[03-Sep-2012 10:46:36 +0300](C615]): [S: Envelope-to: info@peek.ru
[03-Sep-2012 10:46:36 +0300](C615]): [S: Delivery-date: Mon, 03 Sep 2012 10:34:33 +0300
[03-Sep-2012 10:46:36 +0300](C615]): [S: Received: from bzq-79-180-148-32.red.bezeqint.net ([79.180.148.32](C615]) helo=smtp.bezeqint.net)
[10:46:36 +0300](03-Sep-2012): [S:         by beta.peek.ru with esmtp (Exim 4.80 (FreeBSD))
[03-Sep-2012 10:46:36 +0300](C615]): [S:         (envelope-from <vat@setinfomail.ru>)
[03-Sep-2012 10:46:36 +0300](C615]): [S:         id 1T8RAq-0007bb-Mo
[03-Sep-2012 10:46:36 +0300](C615]): [S:         for info@peek.ru; Mon, 03 Sep 2012 10:34:33 +0300
[03-Sep-2012 10:46:36 +0300](C615]): [S: Message-ID: <C3AF9F63A25102151DFF77E11F0E1310@BZQ-79-180-148-32.RED.BEZEQINT.NET>
[03-Sep-2012 10:46:36 +0300](C615]): [S: From: "=?utf-8?B?0KDQtdCz0LjRgdGC0YDQsNGG0LjRjyDQvtGE0YTRiNC+0YDQvtCy?=" <vat@setinfomail.ru>
[03-Sep-2012 10:46:36 +0300](C615]): [S: To: <info@peek.ru>
[03-Sep-2012 10:46:36 +0300](C615]): [S: Date: Mon, 3 Sep 2012 11:33:34 +0400
[03-Sep-2012 10:46:36 +0300](C615]): [S: MIME-Version: 1.0
[03-Sep-2012 10:46:36 +0300](C615]): [S: Content-Type: multipart/alternative;
[03-Sep-2012 10:46:36 +0300](C615]): [S:         boundary="----=_NextPart_000_0073_01CD89C7.F3D25F00"
[03-Sep-2012 10:46:36 +0300](C615]): [S: X-Priority: 3
[03-Sep-2012 10:46:36 +0300](C615]): [S: X-MSMail-Priority: Normal
[03-Sep-2012 10:46:36 +0300](C615]): [S: Importance: Normal
[03-Sep-2012 10:46:36 +0300](C615]): [S: X-Mailer: Microsoft Windows Live Mail 15.4.3502.922
[03-Sep-2012 10:46:36 +0300](C615]): [S: X-MimeOLE: Produced By Microsoft MimeOLE V15.4.3502.922
[03-Sep-2012 10:46:36 +0300](C615]): [S: Received-SPF: softfail (beta.peek.ru: transitioning domain of setinfomail.ru does not designate 79.180.148.32 as permitted sender) client-ip=79.180.148.32; envelope-from=vat@setinfomail.ru; helo=smtp.bezeqint.net;
[03-Sep-2012 10:46:36 +0300](C615]): [S: X-Spam_score: 8.9
[03-Sep-2012 10:46:36 +0300](C615]): [S: X-Spam_score_int: 89
[03-Sep-2012 10:46:36 +0300](C615]): [S: X-Spam_bar: ++++++++
[03-Sep-2012 10:46:36 +0300](C615]): [S: X-Spam_report: Spam detection software, running on the system "beta.peek.local", has
[03-Sep-2012 10:46:36 +0300](C615]): [S:  identified this incoming email as possible spam.  The original message
[03-Sep-2012 10:46:36 +0300](C615]): [S:  has been attached to this so you can view it (if it isn't spam) or label
[03-Sep-2012 10:46:36 +0300](C615]): [S:  similar future email.  If you have any questions, see
[03-Sep-2012 10:46:36 +0300](C615]): [S:  The administrator of that system for details.
[03-Sep-2012 10:46:36 +0300](C615]): [S:
[03-Sep-2012 10:46:36 +0300](C615]): [S:  Content preview:   :   
[03-Sep-2012 10:46:36 +0300](C615]): [S:      : 14 , , 
[03-Sep-2012 10:46:36 +0300](C615]): [S:      " " [...](C615])
[10:46:36 +0300](03-Sep-2012): [S:
[03-Sep-2012 10:46:36 +0300](C615]): [S:  Content analysis details:   (8.9 points, 5.0 required)
[03-Sep-2012 10:46:36 +0300](C615]): [S:
[03-Sep-2012 10:46:36 +0300](C615]): [S:   pts rule name              description
[03-Sep-2012 10:46:36 +0300](C615]): [S:  ---- ---------------------- --------------------------------------------------
[03-Sep-2012 10:46:36 +0300](C615]): [S:   3.3 RCVD_IN_PBL            RBL: Received via a relay in Spamhaus PBL
[03-Sep-2012 10:46:36 +0300](C615]): [S:                              [79.180.148.32 listed in zen.spamhaus.org](C615])
[10:46:36 +0300](03-Sep-2012): [S:   3.5 BAYES_99               BODY: Bayes spam probability is 99 to 100%
[03-Sep-2012 10:46:36 +0300](C615]): [S:                              [score: 1.0000](C615])
[10:46:36 +0300](03-Sep-2012): [S:   0.0 HTML_MESSAGE           BODY: HTML included in message
[03-Sep-2012 10:46:36 +0300](C615]): [S:   1.1 DCC_CHECK              Detected as bulk mail by DCC (dcc-servers.net)
[03-Sep-2012 10:46:36 +0300](C615]): [S:   1.0 RDNS_DYNAMIC           Delivered to internal network by host with
[03-Sep-2012 10:46:36 +0300](C615]): [S:                              dynamic-looking rDNS
[03-Sep-2012 10:46:36 +0300](C615]): [S: X-SA-Status: Yes
[03-Sep-2012 10:46:36 +0300](C615]): [S: X-SA-Exim-Connect-IP: 79.180.148.32
[03-Sep-2012 10:46:36 +0300](C615]): [S: X-SA-Exim-Mail-From: vat@setinfomail.ru
[03-Sep-2012 10:46:36 +0300](C615]): [S: Subject: =?utf-8?B?0JzQtdC20LTRg9C90LDRgNC+0LTQvdC+0LUg0L3QsNC70L7Qs9C+0LLQvtC1INC/0LvQsNC90LjRgNC+0LLQsNC90LjQtQ==?=
[03-Sep-2012 10:46:36 +0300](C615]): [S: X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on beta.peek.local
[03-Sep-2012 10:46:36 +0300](C615]): [S: X-Spam-Flag: YES
[03-Sep-2012 10:46:36 +0300](C615]): [S: X-Spam-Level: ********
[03-Sep-2012 10:46:36 +0300](C615]): [S: X-Spam-Status: Yes, score=8.9 required=5.0 tests=BAYES_99,DCC_CHECK,
[03-Sep-2012 10:46:36 +0300](C615]): [S:         HTML_MESSAGE,RCVD_IN_PBL,RDNS_DYNAMIC autolearn=no version=3.3.2
[03-Sep-2012 10:46:36 +0300](C615]): [S: X-SA-Exim-Version: 4.2
[03-Sep-2012 10:46:36 +0300](C615]): [S: X-SA-Exim-Scanned: Yes (on beta.peek.ru)
[03-Sep-2012 10:46:36 +0300](C615]): [S: X-Spam-Description: message will be marked as SPAM for spam score >= 60
[03-Sep-2012 10:46:36 +0300](C615]): [S: X-Spam-Score: 100
[03-Sep-2012 10:46:36 +0300](C615]): [S: X-Spam-Legend: DNSBL(50) host is blacklisted by zen.spamhaus.org list; DOTHYP(10) too many dots and hyphens in hostname; HLNOTHST(20) HELO do not point to host address; SPFSOFL(20) SPF record can't absolutely confirm that senders host authorized to send mail from queried domain;
[03-Sep-2012 10:46:36 +0300](C615]): [S: X-Spam: Yes
[03-Sep-2012 10:46:36 +0300](C615]): [S:
[03-Sep-2012 10:46:36 +0300](C615]): [S: )
[03-Sep-2012 10:46:36 +0300](C615]): [S: A0004 OK Fetch completed.
[03-Sep-2012 10:46:36 +0300](C615]): [C: A0005 LOGOUT
[03-Sep-2012 10:46:36 +0300](C615]): [S: * BYE Logging out
[03-Sep-2012 10:46:36 +0300](C615]): [C615] S: A0005 OK Logout completed.
rcubetrac commented 11 years ago

Comment by mxm on 3 Sep 2012 08:10 UTC

Here is HTTP debug log (lighttpd/1.4.31) after opening e-mail message and clicking on "down" arrow at headers tab (see attachement).

rcubetrac commented 11 years ago

Comment by mxm on 3 Sep 2012 08:16 UTC

Just rechecked problem in Mozilla/5.0 (Windows NT 6.1; WOW64; rv:15.0) Gecko/20120826 Firefox/15.0 SeaMonkey/2.12 agent. Negative result - tab stills blank.

rcubetrac commented 11 years ago

Comment by @alecpl on 3 Sep 2012 08:32 UTC

Please, check the content of the HTTP response. In Firefox you can use Firebug for this.

rcubetrac commented 11 years ago

Comment by @alecpl on 3 Sep 2012 08:33 UTC

Does your browser show any error message? Disable antivirus software.

rcubetrac commented 11 years ago

Comment by mxm on 3 Sep 2012 08:38 UTC

Browsers works fine without any error. Just rechecked problem with disabled antivirus (Symantec Endpoint) - all still the same.

rcubetrac commented 11 years ago

Comment by mxm on 3 Sep 2012 08:46 UTC

Here is output of firebug 

HTTP GET: ./?_task=mail&_action=pagenav&_uid=3514&_mbox=Junk&_remote=1

app.js...5231947 ( 22)
GET https://beta.peek.ru/?_task=mail&_action=pagenav...e=1&_unlock=loading1346661781748&_=1346661781756

200 OK
        286ms   
jquery...5231931 ( 4)
JSONCookies

{"action":"pagenav","unlock":"loading1346661781748","env":{"next_uid":3512,"last_uid":3501,"messagecount":1},"exec":"this.enable_command(\"nextmessage\",\"lastmessage\",true);\nthis.set_rowcount(\"\\u0421\\u043e\\u043e\\u0431\\u0449\\u0435\\u043d\\u0438\\u0435 1 \\u0438\\u0437 16\");\n"}

HTTP GET: ./?_task=mail&_action=getunread&&_remote=1

app.js...5231947 ( 22)
GET https://beta.peek.ru/?_task=mail&_action=getunread&&_remote=1&_unlock=0&_=1346661781762

200 OK
        320ms   
jquery...5231931 ( 4)
JSONCookies

{"action":"getunread","exec":"this.set_unread_count(\"INBOX.price\",1,false);\nthis.set_unread_count(\"Junk\",15,false);\nthis.set_unread_count(\"public.mailing\",2,false);\nthis.set_unread_count(\"public.mailing.oki\",1,false);\nthis.set_unread_count(\"public.mailing.ricoh\",2,false);\nthis.set_unread_count(\"public.mailing.suppliers\",408,false);\nthis.set_unread_count(\"public.mailing.xerox\",30,false);\nthis.set_unread_count(\"public.news\",761,false);\n"}

this.enable_command("nextmessage","lastmessage",true);
this.set_rowcount("\u0421\u043e\u043e\u0431\u0449\u0435\u043d\u0438\u0435 1 \u0438\u0437 16");

app.js...5231947 ( 22)

this.set_unread_count("INBOX.price",1,false);
this.set_unread_count("Junk",15,false);
this.set_unread_count("public.mailing",2,false);
this.set_unread_count("public.mailing.oki",1,false);
this.set_unread_count("public.mailing.ricoh",2,false);
this.set_unread_count("public.mailing.suppliers",408,false);
this.set_unread_count("public.mailing.xerox",30,false);
this.set_unread_count("public.news",761,false);

app.js...5231947 ( 22)

HTTP POST: ./?_task=mail&_action=headers

app.js...5231947 ( 22)
POST https://beta.peek.ru/?_task=mail&_action=headers

200 OK
        370ms   
jquery...5231931 ( 4)
PostJSONCookies

{"action":"headers","unlock":"loading1346661784583","exec":"this.set_headers(\"\");\n"}

this.set_headers("");

Has no errors.

rcubetrac commented 11 years ago

Comment by @alecpl on 3 Sep 2012 09:04 UTC

Replying to mxm:

Here is output of firebug 

this.set_headers("");

Ok. Now you see the headers string is sent empty. Didn't you find any errors in Roundcube log? You could debug program/steps/mail/headers.inc. Add console($source); in line 25 and 39. Then check logs/console.

rcubetrac commented 11 years ago

Comment by mxm on 3 Sep 2012 10:13 UTC

Roundcube error log is says nothing. I done what you ask but second console($source); I insterted to line 41 to prevent breaking preg_replace(). Thats what I got:

[13:13:18 +0300](03-Sep-2012): NULL
[13:13:18 +0300](03-Sep-2012):
rcubetrac commented 11 years ago

Comment by mxm on 3 Sep 2012 10:19 UTC

After little play with checkpoints I found that $source variable cleaning after

        $source = htmlspecialchars(trim($source));

in program/steps/mail/headers.inc

rcubetrac commented 11 years ago

Comment by mxm on 3 Sep 2012 10:30 UTC

May be better to use ENT_SUBSTITUTE in this function?

rcubetrac commented 11 years ago

Comment by @alecpl on 3 Sep 2012 10:55 UTC

or ENT_IGNORE. So, you have PHP 5.4? In any case we need to detect constant existance before using it. You can try. Another solution would be to use some cleanup function e.g. rcube_charset::clean();

rcubetrac commented 11 years ago

Comment by mxm on 3 Sep 2012 11:11 UTC

Probably it will be better to use rcube_charset::clean(). So can you give me a replacement expression for this string?

   $source = htmlspecialchars(trim($source));

I will check it right now.

rcubetrac commented 11 years ago

Comment by mxm on 3 Sep 2012 11:17 UTC

   $source = htmlspecialchars(trim($source),ENT_SUBSTITUTE);

Works well. So problem was in e-mail message coding (codepage). But I think it will be better to use PHP version-independent solution such as I believe rcube_charset::clean() is.

rcubetrac commented 11 years ago

Comment by @alecpl on 3 Sep 2012 11:19 UTC

Add $source = rcube_charset::clean($source); before the line with htmlspecialchars.

rcubetrac commented 11 years ago

Comment by mxm on 3 Sep 2012 11:23 UTC

Replying to alec:

Add $source = rcube_charset::clean($source); before the line with htmlspecialchars.

Works well too. So, we are found a solution! Alec, thank you for your help. Hope you are will commit this patch to nearest release of Roundcube!

rcubetrac commented 11 years ago

Comment by @alecpl on 3 Sep 2012 17:02 UTC

Fixed in 10141cd6

rcubetrac commented 11 years ago

Status changed by @alecpl on 3 Sep 2012 17:02 UTC

new => closed

rcubetrac commented 11 years ago

Milestone changed by @alecpl on 19 Sep 2012 07:15 UTC

0.9-beta => 0.8.2