Bug with Russian language in attachments

rcubetrac commented 15 years ago

Reported by Anton on 27 Jan 2009 15:15 UTC as Trac ticket #1485705

Hi! (Sorry for my poor English!)

When I send message with attachment from Roundcube-webmail to any other Mail-client, everything is ok. I can see normally all letters, titles e.t.c.

When I send message from any client and view it in roundcube-webmail everything is also good. All symbols are displayed correctly.

(For example, I send message with attachment " LVM.doc" )

But when I send message from rouncube-webmail and try to view it in roundcube-webmail I see, that a lot of symbols are missed, and the title of attachment looks like "?.doc"

In "Sent" folder the title of attachment looks in the same way, "?.doc"

When I try to open this message and then download this file with my web-browser I see, that symbols wasn't converted from UTF-8.

I've attached screenshot to display the problem.

All this problems are occured ONLY when I send messages from roundcube and then receive it in roundcube. From any other client everything is good.

Thank you!

Keywords: russian symbols in attachments Migrated-From: http://trac.roundcube.net/ticket/1485705

rcubetrac commented 15 years ago

Comment by tensor on 28 Jan 2009 01:21 UTC

Please send me the problem file to dennis@nikolaenko.ru. What is the version of Firefox?

rcubetrac commented 15 years ago

Comment by tensor on 28 Jan 2009 01:22 UTC

to "dennis at nikolaenko dot ru" actually.

rcubetrac commented 15 years ago

Comment by Anton on 28 Jan 2009 14:33 UTC

The version of Firefox is 2.0.0.20

But this problem is in IE too.

I've sent message with some attached example files to "dennis at nikolaenko dot ru".

This is the only one bug with russian symbols in roundcube. Maybe, name of the file isn't converted correctly, when roundcube try to upload file to the server.

rcubetrac commented 15 years ago

Comment by Anton on 29 Jan 2009 15:49 UTC

I try to solve this bug. I've noticed, when I compose message and attach the file, the name of the file in database (field "vars" of the table "session") is displayed correctly.

language|s:5:"ru_RU";auth_time|i:1233240458;task|s:4:"mail";user_id|s:1:"1";username|s:10:"a.testuser";imap_host|s:13:"mx.example.com";imap_port|i:143;imap_ssl|N;password|s:12:"ZOvbxIR7E=";login_time|i:1233240470;timezone|d:3;imap_root|s:5:"INBOX";imap_delimiter|s:1:"/";mbox|s:0:"";sort_col|s:4:"date";sort_order|s:4:"DESC";list_columns|a:6:{i:0;s:7:"subject";i:1;s:4:"from";i:2;s:4:"date";i:3;s:4:"size";i:4;s:4:"flag";i:5;s:10:"attachment";}quota_display|s:5:"image";safe_messages|a:6:{i:59;b:0;i:23555;b:0;i:60;b:0;i:23558;b:0;i:62;b:0;i:23559;b:0;}page|i:1;compose|a:3:{s:2:"id";s:23:"10292122614981ca0c997f3";s:5:"param";a:3:{s:5:"_task";s:4:"mail";s:7:"_action";s:7:"compose";s:5:"_mbox";s:5:"INBOX";}s:11:"attachments";a:1:{i:0;a:3:{s:4:"name";s:63:"   .txt";s:8:"mimetype";s:10:"text/plain";s:4:"path";s:20:"temp/rcmAttmntkM8oDC";```

So, maybe the problem appears, when roundcube select the name of the file from the table session.

I've noticed, that the problem appears, when there are more than 9 russian chars in the name of the file. If filename contains less than 9 chars, i see correct name in message and in SENT Folder.

Database is in UTF-8

rcubetrac commented 15 years ago

Comment by simplexe on 2 Feb 2009 08:36 UTC

confirm this bug. i using old version mime library (directory Mail).

rcubetrac commented 15 years ago

Comment by @alecpl on 2 Feb 2009 09:40 UTC

It works for me with svn-trunk, using postgresql and bundled MIME libraries.

rcubetrac commented 15 years ago

Comment by Anton on 3 Feb 2009 06:36 UTC

I found the file, where russian symbols are truncated. /program/lib/Mail/mimePart.php (line 455)

function _buildRFC2047Param($name, $value, $charset, $encoding='quoted-printable', $maxLength=75)

When I set $maxLength to 775 for example, I see, that the name of the file displayed correctly.

But it's not the right resolving of the problem, because when I set the value to 775 it's not RFC2047.

For example, the length of the string " .txt" is 63 symbols. I don't understand, why this string is truncated after nine Russian symbol.

With Mail_mime befor 1.4.0RC3 there is no this problem, but content of the text files are not displayed in roundcube.

With Mail_mime 1.4.0RC4 and Mail_mime 1.4.0 the russian name of the file isn't displayed, but text files, which roundcube opens as text/plain are displyed correctly independently of character encoding of the file (cp-1251, koi8r e.t.c).

rcubetrac commented 15 years ago

Comment by @alecpl on 3 Feb 2009 07:46 UTC

Use Mail_mime files from roundcube (1.4.0 it's very old version) and confirm if this helps, please.

rcubetrac commented 15 years ago

Comment by Anton on 3 Feb 2009 09:43 UTC

Now I'm using roundcube from svn-trunk and mysql-5.0.37 as a database. And of course Mail_mime files from roundcube.

It doesn't work correctly.

But when I edit the file /program/lib/Mail/mimePart.php (line 455) as I wrote upper (I set the value of $maxLength from 75 to 775, for example)

function _buildRFC2047Param($name, $value, $charset, $encoding='quoted-printable', $maxLength=775)

it works.

So, I think, that the length of string with Russian symbols doesn't count correctly.

rcubetrac commented 15 years ago

Comment by @alecpl on 3 Feb 2009 09:57 UTC

I've used your filename and it works for me.

rcubetrac commented 15 years ago

Comment by @alecpl on 3 Feb 2009 10:05 UTC

... show attachment headers source from the broken message. Also check with other imap client e.g. Thunderbird.

rcubetrac commented 15 years ago

Comment by Anton on 3 Feb 2009 11:00 UTC

In Thunderbird everything is OK.

attachment headers:

User-Agent: RoundCube Webmail/0.2
Content-Type: multipart/mixed;
    boundary="=_45adf211bff11221ee0768dffc56ef3b"
X-Virus-Scanned: Antivirus scanner

--=_45adf211bff11221ee0768dffc56ef3b
Content-Transfer-Encoding: 8bit
Content-Type: text/plain; charset="UTF-8"

--=_45adf211bff11221ee0768dffc56ef3b
Content-Transfer-Encoding: base64
Content-Type: text/plain; charset="UTF-8";
 name="=?UTF-8?Q?=D0=B3=D0=B5=D0=BD=D0=B5=D1=80=D0=B0=D1=86=D0=B8=D1=8F_=D0.txt?="; 
Content-Disposition: attachment;
 filename*0*="UTF-8''%D0%B3%D0%B5%D0%BD%D0%B5%D1%80%D0%B0%D1%86%D0%B8%D1%8F "
 filename*1*="%D0%BA%D0%BB%D1%8E%D1%87%D0%B5%D0%B9 %D0%B8 %D1%81%D0%B5%D1%80"
 filename*2*="%D1%82%D0%B5%D1%84%D0%B8%D0%BA%D0%B0%D1%82%D0%BE%D0%B2.txt";

rcubetrac commented 15 years ago

Comment by @alecpl on 3 Feb 2009 11:10 UTC

So, it's a problem with displaying, not sending. Check BODYSTRUCTURE response from the IMAP server.

rcubetrac commented 15 years ago

Comment by Anton on 3 Feb 2009 13:36 UTC

This is bodystructure response from the problem message

a002 fetch 13 bodystructure
* 13 FETCH (BODYSTRUCTURE (("TEXT" "PLAIN" ("CHARSET" "UTF-8") NIL NIL "8BIT" 2 1 NIL NIL NIL)("TEXT" "PLAIN" ("CHARSET"
 "UTF-8" "NAME" "=?UTF-8?Q?=D0=B3=D0=B5=D0=BD=D0=B5=D1=80=D0=B0=D1=86=D0=B8=D1=8F_=D0.txt?=") NIL NIL "BASE64" 2166 27 N
IL ("ATTACHMENT" NIL) NIL) "MIXED" ("BOUNDARY" "=_04629671c376027040dfacb605daeffa") NIL NIL))
a002 OK Completed (0.000 sec)

rcubetrac commented 15 years ago

Comment by Anton on 3 Feb 2009 14:18 UTC

My Imap server is Cyrus-Imapd 2.3.7

rcubetrac commented 15 years ago

Comment by @alecpl on 3 Feb 2009 18:40 UTC

From dovecot-1.0:

... ("text" "plain" ("charset" "UTF-8" "name" "=?UTF-8?Q?=D0=A3=D0=BF=D1=80=D0=B0=D0=B2=D0=BB=D0=B5=D0=BD=D0=B8=D0.doc?=") NIL NIL "base64" 4 0 NIL ("attachment" ("filename*0*" "UTF-8''%D0%A3%D0%BF%D1%80%D0%B0%D0%B2%D0%BB%D0%B5%D0%BD%D0%B8")) NIL) "mixed" ("boundary" "=_187863f35f99b45aaafd8b9719b1df48") NIL NIL)

So, why cyrus isn't returning filename0 info? That's why roundcube isn't able to display the whole filename.

rcubetrac commented 15 years ago

Comment by Anton on 4 Feb 2009 09:50 UTC

Today I've sent two same messages with problem file. One message from roundcube and the second message from Thunderbird. Then I checked bodystructure answer.

From roundcube:

a003 fetch 11 bodystructure
* 11 FETCH (BODYSTRUCTURE (("TEXT" "PLAIN" ("CHARSET" "UTF-8") NIL NIL "8BIT" 15 0 NIL NIL NIL)("TEXT" "PLAIN" ("CHARSET
" "UTF-8" "NAME" "=?UTF-8?Q?=D0=B3=D0=B5=D0=BD=D0=B5=D1=80=D0=B0=D1=86=D0=B8=D1=8F_=D0.txt?=") NIL NIL "BASE64" 2166 27
NIL ("ATTACHMENT" NIL) NIL) "MIXED" ("BOUNDARY" "=_126b0b8cc2b324c733dd48782aa313ca") NIL NIL))
a003 OK Completed (0.000 sec)

From Thunderbird:

a003 fetch 12 bodystructure
* 12 FETCH (BODYSTRUCTURE (("TEXT" "PLAIN" ("CHARSET" "UTF-8" "FORMAT" "flowed") NIL NIL "7BIT" 2 1 NIL NIL NIL)("TEXT"
"PLAIN" ("NAME" "=?UTF-8?B?0LPQtdC90LXRgNCw0YbQuNGPINC60LvRjtGH0LXQuSDQuCDRgdC10YDRgtC10YTQuA==?==?UTF-8?B?0LrQsNGC0L7Qs
i50eHQ=?=") NIL NIL "BASE64" 2170 29 NIL ("INLINE" ("FILENAME*" {196}
UTF-8''%D0%B3%D0%B5%D0%BD%D0%B5%D1%80%D0%B0%D1%86%D0%B8%D1%8F%20%D0%BA%D0%BB%D1%8E%D1%87%D0%B5%D0%B9%20%D0%B8%20%D1%81%D
0%B5%D1%80%D1%82%D0%B5%D1%84%D0%B8%D0%BA%D0%B0%D1%82%D0%BE%D0%B2%2E%74%78%74)) NIL) "MIXED" ("BOUNDARY" "------------030
502050909050508000904") NIL NIL))
a003 OK Completed (0.000 sec)

So, when the message has been sent from Thunderbird, cyrus is returning filename.

Maybe it's useful.

Now I'm trying to check cyrus-imapd configuration and find why it isn't returning filename.

rcubetrac commented 15 years ago

Comment by @alecpl on 4 Feb 2009 10:11 UTC

Please, show the difference in message sources. I see that Thunderbird sent INLINE attachment. Maybe try to ask cyrus developers (?)

rcubetrac commented 15 years ago

Comment by Anton on 4 Feb 2009 10:33 UTC

From thunderbird:

User-Agent: Thunderbird 2.0.0.19 (Windows/20081209)
MIME-Version: 1.0
To: =?UTF-8?B?0JrRg9C00LjQvdC+0LIg0JDQvdGC0L7QvSDQktC40LrRgtC+0YDQvtCy0LjRhw==?=
 <a.koudinov@xxxcom.com>
Subject: =?UTF-8?B?0YHQtdCz0L7QtNC90Y8z?=
Content-Type: multipart/mixed;
 boundary="------------030502050909050508000904"
X-Virus-Scanned: Antivirus scanner

This is a multi-part message in MIME format.
--------------030502050909050508000904
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

--------------030502050909050508000904
Content-Type: text/plain;
 name="=?UTF-8?B?0LPQtdC90LXRgNCw0YbQuNGPINC60LvRjtGH0LXQuSDQuCDRgdC10YDRgtC10YTQuA==?==?UTF-8?B?0LrQsNGC0L7Qsi50eHQ=?="
Content-Transfer-Encoding: base64
Content-Disposition: inline;
 filename*0*=UTF-8''%D0%B3%D0%B5%D0%BD%D0%B5%D1%80%D0%B0%D1%86%D0%B8%D1%8F;
 filename*1*=%20%D0%BA%D0%BB%D1%8E%D1%87%D0%B5%D0%B9%20%D0%B8%20%D1%81%D0;
 filename*2*=%B5%D1%80%D1%82%D0%B5%D1%84%D0%B8%D0%BA%D0%B0%D1%82%D0%BE%D0;
 filename*3*=%B2%2E%74%78%74

From roundcube:

User-Agent: RoundCube Webmail/0.2
Content-Type: multipart/mixed;
    boundary="=_126b0b8cc2b324c733dd48782aa313ca"
X-Virus-Scanned: Antivirus scanner

--=_126b0b8cc2b324c733dd48782aa313ca
Content-Transfer-Encoding: 8bit
Content-Type: text/plain; charset="UTF-8"

2
--=_126b0b8cc2b324c733dd48782aa313ca
Content-Transfer-Encoding: base64
Content-Type: text/plain; charset="UTF-8";
 name="=?UTF-8?Q?=D0=B3=D0=B5=D0=BD=D0=B5=D1=80=D0=B0=D1=86=D0=B8=D1=8F_=D0.txt?="; 
Content-Disposition: attachment;
 filename*0*="UTF-8''%D0%B3%D0%B5%D0%BD%D0%B5%D1%80%D0%B0%D1%86%D0%B8%D1%8F "
 filename*1*="%D0%BA%D0%BB%D1%8E%D1%87%D0%B5%D0%B9 %D0%B8 %D1%81%D0%B5%D1%80"
 filename*2*="%D1%82%D0%B5%D1%84%D0%B8%D0%BA%D0%B0%D1%82%D0%BE%D0%B2.txt";

And when the message was sent from thunderbird it's displayed correctly in roundcube.

rcubetrac commented 15 years ago

Comment by Anton on 4 Feb 2009 12:29 UTC

I've also sent message to mail.ru and gmail.com

In mail.ru I see " txt" In gmail.com I see " .txt"

I sent messages through MTA Postfix and through MTA Sendmail. The same result.

Then I've sent message to mail.ru and gmail.com from Thunderbird and I see the whole name of the file " .txt"

So, maybe it's something wrong during sending message, not displaying?

rcubetrac commented 15 years ago

Comment by @alecpl on 4 Feb 2009 12:33 UTC

No, displaying. Check mime_param_folding option setting.

rcubetrac commented 15 years ago

Comment by Anton on 4 Feb 2009 12:52 UTC

If I set mime_param_folding=0:

I don't see the filename in Roundcube, as there is no attachment (If the filename in English I see it);
I see UTF-8''%D0%B3%D0%B5%D0%BD%D0%B5%D1%80%D0%B0%D1%86%D0%B8%D1%8F %D0 in gmail;
I see whole correctly name in Thunderbird;

If I set mime_param_folding=2:

I see the filename in Roundcube, as I descriped upper, " .txt", but I don't see the content of the text file;
I see .txt in gmail;
I see ?.txt in Thunderbird

If I set mime_param_folding=1:

I see the filename in Roundcube, as I descriped upper, " .txt";
I see .txt in gmail;
I see whole correctly name in Thunderbird;

rcubetrac commented 15 years ago

Comment by @alecpl on 4 Feb 2009 13:08 UTC

Now I give up. You should ask cyrus imap developers why bodystructure response doesn't contain the filename.

rcubetrac commented 15 years ago

Comment by Anton on 4 Feb 2009 15:12 UTC

I'll try to ask.

But why is the filename displayed correctly n roundcube when I send the same message from gmail? And bodystructure response contain the filenmae!

a003 fetch 9 bodystructure
* 9 FETCH (BODYSTRUCTURE (("TEXT" "PLAIN" ("CHARSET" "UTF-8") NIL NIL "8BIT" 2 1 NIL NIL NIL)("TEXT" "PLAIN" ("CHARSET"
"UTF-8" "NAME" "install amavisd CentOS.txt") NIL NIL "BASE64" 4856 62 NIL ("ATTACHMENT" ("FILENAME" "install amavisd Cen
tOS.txt")) NIL)("TEXT" "PLAIN" ("CHARSET" "UTF-8" "NAME" "=?UTF-8?Q?=D0=B3=D0=B5=D0=BD=D0=B5=D1=80=D0=B0=D1=86=D0=B8=D1=
8F_=D0.txt?=") NIL NIL "BASE64" 2166 27 NIL ("ATTACHMENT" NIL) NIL) "MIXED" ("BOUNDARY" "=_bade97c7a2c220ff064c88eba5f7b
4a6") NIL NIL))
a003 OK Completed (0.000 sec)

rcubetrac commented 15 years ago

Comment by Anton on 4 Feb 2009 15:13 UTC

Sorry,

the right one from gmail

a003 fetch 9 bodystructure
* 9 FETCH (BODYSTRUCTURE ((("TEXT" "PLAIN" ("CHARSET" "KOI8-R") NIL NIL "BASE64" 8 0 NIL NIL NIL)("TEXT" "HTML" ("CHARSE
T" "KOI8-R") NIL NIL "BASE64" 12 0 NIL NIL NIL) "ALTERNATIVE" ("BOUNDARY" "001485f7d2727439fd04621920c9") NIL NIL)("TEXT
" "PLAIN" ("CHARSET" "windows-1251" "NAME" "=?KOI8-R?B?x8XOxdLBw8nRIMvMwN7FyiDJINPF0tTFxsnLwdTP?=       =?KOI8-R?B?1y50e
HQ=?=") NIL NIL "BASE64" 2166 27 NIL ("ATTACHMENT" ("FILENAME" "=?KOI8-R?B?x8XOxdLBw8nRIMvMwN7FyiDJINPF0tTFxsnLwdTP?=
=?KOI8-R?B?1y50eHQ=?=")) NIL) "MIXED" ("BOUNDARY" "001485f7d272743a0504621920cb") NIL NIL))
a003 OK Completed (0.000 sec)

rcubetrac commented 15 years ago

Comment by @alecpl on 4 Feb 2009 18:23 UTC

The question is: why that bodystructure reply contains 'filename' item.

rcubetrac commented 15 years ago

Comment by Anton on 5 Feb 2009 07:38 UTC

Excuse me, You wrote, "So, why cyrus isn't returning filename0 info? That's why roundcube isn't able to display the whole filename"

Upper I've posted bodystructure response from message that was sent from gmail, and it's returning filename

("ATTACHMENT" ("FILENAME" "=?KOI8-R?B?x8XOxdLBw8nRIMvMwN7FyiDJINPF0tTFxsnLwdTP?=
=?KOI8-R?B?1y50eHQ=?=")) NIL)

And now, if I hear you aright, you wrote, "why bodystructure reply contains 'filename' item."

So, should I ask cyrus developers, why cyrus isn't returning filename0 info? Am I Right?

rcubetrac commented 15 years ago

Comment by @alecpl on 5 Feb 2009 08:14 UTC

Exactly. Send them the message source and BODYSTRUCTURE response.

rcubetrac commented 15 years ago

Comment by Anton on 5 Feb 2009 10:40 UTC

I sent them the question and I'm waiting for response.

rcubetrac commented 15 years ago

Comment by amatveev on 9 Feb 2009 21:52 UTC

I'd say Ticket #1485727 could be somehow related to this issue Check following page: http://trac.roundcube.net/ticket/148572

rcubetrac commented 15 years ago

Comment by Anton on 10 Feb 2009 07:27 UTC

Now I'm using roundcube from SVN-trunk (r2285/svn) and it works for me.

So, maybe the fix of the problem in this issue http://trac.roundcube.net/ticket/148572 helped me.

Thank you.

rcubetrac commented 15 years ago

Status changed by @alecpl on 10 Feb 2009 07:34 UTC

new => closed

roundcube / roundcubemail

Bug with Russian language in attachments #2058