Japanese text in window titles, bookmark mgr, subject line in mailnews does not render

[GoogleCodeExporter]

However, it *does* render in the body.

Original issue reported on code.google.com by classi...@floodgap.com on 1 Nov 2009 at 7:58

• Blocking: #23

[GoogleCodeExporter]

From reporter,
4.  A couple more places where Japanese text is displayed as ?????'s:
a.  In the title bar of windows.  (it works fine in page bodies)
b.  In the bookmark manager.

Original comment by classi...@floodgap.com on 5 Nov 2009 at 6:00

[GoogleCodeExporter]

Related bugs:

M36689
https://bugzilla.mozilla.org/show_bug.cgi?id=36689
M92503
https://bugzilla.mozilla.org/show_bug.cgi?id=92503

This one has a patch for OS X, but it's commented out for OS 9. M36689 was 
wontfixed but mostly because OS 9 support was being withdrawn.

Original comment by classi...@floodgap.com on 1 Aug 2010 at 7:47

• Changed title: Japanese text in window titles, bookmark mgr, subject line in mailnews does not render

[GoogleCodeExporter]

This is a good test case
http://ja.wikipedia.org/wiki/%E3%83%95%E3%82%A1%E3%82%A4%E3%83%AB:Android-2.0.pn
g

Original comment by classi...@floodgap.com on 1 Aug 2010 at 7:49

[GoogleCodeExporter]

iCab does this wrong, but IE 5 does this *right*, at least for titles, so let's 
fix window titles first.

M92503 changed widget/nsMacWindow::SetTitle to call (in OS X) CoreFoundation's 
CFStringCreateWithCharacters and convert the string. However, Mozilla appears 
to have the equivalent classic code already. In 
widget/nsMacControl::NSStringSetControlTitle, it also calls 
CFStringCreateWithCharacters, but has an alternative path for OS 8/9. Perhaps 
we can adapt this. Upping to high so I take a look at this later.

Original comment by classi...@floodgap.com on 1 Aug 2010 at 8:40

• Added labels: Priority-High
• Removed labels: Priority-Medium

[GoogleCodeExporter]

That code wasn't what worked, but it gave us a clue. Here is what we are doing 
now:

//-------------------------------------------------------------------------
//
// Set this window's title
//
//-------------------------------------------------------------------------
NS_IMETHODIMP nsMacWindow::SetTitle(const nsString& aTitle)
{
#if TARGET_CARBON
  if(nsToolkit::OnMacOSX()) {
    // On MacOS X try to use the unicode friendly CFString version first
    CFStringRef labelRef = ::CFStringCreateWithCharacters(kCFAllocatorDefault, (UniChar*)aTitle.get(), aTitle.Length());
    if(labelRef) {
      ::SetWindowTitleWithCFString(mWindowPtr, labelRef);
      ::CFRelease(labelRef);
      return NS_OK;
    }
  }
#else
/* See 
http://developer.apple.com/mac/library/samplecode/FSCopyObject/Listings/Sources_
FSCopyObject_c.html and
    http://developer.apple.com/mac/library/documentation/Carbon/Conceptual/ProgWithTECM/tecmgr_about/tecmgr_about.html */

// Classilla issue 71
// The problem is that we have no idea what encoding will work, so we have to 
try a few.

    OSErr                   err;
    UnicodeToTextInfo       unicodeTextInfo;
    const PRUnichar*        unicodeText;
    char*                   scriptText;
    size_t                  unicodeTextLengthInBytes, unicodeTextReadInBytes,
                            scriptTextSizeInBytes, scriptTextLengthInBytes;
    TextEncoding            textEncoding;
    int i;
    int j;

    // get the Unicode text and prepare buffers
    unicodeText = aTitle.get();
    unicodeTextLengthInBytes = aTitle.Length() * sizeof(PRUnichar);
    scriptTextSizeInBytes = unicodeTextLengthInBytes * 2;
    scriptText = new char[scriptTextSizeInBytes];

    // Only put ones here we think people are likely to need, otherwise we might slow down.
    // Asian languages have priority because we have a lot of Japanese Classilla users, but most
    // things, praise the Lord and Ford, will be amenable to MacRoman. List in order of expected
    // conversion frequency.
    const TextEncoding mappingsToTry[] = { kTextEncodingMacRoman, 
                                           kTextEncodingMacJapanese, // Nihon no Classilla user wa daisuki da yo!
                                           kTextEncodingMacChineseTrad,
                                           kTextEncodingMacChineseSimp,
                                           kTextEncodingMacKorean,
                                           kTextEncodingMacCyrillic,
                                           kTextEncodingMacCentralEurRoman }; // from TextCommon.h
    // j below needs to be sizeof!

    // Basically do this until we find an encoding that works, or give up.
    j = 7; // number of text encodings to try. KEEP THIS IN SYNC
    for (i = 0; i<j; i++) {

        // create the textinfo object
        textEncoding = mappingsToTry[i];
        err = ::CreateUnicodeToTextInfoByEncoding(textEncoding, &unicodeTextInfo);
        if (err == noErr) {
            err = ::ConvertFromUnicodeToText(unicodeTextInfo, unicodeTextLengthInBytes, NS_REINTERPRET_CAST(const PRUint16*, unicodeText),
                0, /* no flags */
                // kUnicodeLooseMappingsMask, // ????
                0, NULL, 0, NULL,   /* offsetCounts & offsetArrays */
                scriptTextSizeInBytes, &unicodeTextReadInBytes, &scriptTextLengthInBytes,
                scriptText);
        }
        if (err == noErr) { 
            scriptText[scriptTextLengthInBytes] = 0;    // null terminate
            ::SetWTitle(mWindowPtr, c2pstr(scriptText));
            delete [] scriptText;
            return NS_OK;
        }
  }
  delete [] scriptText;
#endif  
// end issue

  // We're out of options. Give up and let nsMacControl try to convert it, and throw in ?'s for what we can't do.
  Str255 title;
  // unicode to file system charset
  nsMacControl::StringToStr255(aTitle, title);
  ::SetWTitle(mWindowPtr, title);
  return NS_OK;
}

Notice that we try as many encodings as we can to get it working. In fact, we 
do this better than IE 5 now. Not only do we seem to encode MacJapanese 
correctly, but we also handle a lot of the less frequent encodings, and we 
still have a fallback. Marking Verified. This should make Sakuya happy. :)

Original comment by classi...@floodgap.com on 9 Aug 2010 at 3:35

• Changed state: Verified

[GoogleCodeExporter]

Forgot some disposes in there; added in.

Original comment by classi...@floodgap.com on 14 Aug 2010 at 3:35

[GoogleCodeExporter]

Reopening. We don't work right on systems without the right language kits 
installed (the native menus now just "empty menu item"). Let's try 
ConvertFromUnicodeToTextRun.

http://developer.apple.com/mac/library/documentation/Carbon/Reference/Text_Encod
in_sion_Manager/Reference/reference.html

Original comment by classi...@floodgap.com on 14 Aug 2010 at 5:07

• Changed state: Started
• Added labels: Priority-Critical
• Removed labels: Priority-High

[GoogleCodeExporter]

Second pass.

// Second pass using TextRunInfo.
#define MAX_RUNS 4
    OSErr                   err;
    UnicodeToTextRunInfo    unicodeTextInfo;
    const PRUnichar*        unicodeText;
    char*                   scriptText;
    size_t                  unicodeTextLengthInBytes, unicodeTextReadInBytes,
                            scriptTextSizeInBytes, scriptTextLengthInBytes, scriptRunCount;
    TextEncodingRun         textEncodingRuns[MAX_RUNS]; /* If it's more complex than this, we have a problem. */

    // get the Unicode text and prepare buffers
    unicodeText = aTitle.get();
    unicodeTextLengthInBytes = aTitle.Length() * sizeof(PRUnichar);
    scriptTextSizeInBytes = unicodeTextLengthInBytes * 2;
    scriptText = new char[scriptTextSizeInBytes];

    // create the conversion object
    err = ::CreateUnicodeToTextRunInfo(0, NULL, &unicodeTextInfo); /* use any available script */
    if (err == noErr) {
        err = ::ConvertFromUnicodeToTextRun(unicodeTextInfo, unicodeTextLengthInBytes, NS_REINTERPRET_CAST(const PRUint16*, unicodeText),
            kUnicodeKeepSameEncodingMask | kUnicodeTextRunMask,
            0, NULL, 0, NULL,   /* offsetCounts & offsetArrays */
            scriptTextSizeInBytes, &unicodeTextReadInBytes, &scriptTextLengthInBytes,
            scriptText,
            MAX_RUNS, &scriptRunCount, textEncodingRuns);
        ::DisposeUnicodeToTextRunInfo(&unicodeTextInfo);
    }

    if (err == noErr) { // || err == kTECUsedFallbacksStatus) { 
        scriptText[scriptTextLengthInBytes] = 0;    // null terminate
        ::SetWTitle(mWindowPtr, c2pstr(scriptText));
        delete [] scriptText;
        return NS_OK;
    }

    delete [] scriptText;

Original comment by classi...@floodgap.com on 14 Aug 2010 at 5:37

[GoogleCodeExporter]

This works better in the title bar, but still makes menus fail (no difference 
if the Language Kit is installed -- still works fine there). Changed 
nsMenuBar.cpp/SetMenuItemText:

void 
MenuHelpers :: SetMenuItemText ( MenuHandle inMacMenuHandle, short inMenuItem, 
const nsString& inMenuString,
                                   const UnicodeToTextRunInfo inConverter )
{
  // ::TruncString() doesn't take the number of characters to truncate to, it takes a pixel with
  // to fit the string in. Ugh. I talked it over with sfraser and we couldn't come up with an 
  // easy way to compute what this should be given the system font, etc, so we're just going
  // to hard code it to something reasonable and bigger fonts will just have to deal.
  const short kMaxItemPixelWidth = 300;

  short themeFontID;
  SInt16 themeFontSize;
  Style themeFontStyle;
  Str255 menuTitle;
  OSErr err;

  char* scriptRunText = ConvertToScriptRun ( inMenuString, inConverter, &themeFontID,
                                               &themeFontSize, &themeFontStyle );
  if ( scriptRunText ) {    
    // convert it to a pascal string
    short scriptRunTextLength = strlen(scriptRunText);
    if (scriptRunTextLength > 255)
      scriptRunTextLength = 255;
    BlockMoveData(scriptRunText, &menuTitle[1], scriptRunTextLength);
    menuTitle[0] = scriptRunTextLength;

    // if the item text is too long, truncate it.
    ::TruncString ( kMaxItemPixelWidth, menuTitle, truncMiddle );
    ::SetMenuItemText(inMacMenuHandle, inMenuItem, menuTitle);
    err = ::SetMenuItemFontID(inMacMenuHandle, inMenuItem, themeFontID);    

    nsMemory::Free(scriptRunText);
  } else {
    // Problem converting/setting the menu item. This might happen if we did the
    // conversion already and we have incomplete script support. (Classilla issue 71)

    // Try converting using the file system charset. This is the nsMacWindow::SetTitle fallback
    // so it should work for us too.
    nsMacControl::StringToStr255(inMenuString, menuTitle);

    // Truncate the string to the pixel width and install it in the menu.
    ::TruncString ( kMaxItemPixelWidth, menuTitle, truncMiddle );
    ::SetMenuItemText(inMacMenuHandle, inMenuItem, menuTitle);

    // Figure out our system font and use that. This doesn't always work (see qa issue 78983),
    // but works most of the time. (Also consider kThemeSmallSystemFont)
    Str255 themeFontName;
    SInt16 outFontSize;
    Style outFontStyle;
    err = ::GetThemeFont(kThemeSystemFont, smSystemScript, themeFontName, &outFontSize, &outFontStyle);
    NS_ASSERTION(err==noErr,"nsMenu::NSStringSetMenuItemText: GetThemeFont failed.");
    if ( err ) { return; } // give up
    ::GetFNum(themeFontName, &themeFontID);

    err = ::SetMenuItemFontID(inMacMenuHandle, inMenuItem, themeFontID);
  } 
} // SetMenuItemText

Original comment by classi...@floodgap.com on 14 Aug 2010 at 6:51

[GoogleCodeExporter]

This looks like a winner. Keeping open while I test on the 1400, 7300 and 
TiBook.

Original comment by classi...@floodgap.com on 14 Aug 2010 at 9:15

[GoogleCodeExporter]

Marking fixed (but see issue 23 for an important difference).

Original comment by classi...@floodgap.com on 14 Aug 2010 at 10:27

[GoogleCodeExporter]

Original comment by classi...@floodgap.com on 17 Aug 2010 at 2:47

• Changed state: Verified

[GoogleCodeExporter]

// ::TruncString() doesn't take the number of characters to truncate to, it 
takes a pixel with
// to fit the string in. Ugh. I talked it over with sfraser and we couldn't 
come up with an 
// easy way to compute what this should be given the system font, etc, so we're 
just going
// to hard code it to something reasonable and bigger fonts will just have to 
deal.
Truncating a pascal string is easy, just change the length byte.

Original comment by yuhongba...@hotmail.com on 24 Jan 2011 at 3:54

[GoogleCodeExporter]

Right, but we don't know how big an arbitrary font will be for an arbitrary 
number of characters.

Original comment by classi...@floodgap.com on 26 Jan 2011 at 5:49