copy Chinese text from view got garbled string

[raidsan]

change font not help

[janwilmans]

Thanks for reporting, I have no idea how to debug this, I cannot read Chinese, could you provide a saved logfile and the expected outcome?

[raidsan]

just use c# Trace.WriteLine("some 中文"), in the view , It looks ok, export log to file is also ok, but copy to clipboard will became garbled

[raidsan]

same garbled in statusbar: searching: ...

[janwilmans]

thanks ! I have tried to reproduce, but it seem to behave different:

maybe I'm not doing it right, can you send a zip-file with the executable that works to email my janwilmans@gmail.com

Is there a specific font I should use? I used FangSong because it was installed here...

[raidsan]

hi, I test with the source code and do some change to make it show multibyte chars correct:

DebugViewPP\Win32Lib\Win32Lib.cpp: std::wstring MultiByteToWideChar(const char* str, int len) ::MultiByteToWideChar is not reliable , change to use mbstowcs

std::wstring MultiByteToWideChar(const char* str, int len)
{
    setlocale(LC_ALL, "");
    std::wstring ws(len, L'\0');
    ws.resize(std::mbstowcs(&ws[0], str, len)); // Shrink to fit.
    return ws;
}

DebugViewPP\DebugView++\LogView.cpp copy to Clipboard, change to use wchar:

void CLogView::Copy()
{
    std::ostringstream ss;
    std::wstring wstr;

    if (!m_highlightText.empty())
    {
        wstr = m_highlightText;
    }
    else
    {
        int item = -2;
        while ((item = GetNextItem(item, LVNI_ALL | LVNI_SELECTED)) >= 0)
        {
            ss << GetItemText(item) << "\r\n";
        }
        if (startItem == -1)
            return;
        const std::string& str = ss.str();
        wstr = WStr(str).str();
    }

    if (OpenClipboard())
    {
        EmptyClipboard();
        HGLOBAL hClipboardData = GlobalAlloc(GMEM_DDESHARE, 2 * (wcslen(wstr.c_str()) + 1)); 
        WCHAR* pchData = (WCHAR*)GlobalLock(hClipboardData);
        wcscpy(pchData, wstr.c_str());
        GlobalUnlock(hClipboardData);
        SetClipboardData(CF_UNICODETEXT, hClipboardData);
        CloseClipboard();
    }

status bar show "searching xxxx" void CMainFrame::UpdateStatusBar() call wstringbuilder() to compose wstring, but for wstringbuilder, the following code in "DebugViewPP\include\CobaltFusion\stringbuilder.h" will got garbled result:

    basic_stringbuilder& operator<<(const std::wstring& str)
    {
        m_ss << Str(str).str().c_str();
        return *this;
    }

so replace:

       std::wstring search = wstringbuilder() << L"Searching: \"" << isearch << L"\"";

with:

        std::wostringstream w;
    w << L"Searching: \"" << isearch << L"\"";
    std::wstring search = w.str();

still have another issue: when use left mouse button to select mbcs text, it has wrong to select double size of string, e.g: message string: "中文信息", if I select "中文" , after release left mouse button, it auto turn to highlight select range to: "中文信息"

[janwilmans]

Wow, nice work, I'll process this!

[janwilmans]

you propose to change the use of MultiByteToWideChar to std:: mbstowcs, and to copy data to the clipboard using the CF_UNICODETEXT flag, both changes sound good. However, I still cannot reproduce the issue.

If you run this program, do the characters show up in debugview++ ? (I see this:

Test program: showchinesechars.zip

[janwilmans]

I'm trying very hard to make sense of this, in your example, you used

std::mbstowcs preceeded by setlocale(LC_ALL, "");

This does not make sense to me, I think std::mbstowcs is not affected by setlocale, I think it should be std::setlocale from <clocale>

or even std::locale::global() from <locale>

However, I can't really make sense of any of it. Any nothing seems to convince debugview++ on my PC to display unicode characters... only ?? coming up

references:

• https://en.cppreference.com/w/cpp/locale/setlocale
• https://en.cppreference.com/w/cpp/locale/locale/global

[janwilmans]

Please try how this works: https://github.com/CobaltFusion/DebugViewPP/releases/tag/v1.8.0.21

I have tried to simulate the conversions in a test, but I still don't actually 'see' the behavior in debugview++ :) You did uncover a real problem in

basic_stringbuilder& operator<<(const std::wstring& str)

It was very much broken and could crash (now fixed)

[janwilmans]

I tried to set my windows locale to Chinese, aside from being really difficult to understand my system dialogs, it did not seem to effect debugview

I'm still getting ?? and ???? instead of proper characters.

[raidsan]

make sure set windows locale to chinese, for windows 10 english version: Setting: =>"Time & Language” => "Region & language – Additional date, time & regional settings”=>"Change date, time or number formats" => "Administrative - Change system locale…", choose: “Chinese（Simplified, China）", restart

[janwilmans]

I will try this. Did the new version I put up work any better?

[raidsan]

I just test v1.8.0.21 - version, it act same as the old version

[janwilmans]

That is very sad, it spend several hours on that :)

On the bright side: Changing this, as you suggested, worked:

see also #332

[janwilmans]

Hm I do see a difference, when copying a highlighted piece text, the amount of characters copied is off, but the encoding seems right (last character broken)

some 中文 and more 中文信o

When copying an entire line, the encoding is messed up:

some ÖÐÎÄ and more ÖÐÎÄÐÅÏ¢

[janwilmans]

reopened, to get confirmation that is it actually working correctly now. Here is what I tested:

The select and highlight but is not fixed yet, I have split that into a new issue #332

[janwilmans]

I've just put up v1.8.0.24, please re-test it :)

[raidsan]

yes, I test it copy text ok, except select and highlight text length

[raidsan]

but status text show: searching : .... is still garbled