IMR_DOCUMENTFEED

[maboroshin]

IMR_DOCUMENTFEED (I forgot the official name) is useful for Japanese IME (Maybe even in Chinese). However, it is mainly Japanese information. It is also mysterious because there is no official website. I tried to implement it myself last year. It ’s too difficult for me. I give up. I want you to implement it if possible.

This function distinguishes kanji with the same pronunciation. (As explanation. Actually more complicated)

The IME sends the following twice : msg:WM_IME_REQUEST/wParam:IMR_DOCUMENTFEED . So catch it : if (wParam == IMR_DOCUMENTFEED) .

• First time. IME sends: lParam:0 .
• The application returns the size. sizeof(RECONVERTSTRING)+lengthOfString(byte, including the terminating zero).
• The IME reserves a data area. The IME sends the pointer.
• Pseudo code to imagine it: struct { RECONVERTSTRING reconv; TCHAR compstr[n]; };
• Detect the second time: if (lParam != 0)
• The application returns the necessary data. Details below.

Reference site (Japanese) : website 1, website 2, website 3

A single paragraph is sufficient for the length of the string. e.g. SetReconvertStruct() get 50 or 200 characters.

Sample code (Japanese) .

• Code for XP (Probably for IMM32) Even simple code seems to work.
• Sakura Editor's Code that seems to work even now (SetReconvertStruct(), Very long code compared to above) And Caller.

[zufuliu]

The name is correct, official document at https://docs.microsoft.com/en-us/windows/win32/intl/imr-documentfeed

[maboroshin]

Thank you!

This function is commonly called “Before and after feedback”( 前後フィードバック or 前後参照 ) in Japanese. This is probably not a search hit in English. There seems to be no official information on the details of the structure.

• In Gecko, HandleDocumentFeed function, Line 1689, long code. (EN)
• And its .h file. (EN)
• Tera Term's Code, Line 407 (Line 514 is when the internal processing is ANCI). (JP)
• Tera Term's Release note (EN)

[maboroshin]

This function should be turned on/off from Notepad2 (via Scintilla's variable). Beacuse, It is written about the bug of IME in 2009. (JP) (But 10 years ago) crash IME : In case IME's dictionary file is broken with IME2002/2003. (JP)

[zufuliu]

I filled a feature request at https://sourceforge.net/p/scintilla/feature-requests/1310/. It would helpful if there are steps to let IME makes the request.

[maboroshin]

(Added) This test may not work. It does not seem to retrieve the continuation of the text. It retrieves before the text. See below.

You said "IME makes the request." I thought it was about program code. I made a mistake. Perhaps it's a test.

Scintilla requires testability. But it doesn't work. I tested with Sakura Editor and FireFox. Cannot confirm it.

Contents of Sample.txt :

が痛い
が咲く

Type "hana" before each Japanese, then press Space (Appearing conversion candidates). The result is the ideal case. Confirm the distinction of HANA.

• hana 鼻 (nose) : 鼻が痛い (Whole sentence : HANA GA ITAI)
• hana 花 (flower) : 花が咲く (Whole sentence : HANA GA SAKU)

However, there was no distinction, probably because dictionary learning was prioritized.

• Initialization can be considered if your IME is used frequently. Clear IME's learning/history.

I have not tried the clear. I'll try using Macrium Reflect to make the OS recoverable. later.

[maboroshin]

(Added) However this test did not work in 2023.

I found a good test. Some successful with Sakura editor. Refference site(jp): 1 2

Input: ippatu. Sample text:

危機
ファイト

Ideal result:

• 危機一髪
• ファイト一発

1. It means input "ippatu" after "危機".
2. Then push Space key. Candidates come out.

3. Then push Esc key. Cenccel candidates. This is important. Learned unless canceled. It learns even with unusual combinations.

   (Added) This test may not work. It does not seem to retrieve the continuation of the text. It retrieves before the text. See below.

Input: atui. Sample text: 夏にお茶を飲みながら、本を読む。

Ideal: 暑い夏に熱いお茶を飲みながら、厚い本を読む。

The second "atui" does not succeed in my IME learning

[zufuliu]

updated feature request at https://sourceforge.net/p/scintilla/feature-requests/1310/. Thanks for the tests.

[maboroshin]

Thank you.

By the way, IME cancellation. According to https://github.com/zufuliu/notepad2/commit/f75470d5be2dd27c8e05d87adf19fbf7197c87c0, Scintilla will be changed by moving AddWString in the code. Should it be as follows?

if (lParam & GCS_RESULTSTR) {
 AddWString(imc.GetCompositionString(GCS_RESULTSTR), CharacterSource::imeResult);
 initialCompose = true;

If this is adopted in Scintilla, the behavior of IME will be no different from common software. All that remains is DOCUMENTFEED only. Please upstream this AddWString location

[zufuliu]

This is now implemented (see https://sourceforge.net/p/scintilla/feature-requests/1310/), please test latest builds (e.g. from https://github.com/zufuliu/notepad2/actions/runs/4901319426).

[maboroshin]

Nice news! I think it is working. The previously proposed test case did not work. Sorry.

Test Procedure: Notepad2 from 4901319426 above was used. And Notepad3 is scintilla not updated. I used Win10 which I don't use much. And This PC is a clean install once every few months.

1. 夏は暑い NATU HA [ENTER] ATUI [ESC]
2. 本は厚い HON HA [ENTER] ATUI [ESC] (And Typo "AYU HA")
3. お湯は熱い OYU HA [ENTER] ATUI [ESC]

I tried with Notepad2. Text 1 and 3 were correct. Text 2 was an error. This (熱い) is a candidate for Text 3. Thus, this feature (IMR_DOCUMENTFEED) may not be powerful.

Only Text 3 is correct when trying with Notepad3. All are "熱い".

I tried again with Notepad2. Text 1 and 3 were correct. Thus, this feature is weak but working.

[maboroshin]

https://www.youtube.com/watch?v=8-qaEHzaW24 johnsonj says in this video. Works only if text is on the left side : I'm typing a sentence that continues, Instead of injecting it into the previous text. It is the same behavior in other programs. (Chrome, Firefox, EmEditor, Sakura editor)

[maboroshin]

This test is recommended. In 2023 - 2024, this test case works well for Windows 10.

Another test case from a previous text (https://github.com/zufuliu/notepad2/issues/127#issuecomment-577182528) works well : Refference site(jp): 2

0. SAITA 最多 : (My case, Win10 preinstalled Microsoft IME) (Standalone case, Initial values or dictionary learning appear)
1. HANA GA SAITA 花が咲いた (Ideal case)
2. NUNO WO SAITA 布を裂いた (Ideal case)
3. JIKAN WO SAITA 時間を割いた (Ideal case)

Sample text:

花が
布を
時間を

In my tests, 1 and 3 were correct in Notepad2 above. All failed in Notepad3, Returned 0 (最多) in all cases.

(This case can use "SAKU" instead of "SAITA". SAITA is past tense. It is natural Japanese. But I don't know if the candidate is appropriate.)