Closed n-etupirka closed 8 months ago
@n-etupirka thanks for reporting this. I'll take a look and see what I need to do to resolve the problem.
@n-etupirka Can you share what the output of whoami /priv
looks like on the endpoint? I changed my Windows 11 VM's primary language to Japanese and see this:
[+] 2023-12-05T13:32:47Z Created C:\WINDOWS\system32\cmd.exe process with an ID of 2380
PRIVILEGES INFORMATION
----------------------
Privilege Name Description State
============================= ==================================== ========
SeShutdownPrivilege Shut down the system Disabled
SeChangeNotifyPrivilege Bypass traverse checking Enabled
SeUndockPrivilege Remove computer from docking station Disabled
SeIncreaseWorkingSetPrivilege Increase a process working set Disabled
SeTimeZonePrivilege Change the time zone Disabled
It looks like I'm able to send Japanese characters back and forth without problem
Merlin[agent][fd05e3d3-bbb3-4697-b766-66befed0825b]» ls C:\\Users\\Rick\\Downloads
Merlin[agent][fd05e3d3-bbb3-4697-b766-66befed0825b]»
[-] 2023-12-05T13:38:42Z Created job fcDFUpydUL for agent fd05e3d3-bbb3-4697-b766-66befed0825b at 2023-12-05T13:38:42Z
[-] 2023-12-05T13:38:58Z Results of job fcDFUpydUL for agent fd05e3d3-bbb3-4697-b766-66befed0825b at 2023-12-05T13:38:58Z
[+] 2023-12-05T13:38:58Z Directory listing for: C:\Users\Rick\Downloads
-rw-rw-rw- 2023-12-01 09:23:24 282 desktop.ini
-rw-rw-rw- 2023-12-05 08:35:45 6 テスト.txt
@Ne0nd0g The result of whoami /priv
in the Japanese environment Windows 11 is output like this.
C:\Users\pentester>whoami /priv
PRIVILEGES INFORMATION
----------------------
特権名 説明 状態
============================= =============================================== ====
SeShutdownPrivilege システムのシャットダウン 無効
SeChangeNotifyPrivilege 走査チェックのバイパス 有効
SeUndockPrivilege ドッキング ステーションからコンピューターを削除 無効
SeIncreaseWorkingSetPrivilege プロセス ワーキング セットの増加 無効
SeTimeZonePrivilege タイム ゾーンの変更 無効
This may be accomplished by setting not only the primary language to Japanese, but also the system locale to Japanese.
(Settings -> Time & language -> Language & region -> Administrative language settings -> Change system locale... -> Japanese)
@Ne0nd0g The ls command, which is built into Merlin, did not cause the error. (The second image is Agent's debug log.)
However, when I run dir from a shell command, I get an error.
This is the correct output of the dir command at the endpoint.
C:\Users\pentester>dir C:\Users\pentester\Downloads
ドライブ C のボリューム ラベルがありません。
ボリューム シリアル番号は 1A28-6E18 です
C:\Users\pentester\Downloads のディレクトリ
2023/12/06 15:27 <DIR> .
2023/12/06 14:11 <DIR> ..
2023/12/06 13:52 9 テスト.txt
1 個のファイル 9 バイト
2 個のディレクトリ 107,650,621,440 バイトの空き領域
I just wanted to follow up and say that I'm able to duplicate the problem and I'm working on a fix.
OK, I have a fix now. It's brittle, but it works. I check to make sure the output is UTF-8. If it isn't, I check the active code page. In this case, the code page is 932
for ShifJIS. I'm then able to decode the data to that encoding format. It is brittle because I've only handled decoding for UTF-8 and ShiftJIS. Anything else will just replace all invalid UTF-8 characters with this character: �. If others need their handling included for a different format, I will add it at that time.
The fix for handling ShiftJS is in the https://github.com/Ne0nd0g/merlin-agent repository. I still need to fix the fact that the gRPC stream broke and did not fix itself.
@n-etupirka Are you comfortable building the Merlin Agent from source code using the dev
branch to see if everything is working as expected? https://github.com/Ne0nd0g/merlin-agent/tree/dev
@Ne0nd0g I have confirmed that everything is working as expected in the Japanese environment!
Thank you for providing a great tool and great response.
Incidentally, are there any plans to fix the bug that breaks gRPC streams in languages other than UTF-8 and ShiftJIS?
Yes, I do plan to fix the bug that broke gRPC. There are two different problems. One is that the gRPC stream received non UTF-8 text. On the Agent, I now detect non UTF-8 and convert it to UTF-8. The second is that the gRPC stream didn't recover. I'll work on that next.
I'm going to use this issue to fix the gRPC error too.
Updated merlin-cli to try and recover broken gRPC message stream identified in this issue https://github.com/Ne0nd0g/merlin-cli/commit/e4ef7b93fc754aeb8e0a23eafe62e51f11c18ed9
I tried it in the following version, and confirmed that even in a non UTF-8 environment, no errors occurred and it was replaced with "�".
Merlin[agent][5986e2d7-a3f3-4eb8-9f3d-0a5fbb4d3313]»
[-] 2023-12-15T07:58:13Z Created job uawOwEVDRd for agent 5986e2d7-a3f3-4eb8-9f3d-0a5fbb4d3313 at 2023-12-15T07:58:13Z
[-] 2023-12-15T07:58:23Z Results of job uawOwEVDRd for agent 5986e2d7-a3f3-4eb8-9f3d-0a5fbb4d3313 at 2023-12-15T07:58:23Z
[+] 2023-12-15T07:58:23Z Created C:\Windows\system32\cmd.exe process with an ID of 10716
***The output was not valid UTF-8 and there isn't a configured decoder for code page 936***
�Ȩ�Ϣ
----------------------
�Ȩ� � ״̬
============================= ==================== ======
SeShutdownPrivilege �ر�ϵͳ �ѽ�
SeChangeNotifyPrivilege �ƹ� �
SeUndockPrivilege �չ�ȡ�¼� �ѽ�
SeIncreaseWorkingSetPrivilege �ӽ�̹� �ѽ�
SeTimeZonePrivilege �ʱ� �ѽ�
Merlin[agent][5986e2d7-a3f3-4eb8-9f3d-0a5fbb4d3313]»
Awesome, thank you for trying it out. I added support for simplified and traditional Chinese as well as Korean in this commit if you want to try that out as well https://github.com/Ne0nd0g/merlin-agent/commit/4c60e45c18722b5dde357e68be492636df5ed972
In Japan, multi-byte characters are used. If the output of a command executed by the shell command contains multi-byte characters, an error will result.
Prerequisite
Environment Data
Expected Behavior
The output of the command is displayed on the server.
Actual Behavior
An error is displayed.
Steps to Reproduce Behavior
Misc Information