How to handle non-ascii string?

[darkr4y]

Event Eg:

- <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
- <System>
  <Provider Name="Service Control Manager" Guid="{555908d1-a6d7-4695-8e1e-26931d2012f4}" EventSourceName="Service Control Manager" /> 
  <EventID Qualifiers="16384">7036</EventID> 
  <Version>0</Version> 
  <Level>4</Level> 
  <Task>0</Task> 
  <Opcode>0</Opcode> 
  <Keywords>0x8080000000000000</Keywords> 
  <TimeCreated SystemTime="2018-02-24T13:53:23.976418400Z" /> 
  <EventRecordID>2435844</EventRecordID> 
  <Correlation /> 
  <Execution ProcessID="512" ThreadID="2548" /> 
  <Channel>System</Channel> 
  <Computer>ETYY-WSPX</Computer> 
  <Security /> 
  </System>
- <EventData>
  <Data Name="param1">Windows Error Reporting Service</Data> 
--------> Check here ------->    <Data Name="param2">正在运行</Data>    ------->
  <Binary>5700650072005300760063002F0034000000</Binary> 
  </EventData>
  </Event>

when call .ToJson() it will display like this:

{
    "Event": {
        "EventData": {
            "param1": "storvsc",
            "param2": "\u0015\ufffd/\ufffd",
            "param3": "\t\u0000/\ufffd",
            "param4": "storvsc"
        },
        "System": {
            "Channel": "System",
            "Computer": "37L4247F27-26",
            "Correlation": {},
            "EventID": {
                "Qualifiers": "16384",
                "Value": "7040"
            },
            "EventRecordID": "96",
            "Execution": {
                "ProcessID": "956",
                "ThreadID": "540"
            },
            "Keywords": "0x8080000000000000",
            "Level": "4",
            "Opcode": "0",
            "Provider": {
                "EventSourceName": "Service Control Manager",
                "Guid": "{555908d1-a6d7-4695-8e1e-26931d2012f4}",
                "Name": "Service Control Manager"
            },
            "Security": {
                "UserID": "S-1-5-18"
            },
            "Task": "0",
            "TimeCreated": {
                "SystemTime": "2016-01-25T04:27:33Z"
            },
            "Version": "0"
        }
    }
} 

param3 is bad.

[qjerome]

Hi,

I think I would need your evtx file to understand where the issue is. Do you think you could share it with me?

[darkr4y]

yep of cuz，I'll upload it to transfer.sh the problem is when evtx file contains chinese ( actully I cant confirm what character set is ) , dump it to json format , the chinese will be converted like this \ufffd/\ufffd

the evtx file download url is https://transfer.sh/Tph0T/chinese.evtx

[darkr4y]

BTW golang-evtx is a very useful lib more than libevtx , thx for ur hardwork ;)

[qjerome]

Thanks, I am glad this lib and tools help people. Actually I decided to start the development facing the limitations of libevtx and other evtx parsing libs. I have retrieved your evtx file and I will have a look in the coming days.

[darkr4y]

Hi,This is the result of my investigation today a event record of json format:

{
    "Event": {
        "EventData": {
            "param1": "VMware Log Browser",
            "param2": "70308",
            "param3": "60000",
            "param4": "1",
            "param5": "Ͱ/\ufffd\r\ufffd" //this string should be "重新启动服务"
        },
        "System": {
            "Channel": "System",
            "Computer": "ETYY-WSPX",
            "Correlation": {},
            "EventID": {
                "Qualifiers": "49152",
                "Value": "7031"
            },
            "EventRecordID": "2365472",
            "Execution": {
                "ProcessID": "512",
                "ThreadID": "3092"
            },
            "Keywords": "0x8080000000000000",
            "Level": "2",
            "Opcode": "0",
            "Provider": {
                "EventSourceName": "Service Control Manager",
                "Guid": "{555908d1-a6d7-4695-8e1e-26931d2012f4}",
                "Name": "Service Control Manager"
            },
            "Security": {},
            "Task": "0",
            "TimeCreated": {
                "SystemTime": "2017-07-13T02:47:06Z"
            },
            "Version": "0"
        }
    }
}

I insert some code into line 797 of file "structs.go"

if (i == 4) {
fmt.Printf("===> ValDesc: %v - Val: %T - I: %d \n",tid.ValDescs[i],tid.Values[i],i)
}
fmt.Printf("ValDesc: %v - Val: %v - Err: %v \n",tid.ValDescs[i],tid.Values[i],err)

and console shows that

ValDesc: Size: 36 ValType: 0x01 Unk: 0x00 - Val: VMware Log Browser - Err: <nil> 
ValDesc: Size: 10 ValType: 0x01 Unk: 0x00 - Val: 70309 - Err: <nil> 
ValDesc: Size: 10 ValType: 0x01 Unk: 0x00 - Val: 60000 - Err: <nil> 
ValDesc: Size: 2 ValType: 0x01 Unk: 0x00 - Val: 1 - Err: <nil> 
===> ValDesc: Size: 12 ValType: 0x01 Unk: 0x00 - Val: *evtx.ValueString - I: 4 
� - Err: <nil> 

so I trace the call stack , and I think the problem is probably because of Parse function about ValueString.

the Parse function located at line 348 of the file "values.go", the original character 重新启动服务 which in utf16 should be \u91CD \u65B0 \u542F \u52A8 \u670D \u52A1].

but after executed these two lines

s.value = make(UTF16String, s.Size/2)
return encoding.UnmarshaInitSlice(reader, &s.value, Endianness)

the s.value will be [cd91 b065 2f54 a852 0d67 a152]

[qjerome]

Hi,

I found out the issue. I will push the update tomorrow evening, the time for me to propagate the changes. Thank you very much for your help, it allowed me to save some time.

Cheers,

[qjerome]

Hi, I committed the changes, could you try that it works as you expect. From what I saw in my tests, it looks correct. Cheers,

[darkr4y]

OK I'll try to test it and give u a quick replay soon.

updated： Now it looks like golang-evtx works very well in anticipation

We can close this issue now Thx very much again ;)

[qjerome]

Thanks to you for the time spent debugging and testing :)