search

thrift-iterator / go

decode/encode thrift message without IDL
Apache License 2.0
266 stars 31 forks source link

panic: unsupported type when decoding data (Need Sets support) #22

Open Numenorean opened 3 years ago

Numenorean commented 3 years ago

Here is data which i'm trying to decode:

Data in hex 800100020000001961757468656e7469636174654c6f6e6753657373696f6e5632000000000c00000a00010000017882717cab0b00020000006c533d733430383a553d643664396431393a453d31633165663564333461623a433d31373838323731376433383a503d3164643a413d656e2d696f732d78617574682d6e65773a563d323a483d64343966303064303839613531303434396364383538326138346133333536320a0003000001c1ef5d34ab0c00040800010d6d9d190b00020000000a7261736f726f383936340b0003000000167261736f726f3839363440697261686164612e636f6d080007000000010a000900000178826ce1880a000a00000178826ce18802000d010b000e00000004733430380c000f020004010b0006000000127261736f726f383936342e333134316163340b00140000000272750a002400000178826ce188000c00100a00010000000003c000000a0002000001792193b9800a00030000000003c00000080004000000000a000e00000178826ce1880800130000000008001700000000000c00110a00010000017882717d5602000200020003000200050002000600020007000200080002000b01000b00130000003868747470733a2f2f7777772e657665726e6f74652e636f6d2f73686172642f733430382f757365722f3232353238373434392f70686f746f080015000000010c0016080001000000320a000200000000019000000a00030000000001900000080004000000640a00050000000003c00000080006000186a0080007000000fa080008000186a00800090000006408000a0000006408000b000003e808000c0000000208000d0000000208000e00000000000c00170a00010000017882717d57020002000200030002000500020006000e0007080000000200000002000000030a00090000017fda22a95702000a00000c001802000101020002000a000300000178826ce1880002001901000b00060000002d68747470733a2f2f7777772e657665726e6f74652e636f6d2f73686172642f733430382f6e6f746573746f72650b00070000002468747470733a2f2f7777772e657665726e6f74652e636f6d2f73686172642f733430382f0c000a0b00010000002d68747470733a2f2f7777772e657665726e6f74652e636f6d2f73686172642f733430382f6e6f746573746f72650b00020000002468747470733a2f2f7777772e657665726e6f74652e636f6d2f73686172642f733430382f0b00030000002d68747470733a2f2f7777772e657665726e6f74652e636f6d2f73686172642f733430382f6564616d2f757365720b00040000002b68747470733a2f2f7777772e657665726e6f74652e636f6d2f73686172642f733430382f7574696c6974790b00050000003068747470733a2f2f7777772e657665726e6f74652e636f6d2f73686172642f733430382f6d65737361676573746f72650b0006000000237773733a2f2f77732e657665726e6f74652e636f6d2f73686172642f733430382f69640b00070000003768747470733a2f2f7777772e657665726e6f74652e636f6d2f73686172642f733430382f636f6d6d756e69636174696f6e656e67696e650b00080000004068747470733a2f2f64617368626f6172642e7376632e7777772e657665726e6f74652e636f6d2f76312f6d61696e2e68746d6c233f763d64617368626f6172640b00090000004668747470733a2f2f64617368626f6172642e7376632e7777772e657665726e6f74652e636f6d2f76312f6d61696e2e68746d6c233f763d73706163652d6469726563746f7279000000

It's okay with another server responses, although, burp suite extension https://github.com/mdsecresearch/ThriftDecoder decodes it perfectly

Decoded data ```yaml { "method": "authenticateLongSessionV2", "length": 1321, "type": "reply", "args": { "fields": [ { "field_id": 0, "field_type": "struct", "value": { "fields": [ { "field_id": 1, "field_type": "i64", "value": 1617096413224 }, { "field_id": 2, "field_type": "string", "value": "S=s408:U=d6d9d19:E=1c1ef60c828:C=17882750f48:P=1dd:A=en-ios-xauth-new:V=2:H=b5946af0e630ce30df9e5c5c38961a07" }, { "field_id": 3, "field_type": "i64", "value": 1932456413224 }, { "field_id": 4, "field_type": "struct", "value": { "fields": [ { "field_id": 1, "field_type": "i32", "value": 225287449 }, { "field_id": 2, "field_type": "string", "value": "rasoro8964" }, { "field_id": 3, "field_type": "string", "value": "rasoro8964@irahada.com" }, { "field_id": 7, "field_type": "i32", "value": 1 }, { "field_id": 9, "field_type": "i64", "value": 1617095877000 }, { "field_id": 10, "field_type": "i64", "value": 1617095877000 }, { "field_id": 13, "field_type": "bool", "value": true }, { "field_id": 14, "field_type": "string", "value": "s408" }, { "field_id": 15, "field_type": "struct", "value": { "fields": [ { "field_id": 4, "field_type": "bool", "value": true }, { "field_id": 6, "field_type": "string", "value": "rasoro8964.3141ac4" }, { "field_id": 20, "field_type": "string", "value": "ru" }, { "field_id": 36, "field_type": "i64", "value": 1617095877000 } ] } }, { "field_id": 16, "field_type": "struct", "value": { "fields": [ { "field_id": 1, "field_type": "i64", "value": 62914560 }, { "field_id": 2, "field_type": "i64", "value": 1619766000000 }, { "field_id": 3, "field_type": "i64", "value": 62914560 }, { "field_id": 4, "field_type": "i32", "value": 0 }, { "field_id": 14, "field_type": "i64", "value": 1617095877000 }, { "field_id": 19, "field_type": "i32", "value": 0 }, { "field_id": 23, "field_type": "i32", "value": 0 } ] } }, { "field_id": 17, "field_type": "struct", "value": { "fields": [ { "field_id": 1, "field_type": "i64", "value": 1617096413401 }, { "field_id": 2, "field_type": "bool", "value": false }, { "field_id": 3, "field_type": "bool", "value": false }, { "field_id": 5, "field_type": "bool", "value": false }, { "field_id": 6, "field_type": "bool", "value": false }, { "field_id": 7, "field_type": "bool", "value": false }, { "field_id": 8, "field_type": "bool", "value": false }, { "field_id": 11, "field_type": "bool", "value": true } ] } }, { "field_id": 19, "field_type": "string", "value": "https://www.evernote.com/shard/s408/user/225287449/photo" }, { "field_id": 21, "field_type": "i32", "value": 1 }, { "field_id": 22, "field_type": "struct", "value": { "fields": [ { "field_id": 1, "field_type": "i32", "value": 50 }, { "field_id": 2, "field_type": "i64", "value": 26214400 }, { "field_id": 3, "field_type": "i64", "value": 26214400 }, { "field_id": 4, "field_type": "i32", "value": 100 }, { "field_id": 5, "field_type": "i64", "value": 62914560 }, { "field_id": 6, "field_type": "i32", "value": 100000 }, { "field_id": 7, "field_type": "i32", "value": 250 }, { "field_id": 8, "field_type": "i32", "value": 100000 }, { "field_id": 9, "field_type": "i32", "value": 100 }, { "field_id": 10, "field_type": "i32", "value": 100 }, { "field_id": 11, "field_type": "i32", "value": 1000 }, { "field_id": 12, "field_type": "i32", "value": 2 }, { "field_id": 13, "field_type": "i32", "value": 2 }, { "field_id": 14, "field_type": "i32", "value": 0 } ] } }, { "field_id": 23, "field_type": "struct", "value": { "fields": [ { "field_id": 1, "field_type": "i64", "value": 1617096413401 }, { "field_id": 2, "field_type": "bool", "value": false }, { "field_id": 3, "field_type": "bool", "value": false }, { "field_id": 5, "field_type": "bool", "value": false }, { "field_id": 6, "field_type": "bool", "value": false }, { "field_id": 7, "field_type": "set", "value": { "values": [ 2, 3 ], "etype": "i32" } }, { "field_id": 9, "field_type": "i64", "value": 1648632413401 }, { "field_id": 10, "field_type": "bool", "value": false } ] } }, { "field_id": 24, "field_type": "struct", "value": { "fields": [ { "field_id": 1, "field_type": "bool", "value": true }, { "field_id": 2, "field_type": "bool", "value": false }, { "field_id": 3, "field_type": "i64", "value": 1617095877000 } ] } }, { "field_id": 25, "field_type": "bool", "value": true } ] } }, { "field_id": 6, "field_type": "string", "value": "https://www.evernote.com/shard/s408/notestore" }, { "field_id": 7, "field_type": "string", "value": "https://www.evernote.com/shard/s408/" }, { "field_id": 10, "field_type": "struct", "value": { "fields": [ { "field_id": 1, "field_type": "string", "value": "https://www.evernote.com/shard/s408/notestore" }, { "field_id": 2, "field_type": "string", "value": "https://www.evernote.com/shard/s408/" }, { "field_id": 3, "field_type": "string", "value": "https://www.evernote.com/shard/s408/edam/user" }, { "field_id": 4, "field_type": "string", "value": "https://www.evernote.com/shard/s408/utility" }, { "field_id": 5, "field_type": "string", "value": "https://www.evernote.com/shard/s408/messagestore" }, { "field_id": 6, "field_type": "string", "value": "wss://ws.evernote.com/shard/s408/id" }, { "field_id": 7, "field_type": "string", "value": "https://www.evernote.com/shard/s408/communicationengine" }, { "field_id": 8, "field_type": "string", "value": "https://dashboard.svc.www.evernote.com/v1/main.html#?v=dashboard" }, { "field_id": 9, "field_type": "string", "value": "https://dashboard.svc.www.evernote.com/v1/main.html#?v=space-directory" } ] } } ] } } ] }, "proto": "binary", "header": null, "seqid": 0 } ```
Numenorean commented 3 years ago

I have found this type(set), which is probably not supported currently. It looks like

{
    "field_id": 7,
    "field_type": "set",
    "value": {
        "values": [
            2,
            3
        ],
        "etype": "i32"
    }
}
Numenorean commented 3 years ago

Found bad, but working solution if you only need to unmarshal data. Manually edit this file: /general/decode.go end edit function generalReaderOf in the end, adding another one case:

case protocol.TypeSet:
    return readList

Now you can decode your data, but it's pretty bad solution, that must be improved