Decode -> Encode produces different binary

[shamis]

Input binary

00 00 00 66 00 00 00 04 00 00 00 00 00 00 00 01 00 01 01 00 01 01 30 37 39 31 31 31 31 31 31 00 00 00 00 00 00 00 00 08 00 3C 00 46 00 72 00 61 00 6E 00 7A 00 F6 00 73 00 69 00 73 00 63 00 68 00 2D 00 54 00 65 00 73 00 74 00 3A 00 E2 00 20 00 C2 00 20 00 E1 00 20 00 C1 00 20 00 E0 00 20 00 C0 00 20 00 E7

Decoded Json

{
    "command_id": "submit_sm",
    "command_length": 204,
    "command_status": "ESME_ROK",
    "data_coding": "UCS2 (ISO/IEC-10646)",
    "dest_addr_npi": "ISDN (E163/E164)",
    "dest_addr_ton": "International",
    "destination_addr": "079111111",
    "esm_class": 0,
    "priority_flag": 0,
    "protocol_id": 0,
    "registered_delivery": 0,
    "replace_if_present_flag": 0,
    "schedule_delivery_time": "",
    "sequence_number": 1,
    "service_type": "",
    "short_message": "\u0000F\u0000r\u0000a\u0000n\u0000z\u0000�\u0000s\u0000i\u0000s\u0000c\u0000h\u0000-\u0000T\u0000e\u0000s\u0000t\u0000:\u0000�\u0000 \u0000�\u0000 \u0000�\u0000 \u0000�\u0000 \u0000�\u0000 \u0000�\u0000 \u0000�",
    "sm_default_msg_id": 0,
    "source_addr": "",
    "source_addr_npi": "ISDN (E163/E164)",
    "source_addr_ton": "International",
    "validity_period": ""
}

Encoded Pdu

00 00 00 76 00 00 00 04 00 00 00 00 00 00 00 01 00 01 01 00 01 01 30 37 39 31 31 31 31 31 31 00 00 00 00 00 00 00 00 08 00 4C 00 46 00 72 00 61 00 6E 00 7A 00 EF BF BD 00 73 00 69 00 73 00 63 00 68 00 2D 00 54 00 65 00 73 00 74 00 3A 00 EF BF BD 00 20 00 EF BF BD 00 20 00 EF BF BD 00 20 00 EF BF BD 00 20 00 EF BF BD 00 20 00 EF BF BD 00 20 00 EF BF BD

[shamis]

Encode (JSON to HEX) :

1. If data_coding is missing its assumed to be LATIN1 and short_message field is unprocessed
2. If data_coding is IA5 same as in 1
3. If data_coding is UCS2 do the necessary UTF8 -> UCS2 encoding
4. For any other data coding assume short_message field is encoded in base64 (or \uXXXX) and error on encoding if it isn't

Decode (HEX to JSON) :

1. For LATIN1, IA5 and short_message field will be plain text
2. If data_coding is IA5 deal with it same as in 1
3. If data_coding is UCS2 do the necessary UCS2 -> UTF8 encoding
4. For any other data coding, encode short_message field binary as base64 (or \uXXXX)