Com-AugustCellars / CoAP-CSharp

CoAP Implementation in C#
Other
41 stars 19 forks source link

Recall that Encoding.UTF8 adds byte-order marks #55

Closed peteroupc closed 5 years ago

peteroupc commented 5 years ago

This is to remind you that Encoding.UTF8 encodes text strings by beginning them with byte-order marks. Thus, code like the following may not work as expected:

 CBORObject.FromObject(Encoding.UTF8.GetBytes("TEXT"))

My library contains a DataUtilities.GetUtf8Bytes method that can be used instead:

 CBORObject.FromObject(DataUtilities.GetUtf8Bytes("TEXT", false))
jimsch commented 5 years ago

That is annoying - and I have never actually seen it emit the preamble in practice. The entire idea of emitting a BOM for a UTF8 encoding annoys me to no end. Its not like it is a useful marker.

peteroupc commented 5 years ago

Closing this issue as moot. As explained in the same documentation, Encoding.UTF8's methods do not add a byte-order mark automatically; it merely provides a byte-order mark with the GetPreamble method.