Format-Hex Updates

[vexx32]

Issue Details

See https://github.com/PowerShell/PowerShell/pull/8674 for full details.

Summary

1. In PS 6.2 and below, Format-Hex doesn't handle piped input as a single input (even if it's a single data type), and renders a completely separate block of output for every item input of the pipeline, regardless of the type of input.
2. In PS 6.1 and below, Format-Hex only handles byte and string input. (Maybe others? The same as it was in v5, I don't recall the full amount). This was changed in 6.2 (I think; @iSazonov authored the PR that expanded the functionality) and it is now able to handle a larger variety of data types. (Essentially anything it can convert to byte data; strings, [char], [int], structs, etc., etc.)

With https://github.com/PowerShell/PowerShell/pull/8674 merged, Format-Hex:

• Handles continuous data stream input as a single input stream, and will display it in the same manner for both Format-Hex -InputObject $byteArray and $byteArray | Format-Hex
  • This includes arrays / streams of all basic value types.
• Handles jagged streams of input by clumping input objects according to their type. This is not expected to handle a jagged array passed directly to the -InputObject parameter by name.
• Adds ByteCollection properties:
  • Label — stores the type name and a random hexadecimal ID. Used to group objects in output formatting. This can be utilised by, for example, Group-Object, in order to enable downstream cmdlets to handle each group of ByteCollection objects separately, if needed.
  • Ascii — stores the ASCII representation of the bytes in the ByteCollection
  • HexBytes — stores the hexadecimal representation of the bytes in the ByteCollection
• Replaces Unicode control characters in the Ascii results with the Unicode <?> glyph 0xFFFD instead, and replaces the NULL character 0x0 with a space.

Behavioural Details

Piped Input

A Primitive Value

1. Each item is converted to bytes and collected into an input buffer.
2. The type of the last InputObject is stored in a private field.
3. If a new InputObject is received of the same type, it is converted to bytes and added to the buffer.
4. If a new InputObject is received of a different type, the buffer is divided into 16-byte segments and each segment is sent to the success output stream as a ByteCollection. Then, the process restarts from the beginning.

System.IO.FileInfo

1. If any data is in the primitives buffer, it is output as described above, the buffer is cleared, as well as the information on what the last type was (resetting the grouping logic).
2. One ByteCollection object is emitted per 16 bytes of the file.
3. All ByteCollection objects are tagged with the input file's path as their Label, which will group their output into a single table.
4. Offset and Count are interpreted as applying to each file, the same way as they would be for -Path inputs.

String

1. If any data is in the primitives buffer, it is output as described above, the buffer is cleared, as well as the information on what the last type was (resetting the grouping logic).
2. All strings are handled on an individual basis (no grouping), and are converted into bytes according to the -Encoding parameter's value.
3. One ByteCollection object is emitted per 16 bytes of the string, and each is tagged with the System.String type name and a shared random hexadecimal ID to ensure that each string is assigned a separate Label group.
4. Offset and Count are interpreted as applying to each input string.

Array of Primitives

1. If any data is in the primitives buffer, it is output as described above, the buffer is cleared, as well as the information on what the last type was (resetting the grouping logic).
2. Each array is treated as a single "group", and will not be grouped with other items in the output.
3. All items in the array are converted into a single chunk of bytes, which then is sliced according to Offset and Count.
4. One ByteCollection object is emitted per 16 bytes, and each is tagged with the array's type and a shared random hexadecimal ID to ensure that each array is assigned a single Label group.

Examples

Output

If you plan on processing output from Format-Hex programmatically, note that the output mode is line-by-line; each ByteCollection created by the cmdlet only holds 16 bytes before creating a new object.

If you wish to handle specific segments of output when working with mixed input (as shown in the examples), you can use their .Label property to determine which ByteCollection objects belong together. Group-Object is one way to separate out the groupings into the groups displayed by the formatter, in order to be able to handle them separately.

For example, to collate all byte values for all input collections while retaining the grouping according to input object, you could do something like this:

$input |
    Format-Hex |
    Group-Object -Property Label |
    Select-Object -Property Label, @{ Name = "Bytes"; Expression = { $_.Group.Bytes } }

Version(s) of document impacted

• [X] Impacts 7 document
• [ ] Impacts 6 document
• [ ] Impacts 5.1 document
• [ ] Impacts 5.0 document
• [ ] Impacts 4.0 document
• [ ] Impacts 3.0 document

Reason(s) for not selecting all version of documents

• [X] The documented feature was introduced in selected version of PowerShell
• [ ] This issue only shows up in selected version of the document

[vexx32]

@sdwheeler updated the description with fleshed-out details from the PR. If you guys have any questions, let me know! 💖