New-HTMLTable - not displaying cells correctly if they contain "\r" or "\n". Ex: "DOMAIN\username"

[matt555]

https://github.com/EvotecIT/PSWriteHTML/blob/31212c8cdf8cd02e51613b839dad45d661679de7/Public/New-HTMLTable.ps1#L965

Issue

I have a table containing DOMAIN\username data.

If that username starts with an "r" or "n", the backslash and first letter are replaced by a space.

I've tested this with multiple versions of PSWriteHTML, including the latest.

Example code

• each column tests a different JS escape sequence
• Row 1 - this is the raw data, simply domain\username
• Row 2 - attempt to manually insert a backtick
• Row 3 - same as 2 but with double quotes instead
• Row 4 - Current Workaround - use double quotes and only insert a backtick on \r and \n

$DataTable1 = [System.Collections.Generic.List[object]]@()
$DataTable1.AddRange(@(
        [PSCustomObject] @{ baseline = 'FOO\mark'; userWithN = 'FOO\nancy'; userWithR = 'FOO\robert'; userWithB = 'FOO\billy'; userWithF = 'FOO\frank'; userWithT = 'FOO\tommy'; userWithV = 'FOO\vinny'; test = 'source data - single backslash' }
        [PSCustomObject] @{ baseline = 'FOO\`mark'; userWithN = 'FOO\`nancy'; userWithR = 'FOO\`robert'; userWithB = 'FOO\`billy'; userWithF = 'FOO\`frank'; userWithT = 'FOO\`tommy'; userWithV = 'FOO\`vinny'; test = 'single quote - add hyphen after backslash' }
        [PSCustomObject] @{ baseline = "FOO\`mark"; userWithN = "FOO\`nancy"; userWithR = "FOO\`robert"; userWithB = "FOO\`billy"; userWithF = "FOO\`frank"; userWithT = "FOO\`tommy"; userWithV = "FOO\`vinny"; test = "double quote - add hyphen after backslash" }
        [PSCustomObject] @{ baseline = "FOO\mark"; userWithN = "FOO\`nancy"; userWithR = "FOO\`robert"; userWithB = "FOO\billy"; userWithF = "FOO\frank"; userWithT = "FOO\tommy"; userWithV = "FOO\vinny"; test = "<span style='color:green;'>double quote - add hyphen after backslash ONLY for R and N</span>" }
))

New-HTML -TitleText "NewLineHell" -Online -ShowHTML {
    New-HTMLMain {
        New-HTMLSection -HeaderText "New-HTMLTable: -DataStore Javascript" -HeaderTextSize '30px' {
            New-HTMLPanel {
                New-HTMLTableOption -DataStore Javascript
                New-HTMLTableStyle -Type Table -FontSize '20px'
                New-HTMLTable -DataTable $DataTable1 -DisablePaging -DisableStateSave -HideButtons -HideFooter -DisableSearch -DisableInfo -HTML {}
            }
        }
    }
}

Fix?

commenting out this line works and also avoids my PS object appearing like this when i need to output to host and export to cli.

https://github.com/EvotecIT/PSWriteHTML/blob/31212c8cdf8cd02e51613b839dad45d661679de7/Public/New-HTMLTable.ps1#L965

I have no idea why its there, but if it is desired, perhaps another option is to add a switch to skip that specific replacement...? I did a before and after dump of the $Options variable and before that line it's perfectly suitable JS.

edit: added screenshot of issue

[matt555]

forgot to add example of what the workaround does to cli output:

[PrzemyslawKlys]

The issue is with change I did switch from:

• https://github.com/EvotecIT/PSSharedGoods/blob/master/Public/Converts/ConvertTo-JsonLiteral.ps1

To:

• https://github.com/EvotecIT/PSSharedGoods/blob/master/Public/Converts/ConvertTo-PrettyObject.ps1

I guess one need to find differences and fix it there...

The change we did was switch from my own ConverTo-JsonString which was a bit slower and had some issues with AMSI to native ConverTo-json, but then it has it's own problems with conversion of chars.

[matt555]

The issue is with change I did switch from:

• https://github.com/EvotecIT/PSSharedGoods/blob/master/Public/Converts/ConvertTo-JsonLiteral.ps1

To:

• https://github.com/EvotecIT/PSSharedGoods/blob/master/Public/Converts/ConvertTo-PrettyObject.ps1

I guess one need to find differences and fix it there...

The change we did was switch from my own ConverTo-JsonString which was a bit slower and had some issues with AMSI to native ConverTo-json, but then it has it's own problems with conversion of chars.

@PrzemyslawKlys I spent quite a bit of time ruling out ConvertTo-PrettyObject. I'm quite positive my issue is due to this: https://github.com/EvotecIT/PSWriteHTML/blob/31212c8cdf8cd02e51613b839dad45d661679de7/Public/New-HTMLTable.ps1#L965

If I comment that line out, it works perfectly.

I tried to figure out why you might have needed that line but i can up short :(

[PrzemyslawKlys]

I added it in last commit:

• https://github.com/EvotecIT/PSWriteHTML/commit/c4c38228ba6a6954a7a61ceb9184978078aea2d1

So I guess I had to have a reason. \r in javascript is return and \n in javascript is new line, so I'm clearly trying to fix new lines, just not sure why you are using it that way ?

[matt555]

I added it in last commit:

• c4c3822

So I guess I had to have a reason. \r in javascript is return and \n in javascript is new line, so I'm clearly trying to fix new lines, just not sure why you are using it that way ?

Sorry if that wasn't clear. I simply have a report with domain\username in a column. The first row in my example is the outcome without modifying the strings. The following rows are just various ways i tried to fix it. The final row is the workaround i have to implement; which looks like this:

$_.name.Replace('\n', "\`n").replace('\r', "\`r")

My first thought was to just wrap them in <pre> tags but that replace command doesn't care about pre tags and the end result is has the first letter of their name missing.

In this scenario, the report is related to compliance auditing. So if the source of my data reports domain\username i need to show exactly that. If it all possible, not modifying the source strings is highly desired.

The thing is, without that replace line in new-htmltable, everything seems fine. I'm not sure what we're trying to fix or prevent with it. Just to clarify, it only affects \r and \n. I added the other javascript escape strings just to test various workarounds.

What's odd is i ran this on an older release and it still happened. In fact this is my requires line for my prod script:

#requires -modules @{ ModuleName="PSWriteHTML"; ModuleVersion="1.14.0"; MaximumVersion="1.15.0" }

...i'm still digging :wink:

[PrzemyslawKlys]

nah this has to be fixed on json side without pre tags. Maybe something like this would work? I guess we have to figure out the use of new line vs domain\nato - I'll try to test it too

$Options = $Options -replace '\\\\r', '\\r' -replace "\\\\n", "\\n" 

[matt555]

nah this has to be fixed on json side without pre tags. Maybe something like this would work? I guess we have to figure out the use of new line vs domain\nato - I'll try to test it too

$Options = $Options -replace '\\\\r', '\\r' -replace "\\\\n", "\\n" 

I suspect that ConverTo-Json from line 939 and 958 are already covering this.

Try this test. Edit your new-htmltable.ps1 to dump the options before and after:

        $options | Out-File  "C:\temp\options_before.json"
        $Options = $Options -replace '\\\\r', '\r' -replace "\\\\n", "\n"
        $options | Out-File  "C:\temp\options_after.json"

A compare of the before (left) and after:

Also here's a bit simpler test with some extra columns for a few different line endings. For those tests rows1 and 2 should have the same result.

$String1 = "this line `n should have a \n line \\n break"
$String2 = "this line `r should have a \r carriage \\r return"
$String3 = "this line `r`n should have a \r\n line break \\r\\n and carriage return"

$DataTable1 = [System.Collections.Generic.List[object]]@()
$DataTable1.AddRange(@(
        [PSCustomObject] @{ baseline = 'FOO\mark';     userWithN = 'FOO\nancy';     userWithR = 'FOO\robert';     userWithB = 'FOO\billy';     userWithF = 'FOO\frank';     userWithT = 'FOO\tommy';     userWithV = 'FOO\vinny'; string1 = $String1; string2 = $String2; String3 = $String3; test = "unmodified source data" }
        [PSCustomObject] @{ baseline = "FOO\mark";     userWithN = "FOO\`nancy";    userWithR = "FOO\`robert";    userWithB = "FOO\billy";     userWithF = "FOO\frank";     userWithT = "FOO\tommy";     userWithV = "FOO\vinny"; string1 = $String1; string2 = $String2; String3 = $String3; test = "<span style='color:green;'>workaround</span>" }
))

New-HTML -TitleText "NewLineHell" -Online -ShowHTML {
    New-HTMLMain {
        New-HTMLSection -HeaderText "New-HTMLTable: -DataStore Javascript" -HeaderTextSize '30px' {
            New-HTMLPanel {
                New-HTMLTableOption -DataStore Javascript
                New-HTMLTableStyle -Type Table -FontSize '20px'
                New-HTMLTable -DataTable $DataTable1 -DisablePaging -DisableStateSave -HideButtons -HideFooter -DisableSearch -DisableInfo -HTML {}
            }
        }
        New-HTMLSection -Invisible {
            New-HTMLSection -HeaderText "pwsh cli output: Format-Table -Property * -AutoSize -Wrap " -HeaderTextSize '30px' {
                New-HTMLPanel {
                    New-HTMLCodeBlock -Style powershell -Code $($DataTable1 | Format-Table -Property * -AutoSize -Wrap | Out-String)
                }
            }
            New-HTMLSection -HeaderText "strings in code block" -HeaderTextSize '30px' {
                New-HTMLPanel {
                    New-HTMLCodeBlock -Style powershell -Code {
                        $String1 = "this line `n should have a \n line \\n break"
                        $String2 = "this line `r should have a \r carriage \\r return"
                        $String3 = "this line `r`n should have a \r\n line break \\r\\n and carriage return"
                    }
                }
            }
        }
    }
}

As you can see the domain\username data is already correct before (see below test for the strings columns). For those i'm not sure if before is acceptable.

[PrzemyslawKlys]

What about objects with new lines? are those working after removing the options conversion? are there no errors in HTML when you open it up in Edge? I need to test this change with GPOZaurr, especially Invoke-GPozaurr -Type GPOAnalysis as it has lots of weird chars

[PrzemyslawKlys]

$DataTable3 = @(
    [PSCustomObject] @{
        'Test1'        = 'Test' + [System.Environment]::NewLine + 'test3';
        'Test2'        = 'Test' + [System.Environment]::NewLine + 'test3' + "`n test"
        'Test3'        = 'Test' + [System.Environment]::NewLine + 'test3' + "`r`n test"
        'Test4'        = 'Test' + [System.Environment]::NewLine + 'test3' + "`r test"
        'Test5'        = 'Test' + "`r`n" + 'test3' + "test"
        'Test6'        = @"
        Test1
        Test2
        Test3

        Test4
"@
        'Test7'        = 'Test' + "`n`n" + "Oops"
        'Test9'        = 'Test' + "<Br>" + "Oops"
        'Test8"Oopps"' = 'MyTest "Ofcourse"'
        "Test9'Ooops'" = "MyTest 'Ofcourse'"

    }
)
$DataTable3 | Out-HtmlView -DataStore JavaScript -FilePath $PSScriptRoot\Example7_02_DataStoreJava.html -Online

Now it works for both, but what I wonder - I believe with old version it used to work correctly with new lines and it doesn't seem to show it now. Only
works.

[matt555]

$DataTable3 = @(
    [PSCustomObject] @{
        'Test1'        = 'Test' + [System.Environment]::NewLine + 'test3';
        'Test2'        = 'Test' + [System.Environment]::NewLine + 'test3' + "`n test"
        'Test3'        = 'Test' + [System.Environment]::NewLine + 'test3' + "`r`n test"
        'Test4'        = 'Test' + [System.Environment]::NewLine + 'test3' + "`r test"
        'Test5'        = 'Test' + "`r`n" + 'test3' + "test"
        'Test6'        = @"
        Test1
        Test2
        Test3

        Test4
"@
        'Test7'        = 'Test' + "`n`n" + "Oops"
        'Test9'        = 'Test' + "<Br>" + "Oops"
        'Test8"Oopps"' = 'MyTest "Ofcourse"'
        "Test9'Ooops'" = "MyTest 'Ofcourse'"

    }
)
$DataTable3 | Out-HtmlView -DataStore JavaScript -FilePath $PSScriptRoot\Example7_02_DataStoreJava.html -Online

Now it works for both, but what I wonder - I believe with old version it used to work correctly with new lines and it doesn't seem to show it now. Only works.

I forgot to mention to ignore my comment about the issue existing in the "old" version. I thought the change was after 1.14.0.

So, i did a thing. I added a switch to new-htmltable: -SkipUnixLineEndReplace

        # add SkipUnixLineEndReplace to params as well
        if ($SkipUnixLineEndReplace -eq $false) {
            $Options = $Options -replace '\\\\r', '\r' -replace "\\\\n", "\n"
        }

Then ran it with you example, plus the the key usernames:

$DataTable3 = @(
    [PSCustomObject] @{
        'Test1'        = 'Test' + [System.Environment]::NewLine + 'test3';
        'Test2'        = 'Test' + [System.Environment]::NewLine + 'test3' + "`n test"
        'Test3'        = 'Test' + [System.Environment]::NewLine + 'test3' + "`r`n test"
        'Test4'        = 'Test' + [System.Environment]::NewLine + 'test3' + "`r test"
        'Test5'        = 'Test' + "`r`n" + 'test3' + "test"
        'Test6'        = @"
        Test1
        Test2
        Test3

        Test4
"@
        'Test7'        = 'Test' + "`n`n" + "Oops"
        'Test9'        = 'Test' + "<Br>" + "Oops"
        'Test8"Oopps"' = 'MyTest "Ofcourse"'
        "Test9'Ooops'" = "MyTest 'Ofcourse'"
        'userN'        = 'FOO\nancy'
        'userR'        = 'FOO\robert'
        'userBaseline' = 'FOO\mark'
    }
)

New-HTML -TitleText "NewLineHell" -Online -ShowHTML {
    New-HTMLMain {
        New-HTMLPanel -Invisible {
            New-HTMLSection -HeaderText "New-HTMLTable: default" -HeaderTextSize '30px' {
                New-HTMLPanel {
                    New-HTMLTableOption -DataStore Javascript
                    New-HTMLTableStyle -Type Table -FontSize '20px'
                    New-HTMLTable -DataTable $DataTable3 -DisablePaging -DisableStateSave -HideButtons -HideFooter -DisableSearch -DisableInfo -Transpose -HTML {}
                }
            }
        }
        New-HTMLPanel -Invisible {
            New-HTMLSection -HeaderText "New-HTMLTable: -SkipUnixLineEndReplace" -HeaderTextSize '30px' {
                New-HTMLPanel {
                    New-HTMLTableOption -DataStore Javascript
                    New-HTMLTableStyle -Type Table -FontSize '20px'
                    New-HTMLTable -DataTable $DataTable3 -DisablePaging -DisableStateSave -HideButtons -HideFooter -DisableSearch -DisableInfo -Transpose -SkipUnixLineEndReplace -HTML {}
                }
            }
        }
    }
}

And test result:

Soo... i think we have a winner right? Meaning either comment it out or add a switch like i did in the test.

[PrzemyslawKlys]

I will remove it, as soon as I do more testing and potentially understand how it was working before the changes. Something is not right and I don't know why I did what I did yet, without more testing. I'll play it during weekend.

[PrzemyslawKlys]

Ok, i've found 1 bug for new lines, and why those were not working correctly.

[PrzemyslawKlys]

I guess new version should fix your issue. Let me know if it doesn't.