Fix the ANSI regex to also match for `ESC[K` and make `/render` able to consume real chunks from AI output

[daxian-dbw]

PR Summary

Fix the ANSI regex to also match for ESC[K and make /render able to consume real chunks from AI output.

1. The ANSI regex doesn't match ESC[K which is heavily used in the VT markdown rendering. This causes GetPlainText(string) to return text that contains ESC[K in it, which caused problems to the column/row calculations while re-writing during the streaming rendering. When the miscalculation accumulates to some extent, the streaming render may re-write a line with the wrong coordinates in terminal which overwrites its previous line. This is a tricky bug to hunt down, for which I collected the real output chunks from AI endpoint to reproduce the issue.

2. To help reproduce the rendering issue, I updated the /render command to be able to consume an array of string chunks from a JSON file.

Before the fix

GetPlainText(string) return text that contains the ESC[K sequence:

  Sure! Here's the PowerShell script that combines all the steps and includes comments:

    # Step 1: Create a new resource group
    New-AzResourceGroup -Name "MyResourceGroup" -Location "East US"
    
    # Step 2: Create a new virtual network
    $frontendSubnet = New-AzVirtualNetworkSubnetConfig -Name frontendSubnet -AddressPrefix "10.0.1.0/24"
    $backendSubnet  = New-AzVirtualNetworkSubnetConfig -Name backendSubnet  -AddressPrefix "10.0.2.0/24"
    New-AzVirtualNetwork -Name MyVirtualNetwork -ResourceGroupName MyResourceGroup -Location "East US" -AddressPrefix "10.0.0.0/16" -Subnet $frontendSubnet,$backendSubnet
    
    # Step 3: Create a new subnet within the virtual network
    $subnetConfig = New-AzVirtualNetworkSubnetConfig -Name MySubnet -AddressPrefix "10.0.0.0/24"
    New-AzVirtualNetwork -Name MyVirtualNetwork -ResourceGroupName MyResourceGroup -Location "East US" -AddressPrefix "10.0.0.0/16" -Subnet $subnetConfig
    
    # Step 4: Create a new public IP address
    $publicIp = New-AzPublicIpAddress -Name "MyPublicIp" -ResourceGroupName "MyResourceGroup" -Location "East US"
    
    # Step 5: Create a new network security group
    New-AzNetworkSecurityGroup -Name "nsg1" -ResourceGroupName "rg1" -Location "West US"
    
    # Step 6: Create a new network interface with the public IP address and subnet

That causes the comment for Step 7 to be overwritten by the script command following that comment:

After the fix

GetPlainText(string) return plain text as expected:

  Sure! Here's the PowerShell script that combines all the steps and includes comments:

    # Step 1: Create a new resource group
    New-AzResourceGroup -Name "MyResourceGroup" -Location "East US"

    # Step 2: Create a new virtual network
    $frontendSubnet = New-AzVirtualNetworkSubnetConfig -Name frontendSubnet -AddressPrefix "10.0.1.0/24"
    $backendSubnet  = New-AzVirtualNetworkSubnetConfig -Name backendSubnet  -AddressPrefix "10.0.2.0/24"
    New-AzVirtualNetwork -Name MyVirtualNetwork -ResourceGroupName MyResourceGroup -Location "East US" -AddressPrefix "10.0.0.0/16" -Subnet $frontendSubnet,$backendSubnet

    # Step 3: Create a new subnet within the virtual network
    $subnetConfig = New-AzVirtualNetworkSubnetConfig -Name MySubnet -AddressPrefix "10.0.0.0/24"
    New-AzVirtualNetwork -Name MyVirtualNetwork -ResourceGroupName MyResourceGroup -Location "East US" -AddressPrefix "10.0.0.0/16" -Subnet $subnetConfig

    # Step 4: Create a new public IP address
    $publicIp = New-AzPublicIpAddress -Name "MyPublicIp" -ResourceGroupName "MyResourceGroup" -Location "East US"

    # Step 5: Create a new network security group
    New-AzNetworkSecurityGroup -Name "nsg1" -ResourceGroupName "rg1" -Location "West US"

    # Step 6: Create a new network interface with the public IP address and subnet

And the Step 7 comment gets preserved as expected