JSON fields containing unicode escape sequences are potentially rendered incorrectly

[gcmurphy-snyk]

Before Proceeding, please acknowledge:

• [X] I have consulted the Troubleshooting guide.
• [X] I have searched Issues and Discussions in this repo.
• [X] Feature requests should be initiated as Discussions. This is a bug report.

Describe the bug

Any json field loaded from duckdb containing \n or other escape sequences may not be displayed as expected by harlequin. Leading new line characters look as empty values.

To Reproduce Steps to reproduce the behavior.

Given this file example.json:

{                                                                                                                                                                                   
  "example": "normal entry",                                                                                                                                                        
  "hidden": "\n this is hidden",                                                                                                                                                    
  "emoji": "\uD83D\uDCA9"                                                                                                                                                           
}

Run the query:

select * from read_json_auto('example.json')

Expected behavior

Actual behavior

(The text in the hidden field is not visible)

Contributing Are you interested in contributing a fix?

• [ ] Yes
• [ ] Maybe
• [X] No

Additional context Please provide as much as you can, but don't waste your time if it's definitely not relevant or you don't know.

What is the output of harlequin --version?

$ harlequin --version
harlequin, version 1.20.0

Installed Adapters:
  - duckdb, version 1.20.0
  - sqlite, version 1.20.0

What database adapter are you using with Harlequin? (Default is duckdb)

duckdb

What other options are you using when invoking Harlequin? (If you are using a profile, please add relevant items from your profile here).

Can you tell us more about your system?

• Shell: zsh + tmux

• Terminal:

• OS of the shell: Darwin MC22XQ24G7 23.6.0 Darwin Kernel Version 23.6.0: Mon Jul 29 21:14:46 PDT 2024; root:xnu-10063.141.2~1/RELEASE_ARM64_T6031 arm64

• OS of the terminal (if different from the shell): N/A

• Default locale / language for your OS/Terminal/Shell: en-AU

[!TIP] For example, for my system, these are:

• Bash
• Windows Terminal
• Ubuntu 22.04 / WSL2
• Windows 11
• en_US/UTF-8

How did you install Harlequin?

• [X] pipx
• [ ] conda/mamba
• [ ] pip (global install)
• [ ] pip + venv/virtualenv or pipenv
• [ ] Poetry
• [ ] uv/Rye
• [ ] other: _____

[tconbeer]

Thanks for the issue. The data table is truncating everything after the first line for each cell (So "foo\nbar" renders as "foo"). Escaping might be a good idea (and easier than supporting multiple lines per record).