ProTools Session Info Text: TextEdit Character Encoding

[orchetect]

Issue

In Pro Tools during Export → Session Info as Text, if the text encoding is set to TextEdit (which is the default selection), it lossily converts various non-standard/extended characters in text strings (such as Marker name / comments).

Common Examples

Character	Description	Exported	Notes
–	U+2013, En Dash	Ð
—	U+2014, Em Dash	Ñ	Keyboard input character substitution on macOS
’	U+2019, Right Single Quote	Õ	Sometimes when copying from Microsoft Word; should be an apostrophe
…	U+2026, Ellipsis	É	Keyboard input character substitution on macOS

Example 'UTF8' Format

T R A C K  L I S T I N G
M A R K E R S  L I S T I N G
#       LOCATION        TIME REFERENCE      UNITS       NAME                                COMMENTS
1       01:00:00:00     480000              Samples     Test Ellipsis…                      
2       01:01:00:00     3360000             Samples     Test Em Dash —                      
3       01:02:00:00     6240000             Samples     Test En Dash –                      
4       01:03:00:00     9120000             Samples     Right Side Quote’s Not An Apostrophe    

Example 'TextEdit' Format (Pro Tool's default when exporting)

T R A C K  L I S T I N G
M A R K E R S  L I S T I N G
#       LOCATION        TIME REFERENCE      UNITS       NAME                                COMMENTS
1       01:00:00:00     480000              Samples     Test EllipsisÉ                      
2       01:01:00:00     3360000             Samples     Test Em Dash Ñ                      
3       01:02:00:00     6240000             Samples     Test En Dash Ð                      
4       01:03:00:00     9120000             Samples     Right Side QuoteÕs Not An Apostrophe    

Proposed Solutions

• [ ] Add note to documentation suggesting users export using UTF-8 format and not TextEdit format.
• [ ] A heuristic could be employed to convert some of the most common lossy conversions back to their original Unicode characters.

[orchetect]

Text file encoding is now auto-detected and all extended characters should be decoded correctly regardless whether the text file was exported from Pro Tools using TextEdit (Mac OS Roman) or UTF8 format.

Basic unit tests are implemented but further refining may be necessary in as-yet unforeseen edge cases.