Add UTF-8 Validation operation. Behavior is aligned with python str.decode() method.
Example b'A\xC3\x28B' contains invalid UTF-8 characters:
If --utf8_replace_mode="ignore" then b'A\xC3\x28B' -> A)B in
If --utf8_replace_mode="replace" then b'A\xC3\x28B' -> A�)B (invalid bytes will be replaced by �).
Add UTF-8 Validation operation. Behavior is aligned with python
str.decode()
method.Example
b'A\xC3\x28B'
contains invalid UTF-8 characters: If--utf8_replace_mode="ignore"
thenb'A\xC3\x28B'
-> A)B in If--utf8_replace_mode="replace"
thenb'A\xC3\x28B'
-> A�)B (invalid bytes will be replaced by �).CVS-146122 CVS-147110