search

grambank / pygrambank

Apache License 2.0
4 stars 1 forks source link

New `check_encoding` command. #73

Closed johenglisch closed 2 years ago

johenglisch commented 2 years ago

  1. Looks for the character � to detect potential corruption.

  2. Looks for characters that cannot be decoded in UTF-8.

  3. Shows what the line would look like when decoded from cp1252 and macroman (we can always add more encodings later if we need to).

  4. Warns about mixed encodings.

HedvigS commented 2 years ago

great idea @johenglisch !