Auto-detect .ini files encoding

[shelbyKiraM]

Prerequisites

• [X] Put an X between the brackets on this line if you have done all of the following:
  • Reproduced the problem in Safe Mode: https://flight-manual.atom.io/hacking-atom/sections/debugging/#using-safe-mode
  • Followed all applicable steps in the debugging guide: https://flight-manual.atom.io/hacking-atom/sections/debugging/
  • Checked the FAQs on the message board for common solutions: https://discuss.atom.io/c/faq
  • Checked that your issue isn't already filed: https://github.com/issues?utf8=✓&q=is%3Aissue+user%3Aatom
  • Checked that there is not already an Atom package that provides the described functionality: https://atom.io/packages

Description

Atom saves UTF16 .ini files back as with a UTF8 header?

https://github.com/RamonUnch/AltSnap/issues/97#issuecomment-922228480

AltSnap.ini.txt

Yes in the above linked txt file, the encoding is UTF-16LE as it should, however the stream prefix (seen with a hex e,ditor) is EF BB BF indicating UTF-8 encoding.

This is a bug from the text editor you used, the correct prefix for UTF-16 streams is FF FE Which txt editor are you using? (I'm using Atom) it seems it thinks clever to always add UTF-8 prefix even when encoding is different! maybe you enabled some options or something.

If the prefix is wrong, then windows can no longer determine the proper encoding and tries to read utf-16 like if it was utf-8, so some function seems to be crashing, also it seems Notepad converts null characters into spaces and removes line breaks, even Notepad++ is unable to read the file properly and displays NULL every other characters.

Steps to Reproduce

1. Open a standard UTF16 .ini file. E.g.: AltSnap16.ini.txt
2. Save the file.
3. It changes the prefix, resulting in a corrupted .ini.

Expected behavior:

No change in prefix

Actual behavior:

It saves a corrupt file that crashes the program (AltSnap) when it tries to overwrite it.

Reproduces how often:

Unknown

Versions

C:\Users\shelb>atom --version

Atom    : 1.58.0
Electron: 9.4.4
Chrome  : 83.0.4103.122
Node    : 12.14.1

C:\Users\shelb>apm --version
apm  2.6.2
npm  6.14.13
node 12.14.1 x64
atom 1.58.0
python 3.9.7
git 2.33.0.windows.2
visual studio

Edition Windows 11 Pro
Version 21H2
Installed on    ‎7/‎08/‎2021
OS build    22000.194
Experience  Windows Feature Experience Pack 1000.22000.194.0

Additional Information

[shelbyKiraM]

We should be able to at least auto-detect what encoding .ini files are based on the header?

• Via RamonUnch from the other issue:

  Actually I tested the file and the Atom bug is just a configuration quirk. When you open an UTF-16 encoded file, by default Atom still displays it like if it was UTF-8, If you save then it saves the bad version. To open it correctly, before modifying, you have to select the proper encoding in the bottom bar. you can click on the UTF-8 bottom button, then select Auto Detect from the popup menu.

  So, I guess they will ignore your bug report, unless you re-formulate it: It would be nice to have a way so that Atom always auto-detects the file encoding when opening it instead of always displaying as UTF-8.