Git staging individual changes converts UTF-8 with BOM to UTF-8

[ridago]

Type: Bug

1. Start with a UTF-8 with BOM file
2. Make a change to some line
3. Click the change indicator to the left and choose "Stage change". Do NOT stage the whole file
4. Commit the staged change

The file in HEAD is now UTF-8, not UTF-8 with BOM. I.e. VSCode removed the EF BB BF bytes at the start of the file. The file in the current workspace is still UTF-8 with BOM, meaning if you now stage the whole file it will revert back to a UTF-8 with BOM file, but the history will show it first being converted to UTF-8, then back to UTF-8 with BOM.

This is not the same behaviour as staging individual changes with git add -p

VS Code version: Code 1.81.1 (6c3e3dba23e8fadc360aed75ce363ba185c49794, 2023-08-09T22:22:42.175Z) OS version: Windows_NT x64 10.0.22621 Modes:

System Info

|Item|Value| |---|---| |CPUs|Intel(R) Core(TM) i7-8700K CPU @ 3.70GHz (12 x 3696)| |GPU Status|2d_canvas: enabled
canvas_oop_rasterization: disabled_off
direct_rendering_display_compositor: disabled_off_ok
gpu_compositing: enabled
multiple_raster_threads: enabled_on
opengl: enabled_on
rasterization: enabled
raw_draw: disabled_off_ok
video_decode: enabled
video_encode: enabled
vulkan: disabled_off
webgl: enabled
webgl2: enabled
webgpu: enabled| |Load (avg)|undefined| |Memory (System)|31.93GB (22.85GB free)| |Process Argv|--crash-reporter-id 11feb52b-844e-4d4e-a78a-c5f927f3e2df| |Screen Reader|no| |VM|0%|

Extensions (17)

Extension|Author (truncated)|Version ---|---|--- vscode-docker|ms-|1.26.0 python|ms-|2023.14.0 vscode-pylance|ms-|2023.8.40 jupyter|ms-|2023.7.1002162226 jupyter-keymap|ms-|1.1.2 jupyter-renderers|ms-|1.0.17 vscode-jupyter-cell-tags|ms-|0.1.8 vscode-jupyter-slideshow|ms-|0.1.5 remote-containers|ms-|0.304.0 remote-ssh|ms-|0.102.0 remote-ssh-edit|ms-|0.86.0 remote-wsl|ms-|0.81.0 vscode-remote-extensionpack|ms-|0.24.0 cpptools|ms-|1.17.4 hexeditor|ms-|1.9.12 remote-explorer|ms-|0.4.1 remote-server|ms-|1.4.3

A/B Experiments

``` vsliv368cf:30146710 vsreu685:30147344 python383:30185418 vspor879:30202332 vspor708:30202333 vspor363:30204092 vslsvsres303:30308271 vserr242cf:30382550 pythontb:30283811 vsjup518:30340749 pythonptprofiler:30281270 vshan820:30294714 vstes263:30335439 vscod805:30301674 binariesv615:30325510 bridge0708:30335490 bridge0723:30353136 vsaa593cf:30376535 pythonvs932:30410667 py29gd2263:30792226 vsclangdf:30486550 c4g48928:30535728 dsvsc012:30540252 pynewext54:30695312 azure-dev_surveyonecf:30548226 vsccc:30803844 282f8724:30602487 f6dab269:30613381 2i9eh265:30646982 showlangstatbar:30737416 a2ce3375:30757347 pythonfmttext:30731395 fixshowwlkth:30771522 showindicator:30805244 pythongtdpath:30769146 i26e3531:30792625 gsofb:30804716 pythonnosmt12:30797651 pythonidxptcf:30805731 pythonnoceb:30805159 asynctok:30821568 dsvsc013:30795093 dsvsc014:30804076 diffeditorv2:30821572 ```

[Gouvernathor]

Happens to me as well, and causes issues on the repos I'm a part of. I would very much like a solution to this.

[hhmmjjnn]

Me too.

Not only with the Stage Selected Ranges command, but with Unstage Selected Ranges and Revert Selected Ranges, as well.

[Scorg]

Can confirm this unfixed as of 1.90.1

[michaelmesser]

This bug is annoying when working on Visual Studio projects because Visual Studio creates BOMs by default, but VSCode silently removes them leading to a mix of encodings.

[sirtheta]

Our old codebase uses mostly windows-1252, and we need to use this encoding. When we stage only selected ranges, it will commit the file as utf-8. This is very annoying.

[hhmmjjnn]

Right. It doesn't have anything to do with the BOM. Any encoding will be lost in favor of (BOM-less) UTF-8 regardless.

[ridago]

Had another look into this today and realised it's a duplicate of #111915 so it should probably me marked as such