📝 Formatter replaces emojis

[melMass]

Environment information

CLI:
  Version:                      1.7.1
  Color support:                true

Platform:
  CPU Architecture:             x86_64
  OS:                           windows

Environment:
  BIOME_LOG_DIR:                unset
  NO_COLOR:                     unset
  TERM:                         "xterm-256color"
  JS_RUNTIME_VERSION:           "v20.9.0"
  JS_RUNTIME_NAME:              "node"
  NODE_PACKAGE_MANAGER:         unset

Biome Configuration:
  Status:                       Loaded successfully
  Formatter disabled:           false
  Linter disabled:              false
  Organize imports disabled:    false
  VCS disabled:                 true

Formatter:
  Format with errors:           false
  Indent style:                 Tab
  Indent size:                  0
  Indent width:                 0
  Line ending:                  Lf
  Line width:                   80
  Attribute position:           Auto
  Ignore:                       []
  Include:                      []

JavaScript Formatter:
  Enabled:                      false
  JSX quote style:              Double
  Quote properties:             AsNeeded
  Trailing comma:               All
  Semicolons:                   AsNeeded
  Arrow parentheses:            Always
  Bracket spacing:              false
  Bracket same line:            false
  Quote style:                  Single
  Indent style:                 unset
  Indent size:                  unset
  Indent width:                 2
  Line ending:                  unset
  Line width:                   unset
  Attribute position:           Auto

JSON Formatter:
  Enabled:                      true
  Indent style:                 unset
  Indent width:                 unset
  Indent size:                  unset
  Line ending:                  unset
  Line width:                   unset
  Trailing Commas:              unset

Workspace:
  Open Documents:               0

Configuration

{
    "$schema": "https://biomejs.dev/schemas/1.6.1/schema.json",
    "organizeImports": {
        "enabled": true
    },
    "linter": {
        "enabled": true,
        "rules": {
            "recommended": true
        }
    },
    "formatter": {
        "lineEnding": "lf"
    },
    "javascript": {
        "formatter": {
            "quoteStyle": "single",
            "semicolons": "asNeeded",
            "indentWidth": 2
        }
    }
}

Playground link

https://biomejs.dev/playground/?code=CgBlAHgAcABvAHIAdAAgAGMAbwBuAHMAdAAgAGkAbgBmAG8ATABvAGcAZwBlAHIAIAA9ACAAYwByAGUAYQB0AGUATABvAGcAZwBlAHIAKAAnADkhD%2F4nACwAIAAnAHkAZQBsAGwAbwB3ACcAKQAKAGUAeABwAG8AcgB0ACAAYwBvAG4AcwB0ACAAdwBhAHIAbgBMAG8AZwBnAGUAcgAgAD0AIABjAHIAZQBhAHQAZQBMAG8AZwBnAGUAcgAoACcAoCYP%2FicALAAgACcAbwByAGEAbgBnAGUAJwAsACAAJwB3AGEAcgBuACcAKQAKAGUAeABwAG8AcgB0ACAAYwBvAG4AcwB0ACAAZQByAHIAbwByAEwAbwBnAGcAZQByACAAPQAgAGMAcgBlAGEAdABlAEwAbwBnAGcAZQByACgAJwA92CXdJwAsACAAJwByAGUAZAAnACwAIAAnAGUAcgByAG8AcgAnACkACgBlAHgAcABvAHIAdAAgAGMAbwBuAHMAdAAgAHMAdQBjAGMAZQBzAHMATABvAGcAZwBlAHIAIAA9ACAAYwByAGUAYQB0AGUATABvAGcAZwBlAHIAKAAnAAUnJwAsACAAJwBnAHIAZQBlAG4AJwApAAoACgA%3D

Code of Conduct

• [X] I agree to follow Biome's Code of Conduct

[melMass]

I can't reproduce the issue on the playground, but this is what happens in neovim:

I think it was fixed in #693 but I can't find why it happens

[huijiewei]

webstorm has same problem

[redbmk]

@melMass are you using ALE to format or something else? I'm using ALE and see this as well, but it doesn't happen when formatting from the CLI

[redbmk]

looks like it might have something to do with the way neovim is exporting to stdout:

before:

console.log("⚠️ ")

highlight the line with V, then:

:'<,'>!bunx biome format --stdin-file-path=asdf.ts

after:

console.log("! ");

but if I copy and paste that directly into the console it seems fine:

❯ echo 'console.log("⚠<fe0f> ")' | bunx biome check --apply --stdin-file-path=asdf.ts
console.log("⚠️ ");

I'm not entirely sure where the <fe0f> is coming from, but when I paste into zsh/iterm2 that shows up.

[redbmk]

ok it might still have something to do with biome not quite understanding what encoding stdin is in or something...

:'<,'>!cat

leaves it untouched, and:

:'<,'>!cat > temp.ts && bunx biome check --apply temp.ts && cat temp.ts

gives me:

Checked 1 file in 2ms. Fixed 1 file.
console.log("⚠️ ");

[redbmk]

Hmm I'm also finding this pretty confusing. Without --apply it seems to print an error but the exit code is still 0. With --apply the output looks identical.

When putting them through diff or piping through stdin they do look different though, so I guess it's not actually a neovim issue

[redbmk]

OK I think I figured it out. I believe biome is trying to strip ansi characters when it detects that the output is not interactive (e.g. to a pipe or a file). In this case its instead converting the emoji into a plain string. As a workaround, if you use --colors=force then it will keep the emoji.

A slightly different bug seems to be in the check mode where it prints the output first, then a message saying it would have printed the following content. So it's backwards and it's also reporting an error when there actually shouldn't be one.

[ematipico]

I know we have the issue, but I am unable to replicate it, maybe because I have a shell that supports Ansi? I wished there was a consistent way - at least for me - to get an invalid output.

BTW, if anyone wants to help, we use the termcolor crate to setup the stdout stream. Maybe someone can suggest a better option, among of these variants: https://docs.rs/termcolor/1.4.1/termcolor/enum.ColorChoice.html#variants

[melMass]

Sorry I can't track exactly when/how but it's now working properly! Just to note that it wasn't happening for all emojis but only some. I'm going to close the issue Edit: I just saw all redmk messages, I guess it still needs a proper conclusion, feel free to close it