search

reviewdog / action-eslint

Run eslint with reviewdog
https://github.com/marketplace?type=actions&query=reviewdog
MIT License
237 stars 64 forks source link

encodeURIComponent fails on emojis #81

Closed amorriscode closed 3 years ago

amorriscode commented 3 years ago

When calling utf8length, the code will fail if emojis exist in str.

This causes the entire eslint run to fail because this code doesn't gracefully handle exceptions.

haya14busa commented 3 years ago

Can you show actual emojis which cause errors? I cannot reproduce it.

amorriscode commented 3 years ago

Here are some emojis it was upset about: 🎁 🧐

I modified the formatter in node_modules to output the lineText and error every time one occurs, this is what that looks like:

Line text:                        <div className="checkout-subtitle">�

URIError: URI malformed
    at encodeURIComponent (<anonymous>)
    at utf8length (/Users/amorriscode/dev/monthly/frontend/node_modules/eslint-formatter-rdjson/index.js:18:19)
    at positionFromLineAndUTF16CodeUnitOffsetColumn (/Users/amorriscode/dev/monthly/frontend/node_modules/eslint-formatter-rdjson/index.js:53:13)
    at /Users/amorriscode/dev/monthly/frontend/node_modules/eslint-formatter-rdjson/index.js:105:17
    at Array.forEach (<anonymous>)
    at /Users/amorriscode/dev/monthly/frontend/node_modules/eslint-formatter-rdjson/index.js:98:21
    at Array.forEach (<anonymous>)
    at module.exports (/Users/amorriscode/dev/monthly/frontend/node_modules/eslint-formatter-rdjson/index.js:94:11)
    at Object.format (/Users/amorriscode/dev/monthly/frontend/node_modules/eslint/lib/eslint/eslint.js:612:24)
    at printResults (/Users/amorriscode/dev/monthly/frontend/node_modules/eslint/lib/cli.js:179:30)

Line text:        <div className={"gift-claim-page-error-view-emoji"}>�

URIError: URI malformed
    at encodeURIComponent (<anonymous>)
    at utf8length (/Users/amorriscode/dev/monthly/frontend/node_modules/eslint-formatter-rdjson/index.js:18:19)
    at positionFromLineAndUTF16CodeUnitOffsetColumn (/Users/amorriscode/dev/monthly/frontend/node_modules/eslint-formatter-rdjson/index.js:53:13)
    at /Users/amorriscode/dev/monthly/frontend/node_modules/eslint-formatter-rdjson/index.js:105:17
    at Array.forEach (<anonymous>)
    at /Users/amorriscode/dev/monthly/frontend/node_modules/eslint-formatter-rdjson/index.js:98:21
    at Array.forEach (<anonymous>)
    at module.exports (/Users/amorriscode/dev/monthly/frontend/node_modules/eslint-formatter-rdjson/index.js:94:11)
    at Object.format (/Users/amorriscode/dev/monthly/frontend/node_modules/eslint/lib/eslint/eslint.js:612:24)
    at printResults (/Users/amorriscode/dev/monthly/frontend/node_modules/eslint/lib/cli.js:179:30)

Line text:        <div className={"gift-claim-page-error-view-emoji"}>�

URIError: URI malformed
    at encodeURIComponent (<anonymous>)
    at utf8length (/Users/amorriscode/dev/monthly/frontend/node_modules/eslint-formatter-rdjson/index.js:18:19)
    at positionFromLineAndUTF16CodeUnitOffsetColumn (/Users/amorriscode/dev/monthly/frontend/node_modules/eslint-formatter-rdjson/index.js:53:13)
    at /Users/amorriscode/dev/monthly/frontend/node_modules/eslint-formatter-rdjson/index.js:105:17
    at Array.forEach (<anonymous>)
    at /Users/amorriscode/dev/monthly/frontend/node_modules/eslint-formatter-rdjson/index.js:98:21
    at Array.forEach (<anonymous>)
    at module.exports (/Users/amorriscode/dev/monthly/frontend/node_modules/eslint-formatter-rdjson/index.js:94:11)
    at Object.format (/Users/amorriscode/dev/monthly/frontend/node_modules/eslint/lib/eslint/eslint.js:612:24)
    at printResults (/Users/amorriscode/dev/monthly/frontend/node_modules/eslint/lib/cli.js:179:30)

Line text:                        <div className={"gift-claim-page-gift-emoji"}>�

URIError: URI malformed
    at encodeURIComponent (<anonymous>)
    at utf8length (/Users/amorriscode/dev/monthly/frontend/node_modules/eslint-formatter-rdjson/index.js:18:19)
    at positionFromLineAndUTF16CodeUnitOffsetColumn (/Users/amorriscode/dev/monthly/frontend/node_modules/eslint-formatter-rdjson/index.js:53:13)
    at /Users/amorriscode/dev/monthly/frontend/node_modules/eslint-formatter-rdjson/index.js:105:17
    at Array.forEach (<anonymous>)
    at /Users/amorriscode/dev/monthly/frontend/node_modules/eslint-formatter-rdjson/index.js:98:21
    at Array.forEach (<anonymous>)
    at module.exports (/Users/amorriscode/dev/monthly/frontend/node_modules/eslint-formatter-rdjson/index.js:94:11)
    at Object.format (/Users/amorriscode/dev/monthly/frontend/node_modules/eslint/lib/eslint/eslint.js:612:24)
    at printResults (/Users/amorriscode/dev/monthly/frontend/node_modules/eslint/lib/cli.js:179:30)

Line text:            <div className={"gift-claimed-modal-emoji"}>�

URIError: URI malformed
    at encodeURIComponent (<anonymous>)
    at utf8length (/Users/amorriscode/dev/monthly/frontend/node_modules/eslint-formatter-rdjson/index.js:18:19)
    at positionFromLineAndUTF16CodeUnitOffsetColumn (/Users/amorriscode/dev/monthly/frontend/node_modules/eslint-formatter-rdjson/index.js:53:13)
    at /Users/amorriscode/dev/monthly/frontend/node_modules/eslint-formatter-rdjson/index.js:105:17
    at Array.forEach (<anonymous>)
    at /Users/amorriscode/dev/monthly/frontend/node_modules/eslint-formatter-rdjson/index.js:98:21
    at Array.forEach (<anonymous>)
    at module.exports (/Users/amorriscode/dev/monthly/frontend/node_modules/eslint-formatter-rdjson/index.js:94:11)
    at Object.format (/Users/amorriscode/dev/monthly/frontend/node_modules/eslint/lib/eslint/eslint.js:612:24)
    at printResults (/Users/amorriscode/dev/monthly/frontend/node_modules/eslint/lib/cli.js:179:30)

Line text:                <span className={"gift-purchase-modal-section-title-emoji"} aria-label="gift">�

URIError: URI malformed
    at encodeURIComponent (<anonymous>)
    at utf8length (/Users/amorriscode/dev/monthly/frontend/node_modules/eslint-formatter-rdjson/index.js:18:19)
    at positionFromLineAndUTF16CodeUnitOffsetColumn (/Users/amorriscode/dev/monthly/frontend/node_modules/eslint-formatter-rdjson/index.js:53:13)
    at /Users/amorriscode/dev/monthly/frontend/node_modules/eslint-formatter-rdjson/index.js:105:17
    at Array.forEach (<anonymous>)
    at /Users/amorriscode/dev/monthly/frontend/node_modules/eslint-formatter-rdjson/index.js:98:21
    at Array.forEach (<anonymous>)
    at module.exports (/Users/amorriscode/dev/monthly/frontend/node_modules/eslint-formatter-rdjson/index.js:94:11)
    at Object.format (/Users/amorriscode/dev/monthly/frontend/node_modules/eslint/lib/eslint/eslint.js:612:24)
    at printResults (/Users/amorriscode/dev/monthly/frontend/node_modules/eslint/lib/cli.js:179:30)

Line text:                <span className={"gift-purchase-modal-section-title-emoji"} aria-label="gift">�

URIError: URI malformed
    at encodeURIComponent (<anonymous>)
    at utf8length (/Users/amorriscode/dev/monthly/frontend/node_modules/eslint-formatter-rdjson/index.js:18:19)
    at positionFromLineAndUTF16CodeUnitOffsetColumn (/Users/amorriscode/dev/monthly/frontend/node_modules/eslint-formatter-rdjson/index.js:53:13)
    at /Users/amorriscode/dev/monthly/frontend/node_modules/eslint-formatter-rdjson/index.js:104:20
    at Array.forEach (<anonymous>)
    at /Users/amorriscode/dev/monthly/frontend/node_modules/eslint-formatter-rdjson/index.js:98:21
    at Array.forEach (<anonymous>)
    at module.exports (/Users/amorriscode/dev/monthly/frontend/node_modules/eslint-formatter-rdjson/index.js:94:11)
    at Object.format (/Users/amorriscode/dev/monthly/frontend/node_modules/eslint/lib/eslint/eslint.js:612:24)
    at printResults (/Users/amorriscode/dev/monthly/frontend/node_modules/eslint/lib/cli.js:179:30)

Line text:            <div className={"gift-claimed-modal-emoji"}>�

URIError: URI malformed
    at encodeURIComponent (<anonymous>)
    at utf8length (/Users/amorriscode/dev/monthly/frontend/node_modules/eslint-formatter-rdjson/index.js:18:19)
    at positionFromLineAndUTF16CodeUnitOffsetColumn (/Users/amorriscode/dev/monthly/frontend/node_modules/eslint-formatter-rdjson/index.js:53:13)
    at /Users/amorriscode/dev/monthly/frontend/node_modules/eslint-formatter-rdjson/index.js:105:17
    at Array.forEach (<anonymous>)
    at /Users/amorriscode/dev/monthly/frontend/node_modules/eslint-formatter-rdjson/index.js:98:21
    at Array.forEach (<anonymous>)
    at module.exports (/Users/amorriscode/dev/monthly/frontend/node_modules/eslint-formatter-rdjson/index.js:94:11)
    at Object.format (/Users/amorriscode/dev/monthly/frontend/node_modules/eslint/lib/eslint/eslint.js:612:24)
    at printResults (/Users/amorriscode/dev/monthly/frontend/node_modules/eslint/lib/cli.js:179:30)

Line text:            <div className={"onboarding-peer-group-page-tip-icon"}>�

URIError: URI malformed
    at encodeURIComponent (<anonymous>)
    at utf8length (/Users/amorriscode/dev/monthly/frontend/node_modules/eslint-formatter-rdjson/index.js:18:19)
    at positionFromLineAndUTF16CodeUnitOffsetColumn (/Users/amorriscode/dev/monthly/frontend/node_modules/eslint-formatter-rdjson/index.js:53:13)
    at /Users/amorriscode/dev/monthly/frontend/node_modules/eslint-formatter-rdjson/index.js:105:17
    at Array.forEach (<anonymous>)
    at /Users/amorriscode/dev/monthly/frontend/node_modules/eslint-formatter-rdjson/index.js:98:21
    at Array.forEach (<anonymous>)
    at module.exports (/Users/amorriscode/dev/monthly/frontend/node_modules/eslint-formatter-rdjson/index.js:94:11)
    at Object.format (/Users/amorriscode/dev/monthly/frontend/node_modules/eslint/lib/eslint/eslint.js:612:24)
    at printResults (/Users/amorriscode/dev/monthly/frontend/node_modules/eslint/lib/cli.js:179:30)

Line text:            <div className={"onboarding-peer-group-page-tip-icon"}>�

URIError: URI malformed
    at encodeURIComponent (<anonymous>)
    at utf8length (/Users/amorriscode/dev/monthly/frontend/node_modules/eslint-formatter-rdjson/index.js:18:19)
    at positionFromLineAndUTF16CodeUnitOffsetColumn (/Users/amorriscode/dev/monthly/frontend/node_modules/eslint-formatter-rdjson/index.js:53:13)
    at /Users/amorriscode/dev/monthly/frontend/node_modules/eslint-formatter-rdjson/index.js:105:17
    at Array.forEach (<anonymous>)
    at /Users/amorriscode/dev/monthly/frontend/node_modules/eslint-formatter-rdjson/index.js:98:21
    at Array.forEach (<anonymous>)
    at module.exports (/Users/amorriscode/dev/monthly/frontend/node_modules/eslint-formatter-rdjson/index.js:94:11)
    at Object.format (/Users/amorriscode/dev/monthly/frontend/node_modules/eslint/lib/eslint/eslint.js:612:24)
    at printResults (/Users/amorriscode/dev/monthly/frontend/node_modules/eslint/lib/cli.js:179:30)

Line text:            <div className={"onboarding-peer-group-page-tip-icon"}>�

URIError: URI malformed
    at encodeURIComponent (<anonymous>)
    at utf8length (/Users/amorriscode/dev/monthly/frontend/node_modules/eslint-formatter-rdjson/index.js:18:19)
    at positionFromLineAndUTF16CodeUnitOffsetColumn (/Users/amorriscode/dev/monthly/frontend/node_modules/eslint-formatter-rdjson/index.js:53:13)
    at /Users/amorriscode/dev/monthly/frontend/node_modules/eslint-formatter-rdjson/index.js:105:17
    at Array.forEach (<anonymous>)
    at /Users/amorriscode/dev/monthly/frontend/node_modules/eslint-formatter-rdjson/index.js:98:21
    at Array.forEach (<anonymous>)
    at module.exports (/Users/amorriscode/dev/monthly/frontend/node_modules/eslint-formatter-rdjson/index.js:94:11)
    at Object.format (/Users/amorriscode/dev/monthly/frontend/node_modules/eslint/lib/eslint/eslint.js:612:24)
    at printResults (/Users/amorriscode/dev/monthly/frontend/node_modules/eslint/lib/cli.js:179:30)

Line text:              <span className="enrollment-item-purchase-method-emoji">�

URIError: URI malformed
    at encodeURIComponent (<anonymous>)
    at utf8length (/Users/amorriscode/dev/monthly/frontend/node_modules/eslint-formatter-rdjson/index.js:18:19)
    at positionFromLineAndUTF16CodeUnitOffsetColumn (/Users/amorriscode/dev/monthly/frontend/node_modules/eslint-formatter-rdjson/index.js:53:13)
    at /Users/amorriscode/dev/monthly/frontend/node_modules/eslint-formatter-rdjson/index.js:105:17
    at Array.forEach (<anonymous>)
    at /Users/amorriscode/dev/monthly/frontend/node_modules/eslint-formatter-rdjson/index.js:98:21
    at Array.forEach (<anonymous>)
    at module.exports (/Users/amorriscode/dev/monthly/frontend/node_modules/eslint-formatter-rdjson/index.js:94:11)
    at Object.format (/Users/amorriscode/dev/monthly/frontend/node_modules/eslint/lib/eslint/eslint.js:612:24)
    at printResults (/Users/amorriscode/dev/monthly/frontend/node_modules/eslint/lib/cli.js:179:30)

They don't show up as emojis in the eslint-formatter-rdjson code but they are emojis in the code I am linting.

shogo82148 commented 3 years ago

It looks that a half of surrogate pairs in UTF-16 is pass to encodeURIComponent. because commonSuffixLength doesn't handle surrogate pairs correctly. https://github.com/reviewdog/action-eslint/blob/8bb4953c9f04822894c5cb0facdcf7f63a2b9d23/eslint-formatter-rdjson/index.js#L49-L57

amorriscode commented 3 years ago

@haya14busa just curious, what's the release cadence like? I'm looking forward to seeing this in a release. 😀

haya14busa commented 3 years ago

It's already released :) https://github.com/reviewdog/action-eslint/pull/89

amorriscode commented 3 years ago

Interesting... 🤔

I saw the original issue with a GitHub Action that was running this command: eslint -f rdjson packages --ext js | reviewdog -f=rdjson -reporter=github-pr-review.

Today I removed the files with emojis .eslintignore but errors were still occurring. I tried to switch my GitHub Action up to use the action-eslint but that had the following error:

Error: invalid surrogate character
      at utf8length (/home/runner/work/_actions/reviewdog/action-eslint/v1/eslint-formatter-rdjson/index.js:33:15)
      at positionFromLineAndUTF16CodeUnitOffsetColumn (/home/runner/work/_actions/reviewdog/action-eslint/v1/eslint-formatter-rdjson/index.js:71:11)
      at /home/runner/work/_actions/reviewdog/action-eslint/v1/eslint-formatter-rdjson/index.js:130:17
      at Array.forEach (<anonymous>)
      at /home/runner/work/_actions/reviewdog/action-eslint/v1/eslint-formatter-rdjson/index.js:123:21
      at Array.forEach (<anonymous>)
      at module.exports (/home/runner/work/_actions/reviewdog/action-eslint/v1/eslint-formatter-rdjson/index.js:119:11)
      at Object.format (/home/runner/work/frontend/frontend/node_modules/eslint/lib/eslint/eslint.js:612:24)
      at printResults (/home/runner/work/frontend/frontend/node_modules/eslint/lib/cli.js:179:30)
      at async Object.execute (/home/runner/work/frontend/frontend/node_modules/eslint/lib/cli.js:314:13)
  reviewdog: parse error: failed to unmarshal rdjson (DiagnosticResult): proto: syntax error (line 1:1): unexpected token

This is what the GitHub Action looks like:

name: ESLint
on: [pull_request]

jobs:
  eslint:
    name: eslint
    runs-on: ubuntu-latest
    steps:
      # Get the code changes
      - uses: actions/checkout@v2

      # Lint review dog
      - uses: reviewdog/action-eslint@v1
        with:
          eslint_flags: "packages --ext js"

Not sure where to go from here but thanks for being so responsive!

shogo82148 commented 3 years ago

oh... I've reproduced.

diff --git a/testdata/test.js b/testdata/test.js
index 3d2ca59..22be856 100644
--- a/testdata/test.js
+++ b/testdata/test.js
@@ -17,3 +17,5 @@ function f()
 {
   console.log('🐶');
 }
+
+🐶 // test for unexpected character error

on https://github.com/reviewdog/action-eslint/commit/f7bedb968ef348c47a45a68e55944eaccbd771c2

$ npx eslint ./testdata/*.js -f ./eslint-formatter-rdjson/index.js 

Oops! Something went wrong! :(

ESLint: 7.12.1

Error: invalid surrogate character
    at utf8length (/Users/shogoichinose/src/github.com/shogo82148/action-eslint/eslint-formatter-rdjson/index.js:33:15)
    at positionFromLineAndUTF16CodeUnitOffsetColumn (/Users/shogoichinose/src/github.com/shogo82148/action-eslint/eslint-formatter-rdjson/index.js:71:11)
    at /Users/shogoichinose/src/github.com/shogo82148/action-eslint/eslint-formatter-rdjson/index.js:129:20
    at Array.forEach (<anonymous>)
    at /Users/shogoichinose/src/github.com/shogo82148/action-eslint/eslint-formatter-rdjson/index.js:123:21
    at Array.forEach (<anonymous>)
    at module.exports (/Users/shogoichinose/src/github.com/shogo82148/action-eslint/eslint-formatter-rdjson/index.js:119:11)
    at Object.format (/Users/shogoichinose/src/github.com/shogo82148/action-eslint/node_modules/eslint/lib/eslint/eslint.js:603:24)
    at printResults (/Users/shogoichinose/src/github.com/shogo82148/action-eslint/node_modules/eslint/lib/cli.js:176:30)
    at async Object.execute (/Users/shogoichinose/src/github.com/shogo82148/action-eslint/node_modules/eslint/lib/cli.js:307:13)