Shellcheck crashes on certain input with Chinese characters (only reproduced in the snap distribution)

brlin-tw commented 5 years ago

For bugs

Rule Id (if any, e.g. SC1000):
My shellcheck version (shellcheck --version or "online"):
[x] The rule's wiki page does not already cover this (e.g. https://shellcheck.net/wiki/SC2086)
[ ] I tried on shellcheck.net and verified that this is still a problem on the latest commit (not reproduced on shellcheck.net, reproduced on shellcheck snap on edge channel)

Here's a snippet or screenshot that shows the problem:

#!/usr/bin/env bash
declare start_time="$(date +%s)"
sleep 5
declare stop_time="$(date +%s)"

echo "本次脚本运行了$((${stop_time}-${start_time}))秒。"
#echo "This script runs$((${stop_time}-${start_time}))seconds."

Here's what shellcheck currently says:

$ shellcheck shellcheck-bug-reproduction.bash 

In shellcheck-bug-reproduction.bash line 2:
declare start_time="$(date +%s)"
        ^--------^ SC2155: Declare and assign separately to avoid masking return values.

In shellcheck-bug-reproduction.bash line 4:
declare stop_time="$(date +%s)"
        ^-------^ SC2155: Declare and assign separately to avoid masking return values.

In shellcheck-bug-reproduction.bash line 6:
shellcheck-bug-reproduction.bash: <stdout>: commitBuffer: invalid argument (invalid character)
echo "For more information:
  https://www.shellcheck.net/wiki/SC2155 -- Declare and assign separately to ...
  https://www.shellcheck.net/wiki/SC2004 -- $/${} is unnecessary on arithmeti...

Here's what I wanted or expected to see:


In shellcheck-bug-reproduction.bash line 2:
declare start_time="$(date +%s)"
        ^--------^ SC2155: Declare and assign separately to avoid masking return values.

In shellcheck-bug-reproduction.bash line 4:
declare stop_time="$(date +%s)"
        ^-------^ SC2155: Declare and assign separately to avoid masking return values.

In shellcheck-bug-reproduction.bash line 7:
echo "This script runs$((${stop_time}-${start_time}))seconds."
                         ^----------^ SC2004: $/${} is unnecessary on arithmetic variables.
                                      ^-----------^ SC2004: $/${} is unnecessary on arithmetic variables.

For more information:
  https://www.shellcheck.net/wiki/SC2155 -- Declare and assign separately to ...
  https://www.shellcheck.net/wiki/SC2004 -- $/${} is unnecessary on arithmeti...

snap information

tracking:     edge
refresh-date: today at 17:11 CST
channels:
  stable:    v0.6.0                2019-01-11 (329) 3MB -
  candidate: ↑                                          
  beta:      ↑                                          
  edge:      v0.6.0+git201.788cf17 2019-07-05 (690) 3MB -
installed:   v0.6.0+git201.788cf17            (690) 3MB -

Can be reproduced on edge and stable channel

KenSharp commented 4 years ago

The snap struggles with UTF8 characters in general.

#!/bin/sh

curl -s "https://some.url/application?id=$1" | grep -F NAV | sed 's/>/\n/g' | grep -F GBX | cut -d'<' -f1 | sed 's/ /\n/' | tail -1 > $2.csv

$ shellcheck ~/test.sh

In /home/ken/test.sh line 3:
/home/ken/test.sh: <stdout>: commitBuffer: invalid argument (invalid character)

#!/bin/sh

[[ $ARTIST =~ 'ａ'|'ｅ'|'ｉ'|'Ａ'|'Ｅ'|'Ｉ'|'Ｏ'|'Ｓ'|'Ｕ'|'Ｙ' ]] && ARTIST=$(iconv -f UTF-8 -t ASCII//TRANSLIT <<<"$ARTIST")

$ shellcheck ~/test2.sh

In /home/ken/test2.sh line 3:
/home/ken/test2.sh: <stdout>: commitBuffer: invalid argument (invalid character)

shellcheck itself works fine when installed via the repositories (Ubuntu is a few versions behind), and the online version works fine as expected.

This is related the use of snaps and could be a snaps bug. Unless there is a configuration option missing somewhere?

KenSharp commented 4 years ago

brlin-tw commented 4 years ago

I've made up a patch to fix this, please review.

Take too long to come back to this :(

koalaman / shellcheck