_js: publish a new version as the current is stuck behind

[JounQin]

It seems the js module is outdated for a long time.

related https://github.com/rx-ts/prettier/issues/133

lolaus.sh

```sh #!/usr/bin/env bash set -euo pipefail DEBUG=${DEBUG:-} help() { cat < . The command is invoked from each directory context matching the glob. Usage: lolaus "./tests/* :(top,exclude)**requirements.txt" ls lolaus "**" pwd lolaus "*/*/package.json" npm test & lolaus "*/*/requirements.txt" "python test.py" lolaus "apps/*/index.js" lolaus "**" ls target-branch other-branch EOF exit 1 } # lolaus [cmd] [target_ref] [source_ref] function main() { [[ $1 == "-h" || $1 == "--help" ]] && help local cmd cmd="${2:-}" local target_ref target_ref=$(get_target_ref "${3:-}") [ -n "${DEBUG}" ] && echo "target: $target_ref" local source_ref source_ref=$(get_source_ref "${4:-}") [ -n "${DEBUG}" ] && echo "source: $source_ref" local concestor concestor=$(get_concestor "$target_ref" "$source_ref") [ -n "${DEBUG}" ] && echo "concestor: $concestor" local diffs diffs=$(get_chagned_files "$concestor" "$source_ref") [ -n "${DEBUG}" ] && echo "diffs: $diffs" local dirs dirs=$(files_to_dirs "$diffs") [ -n "${DEBUG}" ] && echo "dirs: $dirs" [ -z "${cmd}" ] && echo "$dirs" && exit cmd_runner "$cmd" "$dirs" } # Invokes the provided command in each provided direcotry # cmd_runner "" cmd_runner() { : "${1? 'ERROR: cmd_runner ** "'}" : "${2? 'ERROR: cmd_runner *"*'}" local cmd cmd="$1" declare -a dirs mapfile -t dirs <<<"$2" for d in "${dirs[@]}"; do if [[ ! -d $d ]]; then echo "cmd_runner only works on dirs, not: $d" exit 1 fi local cmds="cd $d; $cmd" output="$(eval "$cmds" 2>&1)" [[ -z "$output" ]] && output="$cmd produced no output" [[ $d == "$PWD" ]] && d='./' relative_path=${d/"$PWD/"//} echo -e "$(tput smso) $relative_path $(tput sgr0)\n$output\n" done } # Returns list of sorted and unique containing directories from a list of files. # files_to_dirs "" files_to_dirs() { : "${1? 'ERROR: files_to_dirs ""'}" [[ $# -gt 1 ]] && echo 'ERROR: files_to_dirs takes only one argument' declare -a files mapfile -t files <<<"$1" local path path=$(pwd) declare -a dirs for file in ${files[*]}; do if [ -n "${file##*/*}" ]; then dirs+=("$path") else dirs+=("$path/${file%/*}") fi done printf "%s\n" "${dirs[@]}" | sort -bu } # exits 1 if the provided named ref is not valid. # is_ref_valid is_ref_valid() { : "${1? 'Error: verify_ref needs an argument.'}" ref=$1 git rev-parse --quiet --verify "$ref" &>/dev/null && return 0 echo "ERROR: $ref, is not a valid git ref." && return 1 } # Determines the closest common ansester from two git refs with a default # strategy of 'fork-point' # get_concestor [STRATEGY] get_concestor() { : "${1? 'ERROR: get_chagned_files ** '}" : "${2? 'ERROR: get_chagned_files **'}" local left_ref=$1 local right_ref=$2 local strategy=${3:-'fp'} local concestor is_ref_valid "$left_ref" || echo 'ERROR: left_ref is not a valid git ref.' is_ref_valid "$right_ref" || echo 'ERROR: right_ref is not a valid git ref.' [[ $strategy == 'fp' ]] && concestor=$(git merge-base --fork-point "$left_ref" "$right_ref") echo "$concestor" } # Returns a list of files with diffs between two git refs with optional glob. # get_chagned_files [glob] get_chagned_files() { : "${1? 'ERROR: get_chagned_files ** [glob]'}" : "${2? 'ERROR: get_chagned_files ** [glob]'}" local left_ref=$1 local right_ref=$2 local glob=${3:-':/**'} is_ref_valid "$left_ref" || echo "ERROR: left_ref is not a valid git ref." is_ref_valid "$right_ref" || echo "ERROR: right_ref is not a valid git ref." git diff --name-only "$left_ref".."$right_ref" -- "$glob" } # Returns the verified git reference to target. Defaults to current HEAD. # get_source_ref [branch name] get_source_ref() { local ref=${1:-} [[ -z "$ref" ]] && git symbolic-ref --short HEAD && exit [[ -n "$ref" ]] && is_ref_valid "$ref" && echo "$ref" && return 0 echo "ERROR: $ref, is not a valid source git ref." && exit 1 } # Returns the verified git reference to target. Defaults to next|master|main. # get_target_ref [branch name] get_target_ref() { local ref=${1:-} local defaultTargets=('next' 'master' 'main') if [[ -z $ref ]]; then for t in "${defaultTargets[@]}"; do git rev-parse --quiet --verify "$t" &>/dev/null && echo "$t" && exit done fi ! is_ref_valid "$ref" && echo "ERROR: $ref, is not a valid target git ref." && exit 1 echo "$ref" } main "$@" ```

lolaus.sh
[error] lolaus.sh: SyntaxError: reached EOF without closing quote "
[error]   186 | }
[error]   187 |
[error] > 188 | main "$@"
[error]       |         ^
[error]   189 |

[mvdan]

Thanks for the nudge; this has been in the back of my mind. GopherJS is being maintained again, so we're no longer blocked by that.

I've been thinking that the JS layer should be made more generic so it can be compiled to both JS and WASM. Right now, it's a bit too tightly coupled with GopherJS.

[tdurieux]

Hi @mvdan,

Can we do something to assist you in this task? I would also need the new version of the lib.

Thank you for your lib!

[mvdan]

Thanks for the offer! If someone wants to take a stab at exposing this Go library via JS or Wasm, by all means go ahead. I think the current design I went with wasn't the right approach, and if/when I find enough time to try again, I'd probably start from scratch. It is on my radar, but I'm a bit busy with other projects right at this moment. Perhaps someone else taking this on would be for the best, given that I am by no means an expert at either JS or Wasm.

[tdurieux]

I try to use the Wasm compiler but I did not succeed to go far. I don't have enough knowledge to succeed. It seems that Wasm compiler does not seem to be adapted to do back and forth between the JS and Go since everything needs to be wrapped and converted into primitive types. It is probably better to write the API in JS and then call the Wasm for parsing the bash and output a JSON file that can be processed in JS.

[JounQin]

Here is my naive attempt to use wasm https://github.com/rx-ts/sh-syntax/blob/main/main.go#L208

The parse function is basically working as intended to parse a shell script string to AST by converting syntax.File to map[string]interface {}, but I can't get it work for print, because I have no idea how to convert a js object into map[string]interface {} and then syntax.File because there are many private fields inside.

Unlike solution like gopherjs, js and wasm can't share all data structure easily like @tdurieux said.

Here's my attempt to use latest gopherjs with makeFullWrapper support: https://github.com/mvdan/sh/pull/840.

Hope they help.

If you guys have any other good ideas, I'll try to follow, although I'm not a go expert actually. 😂

[JounQin]

@mvdan Do you have any advice for the WASM module?

[mvdan]

It's time to admit I won't have time to properly do this. And by that I mean one of these two options:

1) Figure out how to publish a newer javascript module with a newer version of Go, GopherJS, and the shell module. 2) Figure out how to publish a new javascript module using Go's webassembly support (perhaps with tinygo to keep the size down).

If anyone plans to work on either of those and succeeds, I am more than happy to deprecate https://www.npmjs.com/package/mvdan-sh in favor of it, or hand it over if that would be easier. I am also of course happy to give my opinions, reviews, advice from the Go side, etc. I just don't have the experience or time to commit to this :smiling_face_with_tear:

[JounQin]

@mvdan

For the first option, see https://github.com/mvdan/sh/pull/840

For the second option, see https://github.com/rx-ts/sh-syntax

[mvdan]

Thanks, I've now merged #840, and will look into a JS release.

I hadn't noticed you had already published sh-syntax as an NPM module that uses WASM. What are the pros and cons compared to the GopherJS version, do you think? I can see it's larger, where presumably most of the size comes from GOOS=js GOARCH=wasm go build. Have you tried tinygo to reduce the size?

I also imagine the wasm version is faster, but that really depends on how good Go/TinyGo are at compiling wasm versus GopherJS with JS.

Longer term I imagine WASM will be the best bet, also because it should be possible to use the shell APIs without needing to use JS at all. Think like a Rust or Python program using a WASM runtime to run the parser, for example.

One more thought: I see you have some glue code to convert the AST nodes to map, so that you can use js.ValueOf and expose them as javascript maps. I have something extremely similar in https://github.com/mvdan/sh/blob/839f7b77f6487282e1d50943429a1f3f4f68ac76/cmd/shfmt/json.go for shfmt -tojson, so perhaps I could expose that as an API. I think what's going to work best for both JS and pure Wasm is to have the Go APIs in Wasm parse shell into a JSON-encoded AST string, as described in https://github.com/mvdan/sh/issues/760#issuecomment-1087300095. Then, the "print" form would be the opposite: take a JSON-encoded AST string, and produce shell as a string.

[mvdan]

I've now published v0.10.0 - note that the unpacked size has jumped from 1.85MiB to 2.3 MiB.

I've published v0.10.1 with JS minification, which brings us back down to 1.5MiB. It would be nice to also bring down the size of the sh-syntax package with Wasm.

[mvdan]

Closing for now, as the "version stuck behind" problem is now fixed. More than happy to deprecate the GopherJS pacakge if we agree that https://github.com/rx-ts/sh-syntax is better.