Tokenize fails with busybox 1.11.2

[wvengen]

Running on an embedded system with Busybox 1.11.2, any input fails:

$ echo {} | ./JSON.sh
EXPECTED string GOT EOF

$ egrep --version
egrep: unrecognized option `--version'
BusyBox v1.11.2 (2016-05-23 16:56:16 EEST) multi-call binary

No help available.
$ busybox --help
BusyBox v1.11.2 (2016-05-23 16:56:16 EEST) multi-call binary
Copyright (C) 1998-2008 Erik Andersen, Rob Landley, Denys Vlasenko
and others. Licensed under GPLv2.
See source distribution for full notice.

Usage: busybox [function] [arguments]...
   or: function [arguments]...

    BusyBox is a multi-call binary that combines many common Unix
    utilities into a single executable.  Most people will create a
    link to busybox for each function they wish to use and BusyBox
    will act like whatever it was invoked as!

Currently defined functions:
    [, [[, adjtimex, arping, ash, awk, basename, brctl, bunzip2, bzcat, cat, chgrp, chmod, chown, chroot, clear, cp, crond, crontab, cut, date, dd, df, diff, dirname,
    dmesg, du, echo, egrep, env, expr, false, fgrep, find, free, garp, getty, grep, gunzip, gzip, halt, head, hexdump, hostid, hwclock, id, ifconfig, init, insmod,
    kill, killall, killall5, klogd, length, less, ln, lock, logger, login, logread, ls, lsmod, md5sum, mesg, mkdir, mkfifo, mknod, mktemp, mount, mv, nc, netmsg,
    netstat, nice, nslookup, passwd, pgrep, pidof, ping, ping6, pivot_root, pkill, poweroff, printf, ps, pwd, pwdog, rdate, realpath, reboot, reset, rm, rmdir,
    rmmod, route, sed, seq, sh, sleep, sort, start-stop-daemon, strings, switch_root, sync, sysctl, syslogd, tail, tar, tee, telnet, telnetd, test, tftp, time,
    top, touch, tr, traceroute, true, udhcpc, umount, uname, uniq, uptime, uudecode, uuencode, vconfig, vi, watchdog, wc, wget, which, xargs, yes, zcat, zcip

Then I looked at tokenization only. Sourcing JSON.sh doesn't work (still expecting input), so I changed the last lines like this:

#if ([ "$0" = "$BASH_SOURCE" ] || ! [ -n "$BASH_SOURCE" ]);
#then
#  parse_options "$@"
#  tokenize | parse
#fi
parse_options "$@"
tokenize

What then happens, is:

$ echo {} | shinc/JSON.sh
{
$ echo '{"foo": 1234, "bar":{"something": "hi there"}}' | shinc/JSON.sh
{

Running all tests indeed shows failures.

[wvengen]

I've tested with a newer version of busybox 1.21.1 from https://busybox.net/downloads/binaries/latest/ Adding this busybox'es egrep and grep to PATH lets it work. So the problem must be the (older) grep/egrep version in busybox 1.11.1.

[dominictarr]

@wvengen if we can change the grep commands to something that works across versions we can merge that