Minifying fails when strings contain hex escapes or nulls

[tve]

The following simple program works when run from RAM with the IDE (0.73.7) on an esp8266 2v06:

console.log("The length should be 7: " + '\xC3\xBF\xC3\0\xBF\xC3\xBF'.length);

But if I choose "save to flash" with esprima minification, it fails:

 ____                 _
|  __|___ ___ ___ _ _|_|___ ___
|  __|_ -| . |  _| | | |   | . |
|____|___|  _|_| |___|_|_|_|___|
         |_| espruino.com
 2v06 (c) 2019 G.Williams
Espruino is Open Source. Our work is supported
only by sales of official boards and donations:
http://espruino.com/Donate
Flash map 4MB:1024/1024, manuf 0xe0 chip 0x4016
>
>load()
=undefined
Uncaught SyntaxError: Got UNFINISHED STRING expected EOF
 at line 1 col 40
console.log('The length should be 7: '+'extends typeof extendstypeof extends typeof'.length)
                                                                           ^

If I try espruino-cli (a little older version "espruino@0.1.23", I'll try with something newer tomorrow) and use -m -o bug.min.js the hex escapes in the minified file look suspicious:

require("Storage").write(".bootcde","console.log('The length should be 6: '+'Ã\\xBFÃ\\0\\xBFÃ\\xBF';
load()

From my tests the IDE fails due to the \0 and the espruino-cli fails due to the \xC3.

[MaBecker]

Have a look at this result of a very similar issue and how it can be fixed.

https://github.com/espruino/Espruino/issues/1583#issuecomment-474372842

[gfwilliams]

Interestingly, an upload does:

"require(\"Storage\").write(\".bootcde\",\"console.log('The length should be 7: '+'ÿÃ\\u0000¿Ã¿'.length)"

So minification decided there wasn't any point keeping the escaping, apart from \0.

and the result is:

>print(require("Storage").read(".bootcde"))
console.log('The length should be 7: '+'ÿÿÿ'.length)
>print(E.toJS(require("Storage").read(".bootcde")))
"console.log('The length should be 7: '+'\xC3\xBF\xC3\0\xBF\xC3\xBF'.length)"

(E.toJS is like JSON.stringify but is more compact)

So it looks like the data has got in there uncorrupted - but the issue is that somewhere along the line the \\u0000 got interpreted when it should have been kept in the string...

[gfwilliams]

Ok, that was easy. Turns out Esprima was configured to remove all escape characters, including \0 (I don't know why it seemed it didn't, but the debugger shows it does).

I just stopped it taking them out, which I think is fair.