search

dansanderson / picotool

Tools and Python libraries for manipulating Pico-8 game files. http://www.lexaloffle.com/pico-8.php
MIT License
367 stars 46 forks source link

Glyph support #36

Closed dansanderson closed 2 years ago

dansanderson commented 6 years ago

Picotool currently uses the 'ascii' encoding when reading and writing .p8 files. I believe this would fail when code contains glyphs.

The glyph representation in .p8 files has changed as of 0.1.11e to use a UTF-8 encoding, so it's likely that the correct solution is to switch to that encoding, then carefully make sure these sequences are converted to binary correctly. It might be worth accepting only the narrowest range of supported code points and not all UTF-8 values. IIRC previous behavior with high chars in Lua source was undefined in Pico-8, usually accepted silently or possibly converted invisibly.

Also, glyphs can now be used in identifiers, so this likely requires an update to the lexer.

dansanderson commented 2 years ago

picotool now treats all Lua code as P8SCII bytestrings internally. When loading and saving .p8 files, P8SCII is converted from and to Unicode by a fixed mapping of the set of 256 characters.

https://github.com/dansanderson/picotool/commit/750301a524b806b0daebcb2b20abf85f20124fc9 adds support for high chars in glyphs.