search

dansanderson / picotool

Tools and Python libraries for manipulating Pico-8 game files. http://www.lexaloffle.com/pico-8.php
MIT License
367 stars 46 forks source link

Added missing PICO-8 built-ins, and reordered table to match docs. #54

Closed Feneric closed 2 years ago

Feneric commented 5 years ago

A handful of PICO-8 built-ins were missing and getting their names incorrectly changed by the minifier; there was also a built-in included twice. These differences were hard to spot because the ordering of the built-ins was almost, but not exactly, like the ordering in the PICO-8 manual. This change puts all the built-ins in the order found in the PICO-8 manual divided up into the same sections used by the manual. The built-ins that are not documented in the manual are included in separate sections following.

Feneric commented 5 years ago

I stumbled onto this while trying to minimize my own project (ironically called "Minima"). Minima is big enough that it won't compress into a cartridge even though it is within the token limit. Fortunately the picotool minimizer fixes that problem. Unfortunately there seem to be three issues that cause problems. The easiest of these is addressed by this fix: a handful of PICO-8 built-ins are missing (including lshr which Minima uses) and are getting wrongly substituted.

The other two are a little more subtle and I'm looking into them now. The simpler one is that escaped special characters like the button graphics are getting substituted with question marks. The more complicated one is that table row names are getting short names applied to them, too, and that breaks function pointer references and the like.

Feneric commented 5 years ago

I've now included a fix for the second issue, too. Special characters are now preserved. Another minor thing I noticed (the removal of the two special comment lines PICO-8 uses to label its cartridges) is also addressed here.

Feneric commented 5 years ago

OK, a fix for the final issue is now included, too. Essentially I've exposed the already-existing alternative AST-aware minimizer to the tool via a new command-line argument. This one acts like the other but is a little more conservative and doesn't minimize table field names. This means that programs that programmatically select field names will continue to work properly even after minimization.

dansanderson commented 2 years ago

Extremely belated thanks for this PR! Lots of good ideas. I'm going to fold most of this in gradually with minor changes. (I'll try to make luamin2 a flag instead of a separate command, for example.) The lexer needs to stay at near 100% unit test coverage too, so I'll work on that as I fold.

dansanderson commented 2 years ago

picotool now has the manual-order organized keyword list, including new ones added since this PR. Good idea.

luamin now has --keep-names-from-file= for project-specific property name preservation (or just working around my delays in adding new built-ins), as well as --keep-all-names to restrict minification to comments and whitespace.

I've decided not to support a luamin flag that exempts all properties. Any project that refers to the same property with dot notation and index notation is already dealing with a fixed quantity of properties that the author can put in a file. This PR does the smart thing to crawl the AST for property names, but this is now slightly challenging as luamin now (I think newer than this PR) does a lexer-only processing of the source, and I'm not quite ready to improve the parser enough to be useful for this purpose for most users.

This PR also adds partial support for high characters. This actually needs to be smart enough to translate between P8SCII (PICO-8 binary) and Unicode (.p8 equivalent), which I think is another development newer than the PR. I'm tracking this at https://github.com/dansanderson/picotool/issues/36 and will pursue a thorough solution.

Many thanks again for this submission!

Feneric commented 2 years ago

You're welcome, and thank you. It's a useful tool and neither Minima nor Anteform could have existed without it. I'm glad the pieces have been helpful. I'll probably end up sticking with my fork for Minima & Anteform for at least the near future as the answer to which properties get indexed is generally "all of them". At some point I may get around to making dedicated lists for each, though.