Suggestions and comparison with Alone in the Dark

[nummacway]

This is a long one. Feel free to split the accepted suggestions into multiple issues.

First of all, I like the fact that this tool accepts full-width and arbitrary-height input. I also like the fact that the widths per palette aren't fixed anymore but can be adjusted. (For full-width, you kinda had to.)

Let's start with a question:

• --addendcolor: I don't understand the use of this switch. Why set all 64 palette bytes to the same single byte at the end? Wouldn't it make more sense to make the tool set a selectable number of palettes to an individual set of four colors per palette?

Here are some suggestions for features:

• I would need this tool to process a horizontally-scrollable image of 320x72 pixels, and place the resulting two tilemaps below each other (so it uses a 240x144 pixel area of the tilemap, where (80,0,240,72) and (0,72,160,144) would be identical), making it easy to scroll horizontally. Once the 72 pixels are rendered, I'd scroll to another area of the map, so it's perfectly fine to have the two parts right below eachother.
• Allow the user to limit the number of palettes used, optionally supplying static values for the others so these could be used when they fit by conincidence.
• Allow the user to limit the number of palette changes per scanline if one expects HBlank to be affected by having OBJs present. For example, Alone in the Dark also has OBJs on HiColor backgrounds. If further optimizations (see below) were added, it might make sense to set this parameter in cycles rather than palette changes.
  • Note: In additon to OBJs, HBlank is also affected by the Window, but a (traditional) Window wouldn't make sense without an option to limit palettes to leave one for the Window.

Here are some ideas how image quality could be improved. Some of which will likely increase complexity beyond an acceptable point, especially when combined.

• Allow the "palette widths" to become discontigous regions, e.g. 0010022133.
• Currently, dynamically updating the "palette widths" in the left part isn't possible, since palette update can never coincide with palette index updates. Maybe there is a way to improve this.
• "Left-and-right-split" could however be completely omitted if the user was using less palettes, so all palettes could be updated in HBlank and OAM scan. So in this case, the entire image could easily have its palette widths optimized every 8 rows, by treating every 8 rows of pixel as an individual image. -Instead of alternatingly changing strictly the first or second half of the palettes, one could also update the palettes that "need it most" (defined by improvement of square deviation by applying a new palette). I think that's especially useful in rows not divisable by 8. (This would add too much complexity to rows divisable by 8 due to palette widths being optimized as well there.)
• A tile could also benefit from having its palette index re-assigned mid-tile, though that will be way more complex.

After writing the above suggestions, I wanted to know which of these are employed by Alone in the Dark (related code always runs from WRAM at $D000 through $DFCB):

• It uses wider backgrounds (192x144), though not as wide as I requested. Due to using the full height, it can't go much bigger anyway. It already uses 432 tiles in static backgrounds, but more are used in case of an animated background.
• Colors do not depend on scrolling.
• It almost exclusively uses the first four (three in a few scenes, e.g. second one) palettes, and only ever changes these four's (or three's) colors. It does use the other palettes for only a single tile each. I don't think there's anything special about these four (or five) tiles. The static colors aren't global but depend on the current scene/level/map (whatever you call it).
• Background priority tiles (should they exist in a scene) share their palettes with other tiles.
• It always updates all four palettes. I'm surprised that it has time for these ca. 100 cycles (200 dots), since it does use quite a lot of OBJs at times due to the character's large metasprites.
• Palette regions are discontigous.
• Palette regions change at tile boundary (staticly in tile map, except for background animations, which have to change the tile attributes in code).
• It only runs the palette update code every two scanlines. In the meantime, it does stuff at $1D0C through $1DD4. For this, it doesn't use SP or CALL, but HL and JP.
• Though it does sometimes look like palettes changed mid-tile (meaning by code), there's no sign of it in the code: $D000 to $DFCB is only about writing immediate bytes to [HL] (and the required setup), jumping to $1DOC and busy-waiting for HBlank.
• Background animations can change tiles and their palettes (so they have to update both tile map banks in VRAM).

Appendix: Background palette indizes used Alone in the Dark scene 1 and 2, from red (BG0) to pink (BG7): Scene 1 (where you start, at the gate): Scene 2 (the only way to exit scene 1):

[bbbbbr]

Thanks for opening the PR, some of these features sound cool and could be useful. In general I'm not planning to make improvements and in particular algorithmic and structural changes such as might be required by some of the suggestions in the present time. While the code base has been improved somewhat from the original, it's still not particularly pleasant to work in and modify.

If you'd like to work on implementing some of the changes I am open to merging PRs which add them. The requirements would be that they don't make the code more complex and harder to work with (ideally they would improve surrounding code quality and readability somewhat), and are well named, commented and generally documented (something that is still a work in progress from the original).

I don't remember the details from testing exactly, but there was headroom for up to something like 10 objs per line with 4 palettes loading per scanline. Maybe a bit more with the pre-compiled output version.