TwbT structures research

[PeridexisErrant]

Mifki has said that the only thing preventing TwbT being merged into DFHack is the additional structures research required. Instructions here; it would be good to merge this too.

[seylerius]

Would merging this make it possible in the future to use TwbT without breaking other DFHack UI overlays? As an example, when using TwbT and the siege engine mod, the siege engine gui replaces the map, rather than overlaying it.

[lethosor]

No, this would just mean we'd have to locate the hardcoded offsets TwbT needs. I'm not really sure what to do here, since I haven't seen documentation regarding what those offsets actually are.

[PeridexisErrant]

Would merging this make it possible in the future to use TwbT without breaking other DFHack UI overlays?

No, but adding these structures would make it easier to keep TwbT up to date and perhaps include a standard version in DFHack.

DFHack/dfhack#746 will avoid broken overlays, but there's a number of other plugins that need to change before that all shakes out.

[lethosor]

If you ask me, it would make it harder to keep twbt up to date, since

• we don't know what those offsets are or how to represent them in df-structures - are they integers (if so, what size)? pointers to integers? pointers to more complex structures? random addresses in memory? addresses of executable code? If it's the latter, we don't have a good system in place that I know of for either defining those in structures or exposing ones defined in symbols.xml to C++.
• It would add more work to a release. We've left out debug flags, save_on_exit, and startdwarfcount in some cases, so unless we can find a way to locate all of TwbT's offsets automatically, TwbT will either be broken or we'll have to wait for someone to locate them manually (and make sure they're correct, which has been a problem in the past)

Quietust and Expwnent also pointed out several issues in TwbT that could prevent us from including it - I'm not sure how many of them have been fixed since then, but some are still present. Here's an incomplete list:

• It can't be unloaded. Trying to unload it leaves it in a "broken" state, which means none of its commands work but the renderer remains active.
• There are numerous magic numbers for things that are well-defined in df-structures. Some of them were incorrect at one point but have been fixed (e.g. the adventure mode ui menu), and the magic numbers could be outdated in 0.42.
• Some advertised features are broken in unspecified ways (mapshot)
• One offset is not required on Linux but "may cause problems". Apparently the solution requires recompiling libgraphics, which we can't rely on because many DFHack users use their own libgraphics (by choice or through their distro's package manager). It's also unclear exactly what the problem is without this offset, although it's apparently not a crash.
• It breaks other DFHack tools (there has been progress made on this one)
• There is some debug information logged that's irrelevant for users (pretty easy to fix, and most of it has been)

I also haven't confirmed that DFHack/dfhack#746 will avoid broken overlays. In particular, I have no idea why scripts that just draw to the sidebar cause twbt's map display to break. Hopefully that can be figured out once DFHack is stable enough, and we can get rid of the separately-maintained plugins as well.

[fricy]

re:

I'm not really sure what to do here, since I haven't seen documentation regarding what those offsets actually are.

Do you mean this: https://github.com/mifki/df-twbt/blob/master/PATCHES.md

[lethosor]

That just tells you how to find seven offsets of things in memory. I have no idea what those things are, how to describe them in df-structures, or whether they're pointers to the data or executable segments (or both). A couple appear to be function pointers, which we don't have a way of describing (and there are some issues with including function pointers that are a little unclear to me, anyway).

[warmist]

IMHO adding them could be done as generic "offsets" at first (or forever?). As for what they are:

• A_LOAD_MULTI_PDIM function - load image into df graphics
• A_RENDER_MAP function - render map
• A_RENDER_UPDOWN function - render up/down view in adventure mode

The rest i don't know, but i think they are specific offsets for various binary patches.