[buildingplanner] construction issues

[fricy]

There's something off with how buildingplan attaches materials to construction jobs. I noticed in other forts that floors won't get completed if I designate them with the planner, even if I manually select some available materials. But if I cancel them, and use the vanilla interface they get completed in short order.

In this fort I embarked to build a bridge to an island, and efficiency ground to a halt shortly, because it looks like it's always the block manufactured last that gets selected by the logic. It doesn't matter if I have a stockpile nearby, my dwarves rather take a half map trip to get the block from the workshop instead of grabbing the one right next to them.

I tried foring the issue with a burrow, but that doesn't help either, they are just standing around ilde.

Save attached with designations created through building planner after creating the burrow. The burrowed dwarves won't pick up the job until pausing the burrow, or cancelling the floors/walls and recreating them through vanilla.

I think there are two issues at play here: First the building planner doesn't respect the burrow for item selection. The root cause however is with the item selection logic. I'm not sure I interpret it correctly, but it seems like it's trying to attach the item created last? I didn't notice these hangs with furniture, only with floors and walls. Probably because there are more construction jobs, and this issue happens under specific conditons.

[DFHack version 50.08-r1 (release) on x86_64]

[fricy]

Another issue: Buildingplan populates the designation matrix starting form the upper left corner regardless of pathing.

This has several efficiency implications:

Construction won't start until there's a path to an unbuilt tile with an item attached. The blocks attached will be claimed for construction, and the player will be unable to stockpile them closer to the build site.

The current algorithm seems to operate good enough for furnitures and basic wall and floor jobs, but struggles with (multi-floor) megaconstructions and multi-item machinery. My impression is that item selection defaults to the newest item produced, and that results in unnecessary long and often dangerous trips to the origin of the item, like trips to the surface or the caverns for the newest chopped down log, instead of grabbing one from the stockpiles. Even when it's not dangerous, it's horribly inefficient, and it's very hard to force item selection in a sane manner despite using filters. One workaround is to build the workshops on site, and transport the raw materials instead, but that's not always feasible due to size constrains.

Necessary fixes to buildingplan: Respect burrows for item selection. Check for pathing/connectivity, and start on the corner that's closest to a pathable tile.

Further suggestions to implement: Don't attach construction materials in one go. Check connectivity every few ticks, and only attach if there's a valid path available.

Reconsider item selction for bulk (log, blocks) items. Newest item logic unfortunately has too many drawbacks here. I'm not sure why closest material selection was not used. (Z shenanigens? shaving CPU cycles?) Maybe a limited check for materials in a 3x3 or 7x7 radius would help a lot when rebuilding stuff. Right now what happens is a long trip to grab the newest material, while the previous materials remain on the ground where deconstructed.

Higher levels need a tile to stand on. Prioritize constructing bottom walls first.

[fricy]

Another issue that happens:

You designate some walls for building in a tight spot. For example multiple walling project in the caverns without good scaffolding. Yu don't have all the blocks on hand at the beginning, so some walls don't have a material attached yet, the planner fills these as the blocks get made.

Then there's an error, and a dwarf walls itself off. No problem, you deconstruct the blocking wall, let the dwarf out, designate a new wall on the same spot, and send it to the top of the queue.

Except it won't get build. Turns out, the block immedietly got attached to another wall somewhere. Until that wall get's built it's claimed for hauling, and thus unmovable. Any dwarf trying to build on that spot will cancel the job. Rinse, repeat.

This issue can also be fixed by:

Don't attach construction materials in one go. Check connectivity every few ticks, and only attach if there's a valid path available.

[myk002]

thoughts:

it looks like it's always the block manufactured last that gets selected by the logic

This is true. The idea was that free items are more likely to be found at the end of the list rather than the beginning. However, I can see how that can cause issues like what you describe in the case where there are many items available. I'll have to measure the performance impact, but I suspect changing the order of iteration will be fine. We're scanning through the whole list anyway when there are no matches (which is often).

n.b. we don't do "closest item" matching since that turns an O(N) problem into an O(N^2) problem. We can't afford to pay that kind of processing cost in a background task.

doesn't respect the burrow for item selection

burrows affect units, not jobs. Units in burrows may not take a job that is outside of their burrow (if the option for that is set), but I don't think this has anything to do with where the items attached to those jobs are.

Moreover, how is buildingplan supposed to decide whether to restrict item selection to a burrow? Just being within the bounds of a burrow isn't very specific. An "Inside" burrow may encompass the whole fort. A unit's personal burrow may be very small. How would buildingplan be able to decide what is relevant when a job happens to be within bounds of both burrows?

Buildingplan populates the designation matrix starting form the upper left corner regardless of pathing

This could possibly be improved, at least with a heuristic. Bridges are very tricky because the place where a unit stands to build it can be very far away from the building center. one-tile constructions, however, have relatively easy-to-calculate pathability requirements -- if any orthogonally-adjacent tile is pathable, then the construction is likely buildable.

However, this heuristic breaks down once you expand over an un-pathable expanse linked only by planned floors. When the next buildingplan scan cycle comes, there could be zero adjacent tiles that are pathable, so items that could be attached would not, and the global ordering of the item queues would get messy. Anyway, we don't want to just attach items to pathable jobs since that slows down the rate of construction. If we only attach an item to a pathable job, then that means that floors can only be built one tile per half day (buildingplan's periodicity).

Perhaps as a compromise, buildingplan can respect the order that the player indicates by drawing the rectangle. rectangles can be filled starting from the first clicked tile. Would that be sufficient?