Find out what people need most from DFHack

[myk002]

After basic reverse engineering to re-establish our hooks I expect we'll need to validate all DFHack functionality for the steam release. Covering everything before we do a release is a huge task. It would be very useful if we knew what users need most so we could prioritize accordingly. Then we can do smaller incremental releases.

I'm thinking about creating a survey the we can post on Reddit, the forums, and anywhere else DF players congregate. The survey can be hosted on Google forms, and the results can get automatically written to a spreadsheet for analysis.

The big question is, then, what questions should we ask? We have a practical limit to how many questions we ask because if we ask too many, then people won't finish the survey. Maybe limit ourselves to 5 questions?

The idea is to discover what classes of users are using DFHack, where a class is defined by the set of DFHack tools they are using. The remaining questions should focus on discovering the needs of the users in that class.

[myk002]

Potential questions:

• which DFHack services/tools do you use? (checkboxes for all plugins and scripts, maybe broken down by category, maybe bundled into groups)
• what is the one command/screen/service/tool from DFHack that you rely on most?
• what do you wish DFHack would do better?

That first question depends on a taxonomy for DFHack scripts/plugins. I'll see if I can come up with something. If it's good enough, we might be able to reuse the taxonomy to restructure our documentation.

[Tachytaenius]

"Do you find it easy to get into script/plugin development?"

[cppcooper]

I'd just dig into what you want to know, which is how users are using dfhack.

1. What describes best your purpose for using dfhack? cheating, management, ui improvements, automation (we want to be able to visualize how users are distributed along a wide range of uses and intents, so add/remove to the list as needed)
2. What plugins do you appreciate or use the most?
3. Which 5 (or less) scripts do you need the most?

I think it boils down to just a few general questions. If we're talking a survey, we'll want to aggregate the data so we can see what things stand out. And for what its worth, I think automation and management of dwarves(stats) and animals are the most useful aspects of dfhack.

[lethosor]

That first question depends on a taxonomy for DFHack scripts/plugins. I'll see if I can come up with something. If it's good enough, we might be able to reuse the taxonomy to restructure our documentation.

"Tools" is how we've summarized it in the project description at https://github.com/DFHack/dfhack for a while.

[myk002]

"tools" it is, then.

Taxonomy spreadsheet here: https://docs.google.com/spreadsheets/d/1hiDlo8M_bB_1jE-5HRs2RrrA_VZ4cRu9VXaTctX_nwk

I didn't categorize modtools/ or devel/ since they're (ostensibly) not user-facing. They can get documented as categories of their own.

The next step is to create the survey, using the new taxonomy to name groups of tools so we have only a few checkboxes instead of 350+

[myk002]

I'd just dig into what you want to know, which is how users are using dfhack.

1. What describes best your purpose for using dfhack? cheating, management, ui improvements, automation

I think you're right. We don't need to list every tool, and even the category names I came up with aren't very useful for this (though I hope they'll still be useful for improving our documentation)

[myk002]

Survey available here: https://docs.google.com/forms/d/e/1FAIpQLSfXMmRZNi1Fyb5fOsa2W5RC_MTeu9IGucC-PRDLzkn_0r51Lw/viewform?usp=sf_link

[cppcooper]

I'd just dig into what you want to know, which is how users are using dfhack.

1. What describes best your purpose for using dfhack? cheating, management, ui improvements, automation

I think you're right. We don't need to list every tool, and even the category names I came up with aren't very useful for this (though I hope they'll still be useful for improving our documentation)

Well, here is a list I just made up. I think it covers much of the use cases I see dfhack as being for.

• cheat
• quality of life
• automate and manage
• simplify the mundane
• bug fixes
• ui improvements
• new mechanics
• information gathering/analysis

[myk002]

@cppcooper could you try taking the survey to see if the organization of the questions works? Feel free to submit real answers. It all goes to a spreadsheet

[cppcooper]

@cppcooper could you try taking the survey

I had difficulty answering the "I use DFHack to:". I found the section a bit too complicated. I'd look at the idea of the response eg. "Add new mechanics" and then the scripts/plugins it lists and I'd immediately be in conflict. (for me it was with: automate, information for decisions , manage stock, change x/y/z, recover) So I'd look at these and think, oh yea that's me.. Then the scripts and think, ok maybe not.. I wonder what scripts they(myk) considers part of this use.

So in that singular way, the responses feel like they're enforcing a Use to Script mapping. Which maybe is what we want, but it gets tricky for the user trying to choose responses if you're not sure what kind of script DFHack considers your particular script unless explicitly listed.

In terms of designing a survey, we want to make responses easy and natural to choose. So all the mentally taxing stuff should be on the surveyor side of things. Otherwise, we might end up in a situation where the responder part way through says to themselves you know what it doesn't really matter and just leaves or gives half baked responses. In other words, the users should only need to think about how they use dfhack and match that with simple responses, or add their response as "other" with a text box.

In short, we should divorce the responses from their related scripts/plugins [or do the opposite and accept scripts/plugins as responses].. Our asking about which parts they use most allows us the chance to cross reference their idea about how they use dfhack with actual scripts/plugins.

In terms of designing a survey, we want to make responses easy and natural to choose.

To go back for a quick second, even the questions should be short and sweet. So for Q4 I'd write:

What do you wish DFHack would do better? (eg. documentation, script X should also do Y, integrate more things with the GUI, etc.)

Shorten the examples, group them together and make them second-class compared to the question.

As someone with a bit of an information processing problem, simple stuff like this matter more than we can immediately realize.

I feel like a such a hypocrite hah.. Here I am espousing short, simple, and quick response options.. and I've written a huge response about responses.

[myk002]

I feel like a such a hypocrite hah.. Here I am espousing short, simple, and quick response options.. and I've written a huge response about responses.

You should see some of the posts in the quickfort thread sometime... ldog and I got in pretty deep.

Good feedback. Thank you! I updated the survey. Could you take another look?

[cppcooper]

I feel like a such a hypocrite hah.. Here I am espousing short, simple, and quick response options.. and I've written a huge response about responses.

You should see some of the posts in the quickfort thread sometime... ldog and I got in pretty deep.

Good feedback. Thank you! I updated the survey. Could you take another look?

Far less stuff to complicate things for the reader. I think it's good. To be better I think we gotta nail down the breakdown of uses and their order.

• Get information to make good decisions
• Get the UI improvements
• Get bug fixes
• Manually trigger game events
• Manage stock levels
• Quickly perform common tasks
• Design fortresses
• Automate fortress management
• Change the map/dwarves/resources to what I want (??? tilesets?)
• Recover from bad planning / unwinnable situations
• Add new game mechanics
• Other: [textbox]

This is an order that I think makes sense. Least changing of vanilla to most, sort of, was what I was going for. I added the other option, which I think would be useful just in case we neglected some ideology. The change the map/dwarves/resources needs clarification (I don't even remember the scripts you had for it) Though I do recall superdwarf being among them, so perhaps this can be broken into a couple options. or are they all about Modifying game data?

ldog and I got in pretty deep.

What is LDOG?

[myk002]

What is LDOG?

see, for example, http://www.bay12forums.com/smf/index.php?topic=176889.345

To be better I think we gotta nail down the breakdown of uses and their order.

I was actually planning on randomizing the order to avoid "select the first option" bias

I'll add the Other option.

"Change the map/dwarves/resources to what I want" means "edit stuff to casually cheat, like to make the game easier overall", but I'm being careful not to use the word "cheat" since it can be seen as judgmental. I do want to differentiate this type of cheating from the "cheat myself out bad planning/unwinnable situation" option, though

[cppcooper]

edit stuff to casually cheat,

We should use the word edit, I think (maybe modify). Change is pretty close to swap-out or replace. So I think the difference might actually help prevent confusion about whether it might have anything to do with tilesets.

[myk002]

Good point. Changed to "modify"

[myk002]

Draft post text:

Help DFHack get ready for the Dwarf Fortress Steam release! Tell us which tools are important to you so we can get those ready first!

[myk002]

The reddit survey post is live! https://www.reddit.com/r/dwarffortress/comments/vhr2c3/help_dfhack_get_ready_for_the_dwarf_fortress/?utm_source=share&utm_medium=web2x&context=3

[myk002]

Forum post is live here: http://www.bay12forums.com/smf/index.php?topic=180020.new#new

Post text:

When a new DF release comes out, DFHack devs need to spend time adapting to code and behavior changes. This means that we have to go through all our tools and make sure they still work! We need your help figuring out which are the most important DFHack tools to look at first when the Steam release comes out.

If you can, please help us by writing down the DFHack tools you depend on most in this (very short) survey: https://docs.google.com/forms/d/e/1FAIpQLSfXMmRZNi1Fyb5fOsa2W5RC_MTeu9IGucC-PRDLzkn_0r51Lw/viewform

This is the same survey we posted to Reddit, so if you've already submitted something from there, thank you! and no need to submit again from here.

Feel free to comment in this thread, but please fill out the survey form too! Those are the responses that we're looking at.

[myk002]

The "more than 10 explicit mentions club" (as of 2022-06-23):

[myk002]

Here is the report:

I (myk002) am one of the DFHack core developers. I created this survey in order to understand what DFHack's users appreciate and need most so we can set development priorities for the future. This survey allowed me to understand how strongly our users feel about particular pain points (spoiler: our users have a lot to say about GUI integration and documentation!).

Overall, there were 470 people who submitted responses. This is more than enough to get some real, statistically significant results. Thank you all for your help! It really means a lot to us!

Q1: "I Play:"

470 responses.

Q1 analysis and opinion

Almost everybody who uses DFHack plays at least fort mode. About a quarter of respondents also play adventure mode. A small percentage play with mods, and most are small personal mods the players themselves have created. Only Squamous's mods (The Long Night, Fall from Grace, etc.) and Masterwork have more than one respondent playing them.

From my point of view as a DFHack developer, this shows that tools that are useful in fort mode will have the most impact. However, adventure mode has a significant presence. Tools specifically for adventure mode will still be appreciated and used.

I note that mods do not have a huge presence right now. This could be because mods and tilesets -- especially the newer, more complete tilesets -- don't play well together, and people would rather keep their tileset than explore a mod that requires them to play in text mode. However, this all might change drastically once DF supports Steam Workshop and mods can be distributed via Steam. Watch this space!

Q2: "I use DFHack to:"

468 responses.

Q2 analysis and opinion

The top categories all have to do with productivity, toil reduction, and access to information. This is where people are most keenly aware of DFHack's benefits.

These categories are closely followed by bug fixing. Bugfixing happens in the background and is mostly invisible. I was surprised that there were this many people who were aware enough of the bugfixes that they selected this option, though I suspect if people knew the extent of DFHack's bugfixing, the number would be even higher.

Every aspect of DFHack has a significant number of users that care about it, though.

Q3: "What are the DFHack tools that you rely on most?"

306 responses.

There are more than 350 tools in DFHack (including individual bugfixes and tweaks), but only the top 20 are called out here.

Note that these are plugin/script names, not command names. For example, many people wrote digv as the tool they use, but it was consolidated in this graph with the other dig plugin commands. Also, many people seemed uncertain about what to call the manipulator plugin. Many just called it "Dwarf Therapist" or "that DT screen" or "labor spreadsheet". These were all consolidated into the manipulator category, even the ones that were clearly referring the the real external (non-DFHack) tool named Dwarf Therapist, with the rationale being that "a tool that does labor assignment is important to our users".

Q3 analysis and opinion

This is a poll of perception, not actual records of usage, so we must remember that these are not necessarily the most valuable tools, but rather the most valued tools. All the tools listed here are definitely high-value, but the list is not guaranteed to be exhaustive. We will be using the rankings here to identify the "marquee" tools, and we'll combine this list with the rankings in Q2 above to prioritize our work in getting tools ready for the Steam release.

That being said, let's map those tools back to the categories in Q2 and see what we get (note that many tools map to more than one category):

• Get the UI improvements: (search)
• Quickly perform common tasks: (autodump, clean, manipulator, stocks)
• Automate toilsome fortress management: (autobutcher, autofarm, autolabor, autonestbox)
• Get information to make good decisions: (embark-assistant, manipulator, reveal, prospect, showmood, stocks, exportlegends)
• Fix game bugs so they don't bother me: ()
• Manage stock levels: (workflow)
• Design the fortress: (dig, quickfort)
• Modify the map/dwarves/resources: (autodump, clean, createitem, fastdwarf, liquids)
• Add new game mechanics: ()
• Recover from bad planning / unwinnable situations: (createitem)
• Trigger game events: ()

UI improvements and bug fixes are important to users, but I'm not surprised at all that they don't get named (other than search). They are not usually tools that users run directly, so their names are not going to be at the forefront of anyone's minds.

The rest of the tools seem reasonably distributed based on the Q2 orderings, which validates both what users say they value and what tools they actually value.

I am a tad surprised that the smaller productivity tools like unsuspend and unforbid didn't make the list. I know I use those all the time!

Q4: "What do you wish DFHack would do better?"

176 responses. This is the "user pain points" section. The responses here indicate where DFHack users are feeling frustration.

Q4 analysis and opinion

The results could not be more clear. DFHack has a lot of great, powerful features, but users are frustrated with not being able to find them and not being able to use them effectively. There is a large call to make DFHack more accessible from the Dwarf Fortress game screen (the GUI). DFHack should be concentrating developer effort to fix these usability issues. Ideas from the community are welcome here!

Q5: "If you could write your own DFHack feature, what would it be?"

161 responses. Here are the areas where users think DFHack could make an impact if we were to develop a new feature. I made up tool names to combine similar ideas from multiple users, which I will explain below. Only ideas that appeared more than once are included in the charts below.

Now, you may recognize some of the "feature requests" in that chart. Many users were just not aware that DFHack already includes the tool they were requesting. In fact, about 1 in 3 feature requests were for features or tools that already exist in DFHack. Another vote for the "Discoverability" pain point.

Here is the chart with only requests for features that don't already exist in DFHack:

Again, these are made-up names that I used to represent the ideas I was combining from the responses. Here's what they mean:

• military: manage uniforms, squads, and schedules, including importing and exporting data
• force siege: generate a siege
• impregnate: similar to catsplosion (which can actually handle any animal type) but for sapient beings, like your dwarves, and able to target specific individuals
• track-designer: tool for laying down tracks and routes and checking them for breaks
• fix/missingsquad: ensure your squad that went out on a raid comes back (or at least is declared MIA so squad leaders can be reassigned and another squad formed)
• assign-pets: assign pets specific owners, or getting dwarves adopted by specific cats
• max-wave: control how many migrants can come in a migration wave to prevent your fort from growing too fast
• fortminder: advisor that warns you when things are in a bad state and recommends improvements
• gui/assistant: centralized DFHack configuration GUI where you can, for example, select the auto* tools you want enabled
• filter-reports: find important information in the announcement reports
• religion: manage religions in your fort (or in the world), including raising a historical figure to deity status

Q5 analysis and opinion

These requests indicate the areas where players feel they don't have enough control or information. Military-related tools were the number one requested feature, but all of these are good suggestions and we'll take them under consideration. A number of these top requests are being worked on right now, such as filter-reports and max-wave. gui/assistant is an excellent suggestion for improving the usability of DFHack and will be given high priority, given the pain points our users made clear in Q4!

Developers out there: if you want to make a contribution to DFHack but don't know what to write, that is the list of the tools that will be most appreciated by the community!

In an effort to aid discovery, here are the most popular "feature requests" for tools that DFHack already has, in order of popularity, along with brief descriptions and links to each existing tool's documentation:

• quickfort: applies digging, building, stockpile, and zone designations to a map according to a blueprint file. It also has modes that allow you to play back keystroke sequences with the cursor at specific spots on the map. It comes with a library of ready-to-apply blueprints that you can use to design your fortress. Here is an example blueprint for digging, furnishing, and configuring a library and a temple (along with a bunch of other rooms that are prepared for assignment as temples and guildhalls).
• dwarfmonitor: measure your fort's efficiency and add widgets to the screen that display live-updated information.
• buildingplan: place buildings and constructions even if you don't have the materials ready yet. They will get built when the construction materials become available. Very useful for planning out large walls or megaprojects since you can designate constructions in places that aren't reachable yet. Works well with automaterial for designating constructions over large areas. quickfort also uses buildingplan to place buildings and constructions from blueprints.
• automaterial: Moves the last selected material to the top of the list when building more constructions of the same type. Also allows designating very large, optionally hollow, rectangles for construction.
• orders: import and export lists of manager orders. DFHack comes with a library of ready-to-use manager orders that you can import.
• manipulator: Easily manage labors across the dwarves in your fort. Supports saving and applying groups of labors as custom professions. DFHack comes with a library of useful professions that you can apply to your dwarves.
• strangemood: triggers a strange mood. you can optionally choose the dwarf, type of mood, and which skill is used to create the artifact.
• changevein: changes the vein under the cursor to the specified material.
• exportlegends: export legends mode data for viewing in external utilities.
• drain-aquifer: converts all aquifer tiles on the map to regular soil/stone.
• seedwatch: controls whether each seed type can be cooked based on whether you have enough of that seed to support farming.
• diggingInvaders: allows siegers to dig and destroy constructions to get to your dwarves.
• prioritize: watches job creation and ensures jobs of specified types get done quickly. Use this, for example, to ensure your haulers haul your food before it rots.
• gui/gm-unit: edit unit attributes
• mousequery: add mouse controls to the DF interface. Hover your mouse cursor over a tile to get information on what that tile contains. Click on a building or stockpile to get to its settings menu.
• blueprint: record a copy of your fortress in a quickfort blueprint.
• gui/dfstatus: shows a quick overview of critical stock quantities.
• forum-dwarves: export the text on the screen to a file.
• createitem: spawns in an item of your choice. use createitem inspect to discover the properties of an existing item you want to copy.
• gui/settings-manager: GUI interface to customizing the Dwarf Fortress data/init/init.txt and data/init/d_init.txt files.
• prospect: discover mineral content of an embark region
• gui/pathable: displays the tiles that are reachable from the cursor by a walking unit
• autolabor: automatically set labors on your dwarves according to the jobs that need to be done
• gui/gm-editor: directly edit Dwarf Fortress data structures
• autochop: automatically manage tree cutting designations so you always have enough wood in stock
• spawnunit: create a new unit on the map
• autodump: destroys all items selected for dumping or moves them to the cursor location
• zone: assigns animals to pastures
• gui/room-list: lists all rooms owned by a dwarf
• gui/mod-manager: installs/uninstalls DF mods
• tweak makeown: add visitors or merchants to your fort
• fastdwarf: make dwarves magically fast

[lethosor]

I might add to your list at the end: catsplosion can already handle non-cats. Not quite sure if it handles dwarves, and I'm pretty sure it doesn't handle individuals, so there's still some room for new features there.

[myk002]

Thanks! I can add that info to the impregnate feature request explanation. Yeah, catsplosion handles any animal type, but not dwarves (and also not specific units)

[myk002]

Report has been posted to:

• reddit
• DF forums
• DFHack forum thread