Maslow's Heirchial Finite State Machine

[kevingranade]

In order for NPC tactical AI to act sensibly, NPCs must have sensible global goal-setting oriented around at least survival.

I'm thinking a Behavior Tree loosely modeled on Maslow's heirarchy of needs, like.

1. Homeostasis, Wound care, Water, Food, Sleep, Shelter
2. Personal security, Emotional security, Financial security, Health and well-being, Safety needs

For the initial cut, these are it, NPCs don't need to start pondering self actualization or personal fulfilment just yet.

I think shelter and related stuff is going to have a lot of feature creep because NPCs don't know what a shelter IS, but I also don't think we can do much of anything sensible without them having one. Can probably do something like whitelist a bunch of terrain types and call it a day for now, even though that will lead to NPCs setting up camp in burned down buildings.

[I-am-Erk]

Seems like a reasonable start point.

[kevingranade]

In progress but slow due to life and merging other people's stuff. I've switched to a behavior tree for top level goal decisions. I've narrowed the initial scope to just dealing with homeostasis, thirst and food. Basically let's get the NPC AI to the level of a feral animal XD

The output of this system will be selecting an immediate goal, i.e. "find clothes", "find water", "find food". V1 will probably just pick one of those, later versions can use heuristics to decide how to go about it, such as scavenging in town locations vs acquiring natural resources.

[kevingranade]

This issue has been mentioned on Cataclysm: Dark Days Ahead. There might be relevant details there:

https://discourse.cataclysmdda.org/t/creating-a-needs-framework-for-npcs-and-maybe-even-pcs/26620/5

[I-am-Erk]

We should write this out more.

[I-am-Erk]

So we have:

Top priority - physiological needs

If these are not met, the NPC will do whatever it can to meet them. When they become critically low, NPCs may become desperate and potentially violent seeking to meet these needs (eg threatening the player if the player won't let them share their shelter). I have prioritized these in terms of urgency of need, although the last three would depend a bit on circumstances.

1. air
2. water
3. food
4. clothing
5. simple shelter/hiding place
6. sleep

High priority - security needs

NPCs are less likely to become violent if these aren't immediately met, but they will seek these with high urgency as soon as they have the above.

7. weapons/armour
8. safe shelter/home
9. renewable light/heat sources
10. first aid supplies
11. stockpile of food and water and/or renewable food/water sources
12. building/repair materials

Medium priority - emotional needs

NPCs are unlikely to fight someone for these, but when the above has been filled they will start looking for this stuff. When we get a better npc morale system, these things could impact that.

• entertainment
• a pet or another faction member
• contact with a survivor faction
• a journal or survivor notes

[ActualDio]

The least priority needs would need a personality framework of some kind to determine what a character would want in a companion or its mental needs and quirks. I dont know how translatable it is but I think the Rimworld Mod Psychology https://rimworldbase.com/psychology-mod/ could be a decent template on what to do. Although it is pretty in depth so I dont know how much we should take from it lol.

At the moment I think the main short term objective should be getting the priority system working, linking it to the movement and action ai, and then possibly creating different levels of desperation to lead to different kinds of actions

[I-am-Erk]

Kevin has written on discourse:

Where I stalled on this was the difficulty in representing varying degrees of needs mostly within a single level. So if you are just about to starve to death but you were a little bit thirsty you need to prioritize food over water even though normally in a behavior tree you would strictly prioritize a thirst need over a hunger need.

I have two ideas for what to do about it. One is add a simple “largest value” selector to the behavior tree implementation, and have the potentially lethal needs return a, “time until death if need is unmet” value, and the second is to add a utility curve selector and have the needs return a simple level value.

What I would suggest is that - and this will shock everyone - we associate each need with a json object, even if it's just a stump of one. That will let us define anywhere from some things to maybe everything. We could even possibly have scavenger-related needs state what items or categories of item will satisfy them, although that portion might be fine hard-coded. Somewhere in the object we'd include values explaining weighting, so the npc would know when to deprioritize hoarding food and now focus more on reinforcing their base. By making it JSON, we can potentially adjust these on the fly pretty easily, and it is more easy to imagine ways to have different NPCs prioritize things differently. For example, perhaps we could have multiple needs objects for entertainment with different priorities on different items.

[ActualDio]

What I would suggest is that - and this will shock everyone - we associate each need with a json object,

I like this idea a lot, lets us be more modular with it so everyone can easily change and fine tune it

I think for starters going with what @kevingranade was already working on is best: Food, Thirst and Temperature as the most basic and most important needs

I could try making quick change in the JSON to add these needs, but I would like to see what yall think are the best things to put into a template need? I can think of a priority multiplier, so we can scale the priority for different needs and maybe a desperation meter but what else?

[ActualDio]

{ "id":"thirst", "type":"PHYSICAL", "initial_reaction": "A kind of action", "worried_reaction": "A different kind of action", "desperate_reaction": "Desperate last resort action" }

This is what I got so far what do yall think?

[actual-nh]

Do you have in mind what will satisfy this need being hardcoded, or also in JSON?

[actual-nh]

4. clothing
5. simple shelter/hiding place

The need for warmth - possibly also satisfied by a fire - will be related to the above in the correct season (or mod - Aftershock). (Not being soaking wet, or at least having clothing to protect from it being problematic, is also related.)

[I-am-Erk]

The reaction to the need is less what I am thinking of in JSON. For a majority of these, the reaction is going to be "go out and loot (itemtype)"; we'll probably need some hardcoded options for more abstract things like 'reinforce your base' or 'start a farm'.

I mean that the need entry should discuss what items satisfy the need and how much weight they add to satisfaction, to help determine which need is currently at the top of the hierarchy. So for example a need like "entertainment" would have a list of items or item categories that count as entertainment, with associated weights and some decay information (and maybe caps to make sure something like entertainment never becomes so critically rated that an npc goes hunting for reading material while starving). When the NPC gets entertainment items, the current state of that need is then bumped down based on how much that NPC values that entertainment. Aside from the need for water and food based on hunger and thirst, the same mechanism can apply to stockpiling food etc.

[actual-nh]

The reaction to the need is less what I am thinking of in JSON. For a majority of these, the reaction is going to be "go out and loot (itemtype)"; we'll probably need some hardcoded options for more abstract things like 'reinforce your base' or 'start a farm'.

Right. Dialogues prompted by needs may be at least partial exceptions.

[I-am-Erk]

I think what we first need is a machine that drives NPC behaviour on a basic level. They go out, get a base, get food, bring it to base, etc.

once we have that we can start on the more advanced behaviour like dialogue with players and expanding their own bases.

[ActualDio]

Could we define a base as something with a roof on top? As a very basic shelter?

[ActualDio]

Was looking through the src\npc.h file and I found that although it has declarations for item behaviour none of these have definitions in the rest of the code?

// Look around and pick an item void find_item(); // Move to, or grab, our targeted item void pick_up_item(); // Drop wgt and vol, including all items with less value than min_val void drop_items( const units::mass &drop_weight, const units::volume &drop_volume, int min_val = 0 );

Theres also some function definitions and methods relating to npc needs already in the src\npc.cpp but I dont know what they do exactly.

Edit: Nevermind VSC was just being a weirdo, I found the function defs.

[I-am-Erk]

We actually already have flags for maps that make good bases. It's stored in the mapgen data as a "SAFE_PLACE" flag or something along those lines.

[ActualDio]

One thing I have noticed while figuring how to have the NPCs look for items to satiate needs is there seems to be no way for them to prioritize droping an item in order to make space for another. The find_item() function simply returns empty if the NPC doesnt have space for an item. Is there some other mechanism in place already for this that Im just not seeing?

[zachary-kaelan]

I say the top level needs should be roughly weighted and compared based on how long it takes to die without each.

For the sake of example, say:

• It takes 3 minutes to die in smoke
• 72 hours to die without water
• 27 days to die without food
• 90 days to lose the will to live without emotional comfort
• Other needs are ignored

If the NPC has gone 24 hours since their last drink, they should be willing to venture into the smoke for 1 minute to get some water. If it's been 4 hours then they're willing to venture in for 10 seconds.

For the higher level needs it could be more arbitrary, but based roughly on an idea of how long without that need they'd lose the will to live. Say there's a teddy bear that's a 6 hour round trip to grab. If the NPC is completely full on their thirst and hunger needs, ( ( 6 / 72 ) + ( 6 / ( 24 * 27 ) ) ) * 90 = 8.33, so 8.33 days is when the NPC would be willing to make the trek to grab it. If the NPC is thirsty and hungry then it'll be far longer. If the NPC has extra food and water stored then it'll be far shorter.

There is a point where an NPC could be starving, thirsty, and cold but still prioritize getting a teddy bear, and that's how things should be. Emergent behavior that appears irrational unless you know the full story.

This could easily extend to addicts engaging in risk-taking behavior in search of more drugs and things like that.

[kevingranade]

I say the top level needs should be roughly weighted and compared based on how long it takes to die without each.

That's exactly where I got to eventually but didn't note it here. Implementation wise behavior trees need an evaluation node backed by a method that literally returns "how long until I die of <starvation/thirst/exposure>". This could either be a method that takes multiple enums for the different needs or one method for each need. A nice outcome of this is that if a NPC is dying of thirst soon, they might do the quickest thing to relieve thirst, which might be going to a river and drinking straight from it, which will push the death time back, possibly past some other need, then once that is satisfied, thirst might be at the top of the list again, but to push it back further they will realize that they need to acquire liquid containers or a water source in order to push that time back further (the system wouldn't be evaluating just biological thirst but also held inventory and possibly inventory in a residence).

Anyway, the top level "immediate needs" node would invoke the evaluation methods for each "need" and then act on whichever will kill the NPC soonest. Once shelter, water, food, etc are sorted out to some threshold, say a day just for the sake of argument, evaluation would proceed to the next level, which would focus on securing things that would satisfy those needs for even longer. This could be based on again either invoking one master method to evaluate needs, but with flags to indicate different scales, or a number of different methods with those flags baked in. i.e. character::time_to_die_of_thirst_biological() vs time_to_die( enum need = thirst, enum scope = biological ). The heirarchy (I'm somewhat departing from Maslow's here, so be it) would roughly return to the same needs at each level, but evaluate different scopes, like so: Note time_to_die() would be evaluating "how soon will this kill me if I do nothing about it*, so presence of any hostile monster is pretty much going to kick the NPC over to fight or flight. If there is no monster currently attacking/chasing the NPC, then maybe they can stop to drink water, apply immediate wound care, etc. (I forgot to put wound care in the image, it's non-comprehensive, literally everything that can kill you should be in there eventually), and then maybe proceed to the second "tactical" tier where ok nothing is attacking us now but we're in a "dangerous area" and should proceed accordingly.

If the NPC has gone 24 hours since their last drink, they should be willing to venture into the smoke for 1 minute to get some water. If it's been 4 hours then they're willing to venture in for 10 seconds.

This is an area where behavior trees struggle and part of why I'm targeting this as "goal setting" instead of addressing the whole range of decision making. NPCs should be capable of enduring "risk" of danger while attempting to satisfy their other needs, but making them capable of jumping into a fire to try and grab an item without ALSO making them prone to jumping into fires for nonsensical reasons is a tall order. I think we can tolerate NPCs not being heroes for the first cut.

So for example a need like "entertainment" would have a list of items or item categories that count as entertainment, with associated weights and some decay information (and maybe caps to make sure something like entertainment never becomes so critically rated that an npc goes hunting for reading material while starving).

The behavior tree is just for goal setting, once it goes "your top priority is getting more food", its job is done. We need a placeholder for "what to do to satisfy a given goal", but the plan :tm: is for this to be handled by GOAP (Goal-Oriented Action Planning) which is a much more sophisticated system capable of having the NPC make and execute multi-step plans based on a changing environment. Potentially that placeholder could be a behavior tree, but if it is it very much needs to be separate behavior tree so the things don't start growing into each other. (that's fine building and invoking distinct behavior trees is pretty straightforward). This also helps because the functions we need to support behavior trees are very similar to the functions we need to support GOAP, so building out a simpler but more limited behavior tree might get us the infrastructure we need to successfully build a GOAP graph to cover the same area of behavior.

[zachary-kaelan]

backed by a method that literally returns "how long until I die of <starvation/thirst/exposure>". This could either be a method that takes multiple enums for the different needs or one method for each need

Conveniently allowing us cut down on the number of needs in the enums. Optimally there would be some generic character::time_to_die_biological utility function that could be used by all the needs to cut down on the amount of repeated code.

they might do the quickest thing to relieve thirst, which might be going to a river and drinking straight from it

Incoming desperate_npc_drinking_dirty_water test case.

We just need there to be some conflicting need or desire that says river water isn't as good as clean water. It almost feels like an anti-need, with the anti-need being infections, parasites, and diseases.

I'm somewhat departing from Maslow's here, so be it

He didn't have any evidence for his theories anyway, and nobody has found any.

making them capable of jumping into a fire to try and grab an item without ALSO making them prone to jumping into fires for nonsensical reasons is a tall order

The system will definitely call for a lot of inhumane experiments. But my examples just assumed holding one's breath and excluded all the other needs. I think there should be a "no pain" need or anti-need that's priority 1. Enough pain can override any other need. Jumping into a fire would cause a lot of pain. But of course that kind of thing can wait until GOAP is there to evaluate "this plan is trash".

[kevingranade]

But my examples just assumed holding one's breath and excluded all the other needs.

That's not how interacting with smoke works. In general I don't agree that in a survival situation you should ever run through thick smoke/fire/minefield unless you're forced to by immediate and greater danger, say because you're trapped in a burning building. It's very important to remember that the goal here is to have NPCs act somewhat rationally, trying to model the full range of human decision making is literally impossible, and having a foundational level of danger aversion that might preclude survival due to desperation tactics is an ok outcome.

I think there should be a "no pain" need or anti-need that's priority 1. Enough pain can override any other need.

That's present by having "you're in danger of being killed" be the #1priority in the decision tree, that means none of the other needs are even evaluated.

Incoming desperate_npc_drinking_dirty_water test case. We just need there to be some conflicting need or desire that says river water isn't as good as clean water. It almost feels like an anti-need, with the anti-need being infections, parasites, and diseases.

This is pretty straightforward in a behavior tree, just always list clean water sources first as a method of quenching thirst, and only include drinking water as an option if your thirst is severe.

[zachary-kaelan]

That's not how interacting with smoke works.

I only used smoke because I couldn't think of a better scenario where someone would need to hold their breath to fetch a bottle of water. Though thinking about it, you can't actually hold your breath in the game so it wouldn't count against any air need anyway.

the goal here is to have NPCs act somewhat rationally, trying to model the full range of human decision making is literally impossible

Yes, we only need a system detailed enough to implement whatever hierarchy of needs we develop, obviously working our way from the bottom up, but optimally requiring minimal rewriting to add new layers.

having "you're in danger of being killed" be the #1priority in the decision tree

That just goes without saying, when trying to make agents that don't get killed.

[zachary-kaelan]

I think a lot of elements of danger avoidance and risk taking could be rolled into the A pathfinding. The destination is a goal and cost heuristics are risk assessment. I'm curious if most of the plan-generation process could be done using A.

[kevingranade]

Decision making and risk evaluation are two very different things, and nothing about this topic even has anything to do with tactics. It would stop are the mere presence of danger.

[zachary-kaelan]

What are the steps required to get the needs machine into the game?

[kevingranade]

If you look at... src/behavior.cpp/h src/behavior_oracle.cpp/h src/behavior_strategy.cpp/h src/character_oracle.cpp/h src/monster_oracle.cpp/h src/monmove.cpp tests/behavior_test.cpp data/json/monsters/monster_goals.json data/json/npcs/npc_behavior.json

You can see what already exists. It needs this added, "Implementation wise behavior trees need an evaluation node backed by a method that literally returns "how long until I die of <starvation/thirst/exposure>". It has more traditional evaluation nodes implemented but not that one.

I was also getting behavior trees hooked up for monster AI, as for most monsters that should be sufficient, and provide a framework for building more formal AI infrastructure in case we later want to tackle GOAP for monsters as well.