CleverRaven / Cataclysm-DDA

Cataclysm - Dark Days Ahead. A turn-based survival game set in a post-apocalyptic world.
http://cataclysmdda.org
Other
10.62k stars 4.17k forks source link

Rebuilding focus and xp gain to a simpler (LOL) system. #53372

Open I-am-Erk opened 2 years ago

I-am-Erk commented 2 years ago

Problem

Focus is meant to discourage you from spending too much time grinding skills, and tie your skill gain to various factors that can slow learning. However in practice, it tends to be either trivial and ignorable, or an enormous pain in the ass, and nothing in between. The numbers need tweaking, and in doing so we can make our already cool learning model incredibly robust.

Solution

Buckle up, buttercup, we're going to be going into some deep details here. Interestingly, I don't think this will be that difficult to implement: a lot of the time current systems that use focus should still plug and play into this fine.

Plain talk description

Everything that follows is going to be quite technical, and I was quite excited to make it so it may be a little hard to follow. In the interests of clarity, here is my basic write up of what I'm trying to do.

Focus equilibrium

Currently, we have a stat called "focus equilibrium". This is the value your focus gradually trends towards, left to its own devices. This doesn't work too badly right now, I don't want to change it too much, but I think we'll have to change it to a formula calculation anyway.

Equation

Under this model, negative morale starts having an effect after -5. After that, every 5 points of morale drop lead to a new squared drop, so:

Positive morale is just flat added, so if you're in a great mood you can ignore some pain and fatigue. We could look into some diminishing returns from morale at higher levels, but I think there are enough penalties already, we don't need to nerf the one positive.

Similarly, low pain has minimal effect on focus until it rises:

With fatigue:

Update focus equilibrium every time we change stored focus. See "focus expenditure and recovery" for this.

Special: "morning fog"

When you wake up after >4 hours sleep, we should check if you have any remaining fatigue. Add a percent chance equal to (remaining fatigue/10) to gain a "morning fog" effect of duration 1 hour. This effect drops your focus equilibrium by 10. Taking a stimulant will cancel this effect.

Substances

It's outside this issue, but stimulants should give a flat bonus to equilibrium, and many medications should drop it. In particular, opioids should drop focus equilibrium, but in general this should not be as severe as the pain they treat unless you're using them for minor aches.

Rewrite practice/xp gain totally.

OK, here's where things are going to get hairy, people. Hang on to your seats because we're going to rewrite everything.

First, let's normalize XP gain to something we check each turn. No more complex focus equations to mitigate it, we're going to use an accumulator. In the process, we are also going to change XP gains to two types: instant and pool. Whenever you practice, you gain both of those. Instant is applied to your skill immediately, while pool is stored and added to your skill gradually over a few hours. Let's look at how this works before delving into deeper ramifications

Basic algorithm.

Note: Mind rest: A few of these factors rely on tracking if the character is sleeping or doing a "mind rest" activity. We could either flag this in an activity, or determine it by checking if the activity requires a skill or not. Any activity that requires no skill is "mind rest" - listening to music, playing video games, waiting, walking idly in the forest.

distract_me

This routine runs each turn if the current_focus is higher than focus_equilibrium. It drops focus towards equilibrium. It shouldn't do this too quickly, but in general if your equilibrium is very low you lose focus faster than you'd gain it. We will store some of this focus loss in a temporary buffer, so that if your equilibrium is dropped by a transient effect, you get some of your focus back when the effect is gone.

By this model, if your focus is 100 and equilibrium is very low, like 20, your 1distract_increment` will increase by (focus-equilibrium/10)^2=(80/10)^2= 64 each turn and you'll lose 1 focus roughly every 20 turns as increment reaches 1000. At a lower gap (focus 100, equilibrium 50), you'll lose 1 focus every 40 turns. As the gap gets smaller the time increases, and when your equilibrium is 10 points or fewer below current, you'll only lose 1 focus per 15 minutes or so.

Explanation: The goal of this function is to allow immediate distractions to have consequences on your learning and xp distribution, to keep them from being too harshly penalizing in a model where focus comes back much slower otherwise.

practice_now

This will use a practice_incrementer to determine if we should get instant xp this turn or not. A pool_proportion determines how much of your gained xp goes immediately to practice xp, and how much is saved to gain later. pool_mitigator makes your practice drop based on how much xp you have pooled waiting to distribute from the relevant pool.

pracice_now can drop your focus extremely quickly, if you're learning efficiently. Since focus has less immediate effect on learning this is not a disaster.

Note that we will calculate a variable task_simplicity here.

where:

and:

So:

If practice_rate is 1 or more, go on to:

pool_gain = practice_rate - (current_focus/2), min 0.
practice_gain = practice_rate - pool_gain
pool_incrementer += pool_gain
practice_incrementer += practice_gain

as your focus drops, more of your learning goes to the pool and it takes longer to gain.

Note - integration of knowledge vs. practical: I (or whomever does this) should take a good look at how Character::practice and related functions work. We may want to integrate catchup_modifier and knowledge_modifier from Character::practice directly here, having catchup_modifier increase the amount of xp you gain at this stage if you're training a skill for which you have a higher knowledge level. In fact, most of Character::practice and skill train and such may wind up gutted and replaced with this set of functions.

Expected function: Focus has some immediate impact on your skill gain, but note that the impact is reduced. There is less difference in effect for the upper half of focus, then it drops sharply, and hits more or less a floor. At 0 focus you're gaining 20% of your usual XP, which is penalizing but I hope not crippling. However, at 0 focus, all your XP gains go to pooled xp. You don't gain levels as quickly, and it won't take long for your "pool mitigator" factor to rise up past 1 and start further reducing the xp you gain. This means that at low focus, you can still do a bit of learning in the moment, but it rapidly falls off.

Additionally, because pool_mitigator increases with skill level in a linear fashion, but level xp requirements increase geometrically, at higher levels, your focus becomes more and more important because you are less and less able to pool a meaningful amount of xp.

distribute_xp

If we didn't do any learning this turn, and there's xp stored in pooled_xp, check if we should distribute that pooled xp to the appropriate skill this turn. last_distributed = turn number of the last turn we distributed xp. distribute_interval = 15000/(std::min(focus_equilibrium,current_focus) * (int +per) ) If current_turn-last_distributed <= distribute_interval, reduce pooled_xp by 1 and add that xp to the relevant skill. Decrease current_focus by 0.1, unless player is sleeping or doing some "mind rest" (then no effect on focus). Set last_distributed to this turn.

If there is more than one set of pooled xp, distribute xp from all of the pools at this time. Current focus goes down by 0.1 regardless of how many pools we distributed.

Note that distribute interval uses the more volatile focus equilibrium if it is lower than current focus. Things happening to you in the moment will have a more dramatic effect on the speed your xp pools distribute, so being in pain right now means you are probably not thinking about what you can learn from that sparring match you had this morning. This is important because it allows your present focus in the moment to matter, but in a way that doesn't overly penalize you. You're still going to get the xp, just not right now.

Note also that we should never distribute pooled xp on a turn when we also gained instant xp. It's one or the other, sweet cheeks.

focus_normalize

Your focus should trend towards your equilibrium value. However, the rate at which it does this is going to be much, much slower now. Focus should run on a roughly 24 hour cycle. Short term distractions, measured in distract_me, allow it to drop fairly quickly, but unless your equilibrium stays low, it should quickly bounce back and then trend towards an even keel. Tracking back, recall we had a factor called distract_buffer in distract_me that stores how much focus you've lost from being distracted by minor things.

If you did not run practice_now this turn, we can try to normalize focus. distribute_xp does not prevent focus_normalize from running.

This will use a focus_incrementer just as in the processes above , since this will span multiple turns. When focus_incrementer > 50000, increase focus by 1. When focus_incrementer < 50000, decrease focus by 1. Focus_incrementer may also be best as a global variable to allow adjustment of balance.

Otherwise:

This means that focus_adjust will max out at around 100, before the distract buffer and adjustment for rest is included, causing you to regain around 1 focus per 500 turns (10 minutes or so) at the widest gap. As the gap closes, the recovery rate decreases, and it should take around 20-30 hours to recover all your focus from nothing while otherwise busy, and half that if you're resting your brain somehow. The distract buffer, which represents focus you lost temporarily while distracted, can increase this quite rapidly, but it will dissipate if your focus_equilibrium remains low or if your current_focus is at equilibrium.

Display / UI / UX

This model adds a lot of moving parts, and I don't think it's reasonable for the player to have to track them as numbers, nor very helpful. However, the player does need to know what's going on!

First, I think we should change focus to a ||\-- style bar display. Another option would be words, but I like the standardization of the bar. This bar should aggregate not just current_focus but also pooled_xp, so as your pooled xp rises, your focus appears to drop, just to help players understand that this isn't the best time to start learning. I will post a suggested equation shortly.

Second, we do need some way to show that a skill has pooled XP. I think the easiest way is in the @ menu displaying a + next to skill percentages that are still rising, and if they are highlighted, add a line in the detailed description like "You are still thinking about what you've recently learned about this skill".

Describe alternatives you have considered.

I initially was planning on some similar changes to focus equilibrium, and adding three new focus pools (see additional context below). The idea of pooled xp, which overlaps with but isn't the same as focus, worked better with that concept.

Additional context

These calculations assume we will treat all pooled xp as the same. However ideally I would like to keep three separate total_pooled_xp trackers:

These represent three general types of learning, 'academic' ie. studying and thinking, 'fine detail', ie. paying attention to the little things you're doing, and 'broad attention', ie. noticing a lot of disparate details in your environment. By keeping the three separate we will encourage not doing a single thing ad infinitum to gain xp but instead, diversifying and doing lots of different tasks to maximize learning. Later on we may wish to fine tune what counts as what type of learning.

I think we probably shouldn't track these separate pools on a first implementation. We already need to track skill gain and knowledge gain, that's enough for a first pass. It should be easily to extend the system this way in a later PR, and it's a straight buff so it should be well accepted.

estebandellasilva commented 2 years ago

How exactly will this system interact with levelups?
Because for crafting reaching a certain level does not reward exp for lower level crafts - and since you could probably get that level with instant exp all stored exp would be lost because its below that level? Or am i missing something?

I-am-Erk commented 2 years ago

How exactly will this system interact with levelups? Because for crafting reaching a certain level does not reward exp for lower level crafts - and since you could probably get that level with instant exp all stored exp would be lost because its below that level? Or am i missing something?

practice_rate = (50 + current_focus/2 + int + per ) / task_difficulty * pool_mitigator

where task_difficulty is (your skill level - difficulty of what you're doing )^2, min 1 - note that this means more difficult tasks than your level also decrease xp, not increase.

So, task_difficulty serves that role now. We may need a more robust calculation, because I think Kevin wants it to be straight up impossible to gain xp from a difficulty 1 task at skill level 7. In this model, doing a difficulty 1 task when you are level 7 will reduce your xp to 1/36th normal.

One option of course would be to simply make practice rate 0 if task_difficulty (which I should rename "task triviality" actually) is higher than whatever cutoff we want.

This doesn't interact with stored xp at all. The xp you gain from a given task is the same in this model as in our current one, so if crafting a glove gets you 100 practice, now you get 50 of that up front and 50 over the next few minutes to hours, and once you have it pooled that xp will not change value for any reason.

estebandellasilva commented 2 years ago

How exactly will this system interact with levelups? Because for crafting reaching a certain level does not reward exp for lower level crafts - and since you could probably get that level with instant exp all stored exp would be lost because its below that level? Or am i missing something?

practice_rate = (50 + current_focus/2 + int + per ) / task_difficulty * pool_mitigator

where task_difficulty is (your skill level - difficulty of what you're doing )^2, min 1 - note that this means more difficult tasks than your level also decrease xp, not increase.

So, task_difficulty serves that role now. We may need a more robust calculation, because I think Kevin wants it to be straight up impossible to gain xp from a difficulty 1 task at skill level 7. In this model, doing a difficulty 1 task when you are level 7 will reduce your xp to 1/36th normal.

One option of course would be to simply make practice rate 0 if task_difficulty (which I should rename "task triviality" actually) is higher than whatever cutoff we want.

This doesn't interact with stored xp at all. The xp you gain from a given task is the same in this model as in our current one, so if crafting a glove gets you 100 practice, now you get 50 of that up front and 50 over the next few minutes to hours, and once you have it pooled that xp will not change value for any reason.

Ah ok so Pooled exp does not have a value of skill level assigned up to which level it can pull you - technically it would be possible to reach high/max level crafting skills by using low level recipes, which was impossible before as you needed to be inside a certain range of your skill to gain exp.

thanks for clarifying

I-am-Erk commented 2 years ago

We may still keep it impossible to reach high level skill with low level crafting. In the described model here, it's functionally impossible even if it isn't strictly impossible: the xp gain for low level tasks becomes exponentially lower while the xp requirements become exponentially higher, so it would take many thousands of hours.

PatrikLundell commented 2 years ago

I would suggest implementing a difficulty level below which you don't gain any XP (as mentioned above), but instead gain the "mind rest" state, so when you're sufficiently skilled, mending your equipment after today's fighting provides time to mull over what you learned from it without any distraction from the routine task (or the hate object of driving to the fight tanking your focus instead turning into actually useful time as long as the driving is trivial).

I'd also want to be able to see both focus pools up front, i.e. current focus and the backlog of the pool, rather than it mashed mashed into a single aggregate.

ghost commented 2 years ago

pool_mitigator seems to me as the single most important piece of information to keep track of as a player. Knowing when diminishing returns kick in and it's time to take a break. Such information is already present in other systems: I'm in pain, time to disengage from combat; I'm weary, it's time to stop physical activity; I'm tired, time to sleep; My limbs are nearly broken, time to heal;

Similar should be communicated to the player regarding learning based on pool_mitigator, as that will be the new optimal learning limit.

I-am-Erk commented 2 years ago

@PatrikLundell I love that idea. Thanks, will add it.

@Tommy-os my plan is that the player facing information about focus will be an amalgam of focus and pooled xp, and a high pooled xp will show to the player as dropping focus. To work that out though, I need to finalize the pool_mitigstor calculation and j am not quite happy with what I've got yet after running some numbers

jnshi commented 2 years ago

Will NPC teaching continue to have the same effect on focus/xp as it does now?

I-am-Erk commented 1 year ago

Will NPC teaching continue to have the same effect on focus/xp as it does now?

apologies for the very slow reply, but no, I think NPC teaching would function under this same system, so whatever xp you gain would be split into an immediate and a pooled xp gain. All this is meant to model real learning a bit better, where you learn more by slowing down and letting your brain work over the skills you've gained than you do by grinding constantly.

DeciusBrutus commented 1 year ago

Do you expect to have the multiplier for theoretical skill being higher than practical skill to apply to both instant and delayed XP? Are you intending to replace the focus and learning effects in Character::Practice rather than just having this in front of it?

I-am-Erk commented 1 year ago

Do you expect to have the multiplier for theoretical skill being higher than practical skill to apply to both instant and delayed XP?

Yep. When you gain xp, whether instant or pooled, it is applied to skill/knowledge in the same way.

Are you intending to replace the focus and learning effects in Character::Practice rather than just having this in front of it?

Replace. That's what I was getting at with the "No more complex focus equations" bit yes.