Rajathbharadwaj / NetHack-2021

Nethack 2021 codebase implementations
0 stars 0 forks source link

Keeping track of monster status #2

Closed paulkent-um closed 2 years ago

paulkent-um commented 3 years ago

This is a fundamental issue that I believe is best explained by looking at the Tourist.

The Tourist is notoriously hard to work with early on, due to its poor statline, lack of beneficial armor, and mediocre weapons. Indeed, the Tourist has only one piece of equipment that really serves to save its hide against early game threats like Foxes and Gnome Lords – the Expensive Camera. The Camera permanently blinds a monster when used, has a 75% chance to force the monster to turn and run, and can be used from a distance (though to less permanent effect). So effective use of the camera is essential to survive as a Tourist.

But how do we code that? Since the camera does no damage and its effect is pretty much idempotent, the best policy would be to use the camera, and then switch to fighting with your bare hands. But this requires us to track what monsters we've used the camera on and which ones we haven't.

So how can we track individual monsters, to label which ones we've already blinded? This is not an easy question even in theory, and though humans figure it out in routine cases without issue, it's still easy for a human to get mixed up when more than one monster with the same glyph is involved. Moreover, I suspect there may be cases where there's no way to tell which monster is which, even if line of sight is no issue.

Imagine that a fully observed 3x3 area of the floor has three Gnomes, in the following pattern: . . . G G G . . . You know that the one on the right carries a Wand of Death, and therefore MUST be targeted first if you're going to reliably survive. But then, on the next step, this is what you see: . G . . G . . G . Theoretically, each of the three Gnomes could have moved to any of the three positions, since Gnomes can move diagonally. Since you only observed these two discrete moments, and nothing about the transition between them, it would appear that you just have to guess which one has the Wand.

So uh... how should we proceed?

Rajathbharadwaj commented 3 years ago

I guess, instead of tracking which Gnome we blinded, how bout, did we blind a Gnome and if yes, just continue since there is a 75% chance that the monster will flee, if the monster choses to attack immediately switch to bare hands or even better blind the monster and immediately switch to bare hands for benefit of the doubt, more chances that we can take the monster out by surprise if you will.

paulkent-um commented 3 years ago

For a Gnome that should work well enough. But consider a Floating Eye – we need to know for sure if the Eye is blinded (because of its paralyzing gaze attack), or else attacking it is too risky.

Rajathbharadwaj commented 3 years ago

We can also consider mirroring the attack, in which case it won't work, we can Blind ourselves with a blindfold or cream-pie if you will. We can take our chances by throwing rocks or performing melee attacks. I guess we can hardcode for these kind of niche cases when we know the best solution for a specific monster or a particular situation.

paulkent-um commented 3 years ago

If you mean literally using a mirror on a floating eye, sure that paralyzes it, but the gaze attack is still in effect, so it's kind of a moot point. And yes, there are other solutions to floating eyes, but we don't always have the luxury of using them. For example, a Tourist is guaranteed to start with an Expensive Camera, but only has a 4% chance of getting a Towel to self-blind.

paulkent-um commented 3 years ago

It occurs to me that we need a very similar measure for items. We want the agent to step on items so it can get a closer look and evaluate if they're worthwhile, but stepping on the same stack over and over and deciding "nah" each time is definitely not the correct behavior. We need to remember if we've already stepped on an item so we can avoid stepping on it again.

The presence of the starting pet complicates things further, since if the pet steps on an item and then off again, we need to know that it's still the same item, lest we get caught in a loop, repeatedly checking the item only for our pet to repeatedly step on it and make us forget that we checked it.

paulkent-um commented 3 years ago

UPDATE: I think I've managed to make it work. The pet stepping on things isn't as big an issue as I thought, possibly because of pets' tendency to pick up what they step on. (This is actually really good for us right now, the fact that pets pick stuff up, because it's a way to break up untenable stacks into things we can actually register.) I'll commit my changes now.

paulkent-um commented 2 years ago

I was just now looking back at the Issues. The general case of keeping track of monsters is cleanly solved with observant_agent's policy of giving monsters names and remembering individuals that way. This isn't without its downsides – you can't really give or check names when you're blind or hallucinating, and some monsters without names can't be given names – but I'll open separate issues for those situations as we reach them.