Q&A Prompt system

[therationalpi]

So this was an idea I floated in the Writer's discord and wanted to give it some more room for discussion. I've been considering changing out the keyword-based chatbot for another system to address some of its shortcomings.

The point of the keyword system is to make conversing with Monika a little more dynamic and active. Instead of just sitting around waiting for her to tell you something, you actively write things trying to trigger conversations. I think we can all agree, though, that it doesn't always achieve its goals.

My suggestion is to supply the player with prompts for things to ask, and have those prompts be interconnected so going through some prompts unlock others. They can also unlock new random topics and trigger events.

This system is similar to one used in Analogue: A Hate Story. In that game, the player would read journal entries supplied by a librarian AI. As you read journal entries, she would offer related journal entries, until you finally read everything the story had to offer. It was simple, story focused, and it gave the player just enough agency to pick the topics that interested them, but didn't lead to unmanageable branching or dead ends.

For our system, you might click a prompt for "Tell me about your family." And Monika could go into a little spiel about how she is an only child, and that her parents really put a lot of pressure on her as a result. Listening to this would unlock prompts for "Did you ever want a little sister?" and "What kind of grades did you get?"

As the game goes on, this library of questions will fill in more and more. At first is a bunch of main topics with ?????'s in them, but over time those ?????'s get replaced by the prompts. Unread prompts show up in white, and read ones are greyed out (but can be reread). Likewise, there would be a list for random topics and one for story event that would slowly fill over time, so you can reread a story at your leisure.

On the one hand, this system helps players know if they've exhausted the content of the game, but it can also hurt the illusion that Monika is a real person. I think it's a fair tradeoff, though, since Monika already runs out of new topics, and she can always point out that everybody runs out of interesting things to say sometimes. "Haven't you ever started telling a friend a funny story, but they stop you because you've already said it?" "It's kind of embarrassing, especially if you always tell the story the same way."

This would be a big feature change, and would obsolete a big feature with the chatbot. Do you guys think this could be a worthwhile improvement, though?

[ThePotatoGuy]

I think this would change the mod's focus from being a girlfriend simulator to a linear "game" with the end/win condition of listening to all of Monika's topics. And personally, I prefer this mod being a gf sim instead of a game itself.

[ghost]

I second what @ThePotatoGuy said. As cool as this would be in other circumstances, it would really take away from the "gf sim" perspective.

[therationalpi]

@Jxhnathan08 We had some more discussions about this on Discord. What we came to was metering out these prompts over time and ditching the whole ????? thing. Basically you would only get a maximum of a 3-5 prompts per day, so you'll have a reason to come back regularly, and so you won't run out of content for several weeks, even if we don't push out any more content...which we will.

Also, keep in mind that this is not a replacement for Monika's random chatter. This is a parallel system that's meant to replace the topics we have right now that are only triggered by keywords. There are already people that ask what all of the keywords are, so the desire to "see all the content" is already there, but the system is just bad.

This, combined with the Calendar of Events and Major Events ideas should give a good variety of content, and also encourage players to check back often.

[ThePotatoGuy]

We might need to keep some form the keyword trigger functionality for certain interactive topics (change name, say happy birthday).

[therationalpi]

Well, those will actually be easier with this system, since those can just be selected from the prompts.

[therationalpi]

Time to revisit this discussion, since I want this to go out in version 0.7.0. Do we have any ideas on how best to implement this? Should we have a nested tree structure with different topic areas? Should random topics that have been seen in normal play be accessible through this? How can we make this a truly engaging system?

[Guardianvoir]

I think some people wouldn't like the idea of being given specific choices of what to say to Monika, so what if we gave players the option to switch between typing out what they want to say and choosing from a list of things to say? That way people can choose which style they prefer.

[therationalpi]

Well, then we need to support two competing systems, one of which is decidedly broken, and they aren't necessarily compatible with eachother.

[therationalpi]

Alright, so I realize now that I didn't really explain why replacing the keyword system is so necessary, so here's my reasoning:

The goal of the keyword system is to add a way for the player to steer the conversation and have a sense of agency for what monika talks about. The problem is that the system isn't perfect, and whenever and however it fails exposes the simple system underneath and hurts immersion.

For example, if you Ask Monika "Do you think people have free will?" she'll appropriately catch "free will" and rattle off a topic. That's good behavior. But, if you said "Do you think people control their fate?" Well, "Fate" isn't a keyword, and so you get a canned response, even though you asked basically the same question. That breaks immersion, because now you see that Monika isn't a person responding to your questions but a bot hunting for keywords in a dictionary. On a case-by-case basis we could try to fix this, but the fact of the matter is that there will always be synonyms we miss, unexpected phrasing, and other ways that topics that should be hit get missed. The only possible way around that would be to reimplement the entire keyword system using natural language processing, which would handle a lot of noun-phrase synonyms and lexical tokens and such, but technical limitations make that near impossible.

This really colors people's perception of the game by the way. You can't bring up Monika After Story on the DDLC Fan Club discord without someone immediately saying "Oh, it's just a shitty chatbot." Generally it's followed by a story about how they asked a question and got a canned response.

And even if the keyword system somehow worked flawlessly as far as connecting the player's input to an appropriate topic, there are still problems with it. The first is that players will always have topics that they think of that we don't have a script for. There are simply too many things in this world for us to hit all of them. The second is that the player's ability to find the content is then limited by the player's imagination in a weird way. If we have a topic that isn't in the random rotation and the player doesn't think to ask about it, then it will never be seen. That discourages writers from phrasing discussions of obscure topics in any way that can't be part of the random topic pool. It also encourages some players to seek out a keyword list so they don't miss content.

Another big problem is that the content isn't metered out in any intelligent way. There are no breadcrumbs to lead players from one topic to another, there is nothing to limit the player from just spamming words hoping to find a keyword, there's nothing to keep a player from accidentally reading a script multiple times because they just asked for a bunch of synonyms when they really just wanted Monika to expand on something. Hell, expanding on a topic is really difficult to implement in the current system, and that would be a very natural thing for Monika to do in a conversation. Also, from a game flow standpoint the "Exit point" for the keyword system is generally on a low note. The player will keep posting keywords until they get lame responses or repeated topics, then they quit asking and probably never try it again because they just think "Eh, what's the point?"

That's not to say that I don't see the appeal of the keyword system. It feels like it gives the player a lot of agency, because you can literally type anything you want in that text box. But it isn't real agency, and the illusion doesn't last very long. You can only ask a limited set of things, and you just don't know what those are. It's really the same as the prompt system, in the sense that your options are restricted, but they're restricted in a way that's completely obscured from the player.

[therationalpi]

Here are some things I said in discord about the possible new system:

Alright, so I had some ideas with this... We want to have keys for the topics, which will be the event labels... And this will attach to a dict. That dict will essentially be our event object... We can combine the calendar system, our topics, and everything else into this event object framework.(edited) So the event object would include things like unlocked which would determine if that event is available at the moment as a prompt. prompt would be another field, which is what the prompt button displays for the topic. is_random determines if the event can be triggered randomly in monika's dialogue... prompt_available would determine if the prompt is in the pool of available prompts the player could get. For example, you use a prompt to ask Monika about her family... At the end of that, maybe it would set a few events with different ways to get them. For example, it might immediately add a prompt where you can ask about her mother. by setting unlocked = True(edited) It might also make it so Monika could randomly bring up her family pet is_random=True It might also add a few prompts about her parents to the pool of available prompts prompt_availabe = True(edited) Hell, it could even schedule an event on Father's day... As part of the calendar system. Each day, you could get a few free prompts from the pool of available prompts... Also, as you interact with Monika it fills up a hidden XP bar that will also trigger new prompts and special events. This would be balanced in such a way that at first you could get a flood of new content by playing for a few hours with the mod... After that it trails off... And unlocking becomes linear, such that you could only really unlock 2 or 3 additional prompts per day on top of the 2 or 3 you get automatically each day. It also piles up a bit... So if you come in after a week, you might have 5 or 6 prompts waiting for you and potential to unlock 4 or 5 more... More content than normal, but you still unlock the most by checking back with monika daily. Random topics never go away, though. So even if you don't have any more prompts, you can always just sit with monika on the second screen or play some games with her. I think it helps meter out the content, and gets at the general gameplay flow we want. You log on, interact directly with monika for a while and learn more about her and her thoughts... Then you can just hang out with her casually... And the next day do it again. The prompts give you a sense of agency, even if it's limited... But I think that it's better to have a strong illusion of open endedness than the current keyword system which is truly open ended but includes a lot of dead ends and blind alleys.

[therationalpi]

Okay, so for implementation, I think we want to make a persistent dictionary of all of these events. The key would be the event label, and the event object is a dictionary with the following fields:

prompt - String label shown on the button for this topic in the prompt menu.

label - Optional plaintext name for the event, good for calendars.

category - Tuple of strings that define the category structure for the event in the prompt menu.

For example, if the category was ['monika','family'] then this prompt would be under the Monika>Family list in the menu.

unlocked - Boolean that tells if the event appears in the prompt menu.

random - Boolean that tells if the event goes into Monika's random topic pool.

pool - Boolean that tells if the event is in the pool of prompts that get drawn from when new prompts become randomly available.

pool and random are different. Pool is for prompts that get given to the player, random is for things Monika brings up on her own. There can be overlap, but I don't think there should be if we can avoid it.

conditional - This is a conditional expression that's checked for the event at various points to determine if it gets pushed to the event stack or not.

This allows us to modularize our flow for adding new events to the event list on game startup and potentially at other points. The conditional is the expression that we use to determine what we do with the event,

action - A string that tells what to do if the conditional above is true (push,queue,unlock,random,pool,None).

push and queue would be for directly adding the event to the event list when the conditional is True, unlock would unlock the prompt in the prompt list, random and pool would add the item to the random list and prompt pool, respectively.

start_date - Timestamp for when this event is available.

end_date - Timestamp for when this event is no longer available.

Most of these can be None by the way, so not all of these fields need to be defined.

[therationalpi]

For an example, let's look at a few events currently in the game:

monika_god

default persistent.event_database['monika_god'] = {'prompt' : "Do you believe in god?",'category' : ['beliefs','religion'],'random' : True}

This is a standard topic that goes into the random rotation. There's a prompt and category for it for when the event has been seen randomly at which point it is added to the prompt menu for later viewing.

monika_anime

default persistent.event_database['monika_anime'] = {'prompt' : "Do you like anime?",'category' : ['interests','hobbies'],'pool':True}

This is a standard topic that needs a prompt. Monika is unlikely to bring up anime on her own and the topic is written in a way where it sounds like she's responding to a question, so it's added to the pool of randomly assigned prompts.

ch30_reload_2

default persistent.event_database['ch30_reload_2'] = {'conditional' : 'persistent.monika_reload == 2 and not persistent.closed_self', 'action' : 'queue'}

This is where things get cool. This eliminates the need for a special section of code to add reload topics (or anything else like that), because when the game loads it just checks all of the conditionals in the event database and does the action if the conditional evaluates to True. This event won't ever appear in the prompt list, so it doesn't need a prompt or category.

christmas_day

default persistent.event_database['christmas_day']={'label' : "Christmas", 'prompt' : "What was your favorite Christmas?", 'category' : ['Special Events','Christmas'], 'action' : 'push', 'start_date' : 1514160000, 'end_date' : 1514246399}

This one is super neat. This creates a Christmas event that is automatically pushed to the event stack on Christmas day (note that the conditional is not needed here because the default conditional for timed events is 'True'). Also, because it has a prompt and category, we can potentially set unlocked to True at the end of the event so players can revisit this event in the future.

[ThePotatoGuy]

A couple of things:

Regarding the fields:

I like the fields you've defined, however due to the large number of them, I think its best that we incorporate them into a class instead of a dict. This would prevent issues where an event dict is created with a mispelled key or one of those fields is undefined. (Even tho we can get around this via x.get("label", default), its much cleaner and easier to understand x.label) (Also, errors misspelling the fields for a class cause runtime errors instead of silently continuing when using get)

A dictionary of the events would still be fine, except the value of each element is an object.

conditional string checking:

The condition expression is checked at "various points". Are we going to be looping over the entire events dict at those points to check for satisfied conditions? That seems like a ton of processing, mainly since conditionals are strings and must be evaluated using exec, which is bad practice.

If conditional is a required function instead, maybe it won't be so bad to loop over it, but that may be too much for non-programmers to add/use.

EDIT: updated wording of fields section

[therationalpi]

So, just to make a record of things said in discord...

Making a class for events sounds really good, but the problem is that Renpy doesn't combine classes and persistents well. @ThePotatoGuy is looking into options for serializing a class so Renpy doesn't freak out over this.

For conditional string checking, I'm open to alternatives, but I feel like this might actually be one moment where using eval() isn't the devil, since we are only using it to evaluate one-line conditional statements and not run any commands. We can also speed this up because most events won't have a conditional, so if we do find ourselves looping over the event database checking conditionals, we can do something like:

for event in persistent.event_database:
    if event.conditional:
        if eval(event.conditional):
            [some code]

This will do a fast check on if there even is a conditional, then a slower check on what the conditional actually is after that.

There's actually a big advantage here in terms of code readability as well, since this lets us put the conditionals used to determine when an event is queue'd right with the event instead of in a different script.

[JoshS-dev]

How does one join this discord? Is a super-private one for the people behind it, and not for people who want to follow the project? For me, it would be a lot easier to read up on a discord as opposed to looking at GitHub, which I'm still not very used to yet ^^;

[ghost]

https://discord.gg/7P5DnJ4

[ghost]

One last comment for now. I thought I'd give an opinion. I like the idea of pre-defined prompts as long as it isn't linear and doesn't allow you to just read through all her dialogue and exhaust her. Even so, if you ever do get the same prompt, and she repeats herself, even a little teensy immersive comment can really help that. The mod could flag if you've chosen a topic before, and if it ever pops up again, she could say "Hmm... Deja vu much? I thought you asked before, but..." and the dialogue would repeat the topic. She'd need multiple dialogue strings to begin with though, not just a deja vu comment, so there's variety.

[therationalpi]

Yeah, I agree that Monika should comment when you ask a prompt multiple times.

Also, this shouldn't be too linear. Because of all of the different ways we can add prompts, have prompts rely on each other and stuff, the experience should seem very fluid with new prompts popping up regularly, but not so regularly that you can just spam prompts all day long.

As for exhausting her dialogue, if anything this would allow us to meter out her dialogue more deliberately. Right now you could just pull up a list of Monika's keywords and run through them (not that I know of anyone that has actually listed them all). With the prompt system, you will get a mix of questions to ask and just waiting for new topics.

[ThePotatoGuy]

Good news @therationalpi . I've managed to successfully serialize a class and save and load it properly from persistent!

Check the branch serialization-test for the code. I created an Event class in a python early block in definitions.

If you want to test out the serialization:

1. Run the game.
2. Close the game and relaunch.
3. Open dev console.
4. Run the function testSerial()
5. This function outputs a string of testcases. I recommened writing it to a file to get a better view. The test cases have the following format: [<bool>] - <name> -> F: <found> E: <exp> where bool is True if the test case passed, False if not, name is the name of the property this case is testing found is what we found in persistent.serial_objs exp is what we expected to find in the persistent.

In addition, just viewing persistent.serial_objs is enough to see the new Event objects.

I made the Event class in a way so that we can actually start building the chatbot replacement with it now. lmk if you have questions regarding the class.

[therationalpi]

Awesome Potato, I'll be sure to get on this!

[Hypeouseaus]

I think we don't have to necessarily disable chatbot feature even if we're going to implement QnA prompt system. Some people want direct conversation with Monika.

There should be a prompt suggesting direct question on default.

Like

[Do you like chocolates?] [What do you think about France?] .... [Ask directly]

If the gamer choose the last one, he/she can type questions they want and monika will answer it.

[ghost]

The talk system is still going to be salvaged for minor topics. And from there people who are interested can expand it.

[therationalpi]

The key functionality of this is done. The rest of the work on this is polishing.