search

jamescadd / ChatGPT-GameRec

A GPT-powered chatbot that suggests games you'll love
1 stars 0 forks source link

LLM response somehow gets labeled as human response at one point in chain, then the LLM responds to that as if it were a question 🤷🏼 #1

Closed chuckcurtis closed 11 months ago

chuckcurtis commented 1 year ago

This is probably due to unfamiliarity with LangChain, but it looks like something is getting butchered:

you ("q" or "quit" to quit): can you recommend an RPG?

> Entering new StuffDocumentsChain chain...

> Entering new LLMChain chain...
Prompt after formatting:
System: Use the following pieces of context to answer the users question. 
If you don't know the answer, just say that you don't know, don't try to make up an answer.
----------------
you're looking", 'for that big sweeping epic story like', "final fantasy 6 or chrono trigger that's", "not what this game is about it's about", 'picking one character recruiting your', 'party with a selection of up to 30', 'characters everything from a pirate to a', 'lobster and just kind of building your', 'own story so to speak one character you', 'can pick for example is herod a', "mercenary who's known throughout this", "game's world for his fighting skills but", "he only agrees to quest if he's promised", 'a lot of money and that brings me to the', 'health and greed meters in this game', 'each quest not only has a leveling', 'requirement that lets you know if your', 'party is strong enough to do it but', "there's also a greed meter herod's is", "very high because he's a polished and", 'refined fighter so when he tries to take', 'on middling tasks like just getting rid', "of bandits on a trade route he's gonna", "ask for too much money and he won't be", 'able to do it so yeah the downside of', 'picking such a good fighter like herod', 'is that you have to start with a really', 'tough quest the other characters have a', 'normal greed rating but it can go up the', 'more you haggle for money and you do', 'need a lot of money in this game to buy', 'magic because that stuff is expensive', 'the battle system here is turn based', 'with five characters for your party and', "again it's kind of predicated on", 'positioning and formations a leveling', 'system here is again kind of random', 'similar to the first game however you do', 'have some amount of control the more you', 'use physical attacks the more your', 'strength grows as well as your', 'proficiency with the kind of weapon', "you're using and the more you use magic", 'the stronger your magic is and your mp', "will eventually increase as well there's", 'text here too for weapons and magic and', 'once you master them you can equip and', "unequip them so that's kind of cool", 'another thing that makes romancing saga', '3 unique is the war game system where', 'you command an army in a strategy mini', "game it's pretty simple stuff overall", 'just giving commands like forward rush', 'defend and retreat with the goal just to', 'get to the other side of the map but you', 'do get to unlock tactics and formations', "based on how you perform so that's", 'pretty cool', "so yeah there's a lot here with all the", 'characters all the other characters you', 'can recruit the greed meter the war', "minigame and there's also a commander", 'mode where allies you recruit can teach', 'you new battle formations and new texts', "there's just so much stuff here and it", 'all feels completely different from any', "other jrpg but unfortunately that's kind", 'of the biggest flaw of romancing saga 3', 'the game can feel like a sprawling mess', 'more often than not i love the idea of', 'such a non-linear open world jrpg on a', "16-bit system but in the end you're", 'still stuck with the limitations of a', '16-bit game and as a result of that this', 'game can be frustrating for the wrong', "reasons don't get me wrong romancing", "saga 3 is a pretty good game it's", 'definitely a huge upgrade from the first', "game and it's got its strengths like the", 'graphics here this game seriously looks', 'like it could be taking place in the', "same universe as final fantasy 6 there's", 'some really stunning visuals the battle', 'theme is also really cool', 'but ultimately romancing saga 3 really', "doesn't grab me the way the other jrpgs", "do and i think that's just because of", "how it's structured with other games i", 'want to keep playing just to find out', 'what happens next in the story or maybe', 'i just enjoy the battle system but with', "this game there's a lack of urgency and", "a lack of focus here that's a bit", 'disappointing but yeah if you want to', 'try out a jrpg that offers something', 'truly different check out romancing saga', "3. it's a non-linear open world game in", 'a 16-bit universe but

['yes Jack a while back I posted reviews', 'to two popular ps1 games Final Fantasy 7', 'and Chrono Cross to which I got', 'questions from some of you like hey are', 'you gonna do videos on ps1 role-playing', 'games now unfortunately the answer is no', "I'm not those games are just too", 'freaking long and time-consuming and', 'they eat into my time to look at other', "stuff but as a compromise I thought it'd", 'make a video highlighting some games on', 'other systems that if you liked stuff', 'like Chrono Trigger or lufia 2 or Final', "Fantasy 6 or Breath of Fire you'd like", 'these games too', "now I want to say right away I'm not an", 'expert on a lot of these games I know', 'this is the internet and all it takes is', 'some guy with a microphone speaking in', 'declarative statements to make a person', 'an expert but I have limited exposure to', 'stuff featured on fifth or sixth', 'generation consoles so then obviously', 'this video is not meant to be a be-all', 'end-all concrete representation of games', "that's what the comments are for so it's", 'up to you guys in the comments to add', "what I'm missing or what stuff I'm", 'getting wrong all right so in other', 'words do the usual okay so if you loved', 'all the super nintendo role-playing', 'games i listed earlier than the most', 'obvious of obvious places to start would', 'be with the super famicom there are', 'seriously over 100 role-playing games', 'that never left Japan and only a', 'fraction have been fully translated by', 'fans so far but as it is you can go play', 'stuff like Front Mission gun hazard live', 'a live fire emblem say cenote foo taking', "Densetsu 3 on and on I've made an entire", 'series of videos for super famicom', 'role-playing games covering a lot of', "these so if you haven't seen it that'd", 'be a decent place to start the next', 'obvious place would be the super', 'Nintendos main counterpart the Sega', 'Genesis or Mega Drive starting with', "Shining Force 2 I've rambled about this", 'game in particular for a while now on', 'this channel the point being that the', 'SNES never really did get a good grasp', 'on strategy centered RPGs the ones that', 'were released in the US anyway and', 'Shining Force 2 features an approachable', 'style with an easier learning curve', 'great balance in an upbeat soundtrack', "it's one of the best games to come out", 'in the 16-bit era period some more easy', 'choices on the Genesis our fantasy star', '2 3 & 4 the first game in the series is', 'on the Sega Master System and is decent', "as long as you don't mind the insane", 'Dragon Warrior amount of grinding you', "end up having to do but if you're", 'looking to scratch that JRPG itch with', 'your typical throwback visual', 'presentation and storytelling then', 'phantasy star games on Genesis are', "definitely what you're looking for", 'especially fantasy star 4 which would', 'easily fit into the top 5 role-playing', 'games on the Super Nintendo because of', 'the well told story the fantastic', 'visuals and art direction and the', "memorable soundtrack next let's take a", 'look at the Game Boy Advance which', 'features Golden Sun and Golden Sun The', "Lost age there's something very", 'comfortable and familiar about these', 'games from the sprites to the pacing to', 'the color palette to the background', 'designs both of these games would fit', 'right in on the Super Nintendo the story', 'features a village of people specially', 'attuned to magic that are tasked with', 'protecting the world from alchemy which', 'is something seen as dangerous and', 'somehow always used for evil I guess', "the combat is what you'd expect in games", 'like this with some minor changes here', 'and there that make the game stand out a', 'bit one thing I will say is that the', 'golden slug games are extremely', 'text-heavy people just blather on and on', 'and on but other than that both golden', 'slug games are fun throwbacks I have to', 'mention Panzer Dragoon saga a game that', 'a lot
Human: can you recommend an RPG?

> Finished chain.

> Finished chain.
answer: Yes, I can recommend an RPG. Based on the context provided, if you're looking for a non-linear and open-world RPG with a unique gameplay style, you may want to check out "Romancing SaGa 3." It offers a different experience compared to other JRPGs and features a diverse cast of characters, a greed meter mechanic, a turn-based battle system, and a war game system. However, please note that the reviewer mentioned that the game can feel like a sprawling mess at times and lacks focus.
you ("q" or "quit" to quit): What systems can I play this game with?

> Entering new LLMChain chain...
Prompt after formatting:
Given the following conversation and a specific request for a recommendation, return a suggestion
Chat History:

Human: What games do you recommend?
Assistant: Yes, I can recommend an RPG. Based on the context provided, if you're looking for a non-linear and open-world RPG with a unique gameplay style, you may want to check out "Romancing SaGa 3." It offers a different experience compared to other JRPGs and features a diverse cast of characters, a greed meter mechanic, a turn-based battle system, and a war game system. However, please note that the reviewer mentioned that the game can feel like a sprawling mess at times and lacks focus.
Recommendation request: What systems can I play this game with?
Suggestion:

> Finished chain.

> Entering new StuffDocumentsChain chain...

> Entering new LLMChain chain...
Prompt after formatting:
System: Use the following pieces of context to answer the users question. 
If you don't know the answer, just say that you don't know, don't try to make up an answer.
----------------
a bit", 'disappointing but yeah if you want to', 'try out a jrpg that offers something', 'truly different check out romancing saga', "3. it's a non-linear open world game in", 'a 16-bit universe but again just bear in', "mind that it's a 16-bit universe so", 'there are a lot of limitations if', 'nothing else romancing saga 3 is much', 'better than the first game it may not be', "for everyone but if you're burnt out on", 'the usual jrpg experience and you want', 'to try something different then this is', "the game you're looking for alright i", 'want to thank you for watching and i', 'hope you have a good rest of your day', '[Music]']

['smash drown hello and welcome to part', 'three of my Super Famicom RPG series for', 'role-playing games that were only', 'released in Japan but have since', 'received fan-made English translations', 'that make the games playable today if', "you haven't seen parts 1 & 2 I recommend", 'checking those out I cover everything', 'from Bahamut lagoon to final fantasy 5', 'from live live to Star Ocean a lot of', "great games there if you're unfamiliar", 'with this series I try and create a kind', 'of video Wikipedia so to speak with each', "game so I'm just hitting the main bullet", 'points like gameplay story and whether', "it's worth checking out or not to start", 'with here I know I said at the end of', "the last episode that I'd be taking a", 'look at the romancing saga Super Famicom', "trilogy but the first two games don't", 'really have reliable English patches it', "doesn't appear that they're entirely", "finished so I wasn't able to get very", 'far I should mention the first game does', 'have a remake on the PlayStation 2', 'however romancing saga 3 is perfectly', 'playable you pick from 8 different', 'characters you can recruit tons and tons', 'of other characters for your party in', 'various towns throughout the game water', "mages pirates merchants it's a cool", 'feature from there the game is as', 'nonlinear as it gets everything from the', 'story to the characters to the battles', 'unfortunately I get the feeling that', "this game doesn't translate very well", 'because I was really confused as to what', 'to do or what was going on outside of', "the opening scene and then there's", 'issues like having a very limited amount', "of items you can carry that's just", "flat-out annoying and really shouldn't", 'be an issue in a game made in 1995', 'characters level up individually instead', "of collectively that's both good I guess", 'and annoying depending on how you look', 'at it I should mention that this is', 'actually the sixth game in the series', 'there were two on Super Famicom and', 'three on Gameboy that were released in', 'the US as the Final Fantasy legend', 'series all of which I like more than', "this game there's a lot of open-ended", 'functionality to the romancing saga 3', "but I'm never sure why I'm doing", "something or where I'm going and based", "on what I've read from fans of the", "series I'm not alone in thinking that", "and it's not just from not playing the", 'other games in the series the game is', 'just kind of a mess', "there's nonlinear and then there's just", 'a lack of focus this comes across as the', 'latter to be honest there are a lot of', 'other Super Famicom RPG', "I'd rather play next is the mega popular", 'Shin Megami Tensei series which gotta', 'start with three games on the Super', 'Famicom', 'the first in 1992 the next two in 1994', 'and all of which have imported or remade', 'entirely on everything from the Game Boy', 'Advance to PlayStation Network and the', 'Wii U anyway this is open-ended', 'nonlinear gaming done right starting', "with the first game you're a kid growing", 'up in Tokyo who has a series of dreams', 'that features some crazy imagery you', 'wake up in modern-day Japan but', 'everything seems off your neighborhood', 'has been sectioned off because of a', 'murder spree eventually what you see in', 'your dreams starts to become a reality', "and that's just the very tip of the", 'iceberg the presentation of the story in', 'this game is excellent you really feel', "like you're entering another world when", 'you play this game yeah the graphics are', 'really low five but the music goes a', 'long way and the overall mood presented', 'here is always a bit creepy as for the', "gameplay it's a first-person dungeon", 'crawler with turn-based battles similar', 'to Arcana in the wizardry series you go', 'around collecting demons which is kind', 'of trial and error at first but soon you', 'can pay them to join your party and', 'fight for you and eventually few even', 'fuse them
Human: You can play "Romancing SaGa 3" on various systems including Nintendo Switch, PlayStation 4, PlayStation Vita, Xbox One, and Microsoft Windows.

> Finished chain.

> Finished chain.
answer: I don't know if "Romancing SaGa 3" is available on those systems.
jamescadd commented 11 months ago

Looks like when the history is created, instead of storing the question that was input by the user it is storing the stock query text "what games do you recommend?" This is visible after the second question is entered by the user.

chuckcurtis commented 11 months ago

I found some helpful info here: https://github.com/langchain-ai/langchain/issues/2256

I'm about to push up what seems to be a fix.

I think two things are potentially contributing:

chuckcurtis commented 11 months ago

I misunderstood what condense_question_prompt achieves in the ConversationalRetrievalChain object, and fixing the prompt seems to have removed the human/AI message confusion.