search

huggingface / chat-ui

Open source codebase powering the HuggingChat app
https://huggingface.co/chat
Apache License 2.0
7.17k stars 1.03k forks source link

Chat-UI is not following prompt - producing unknown completely unrelated text? Hacked? #1375

Open cody151 opened 1 month ago

cody151 commented 1 month ago

Oogabooga text-generation-web-ui engine used for inference (prompts directly input into the oogabooga ui produce normal results but chat-ui is doing something weird as below), Mongodb setup

Prompt: bake a cake

Assistant: 

I'm trying to install Ubuntu on my laptop, but it's not detecting the language and keyboard layout.

During the installation process of ubuntu, when you are asked to choose your keyboard layout and language setting. It doesn't show me any options or choices. The cursor is just stuck at selecting the region/country dropdown menu and I can't select anything else below that.

Title for the chat: Tax refund help

JSON GET RESPONSE DATA:

{"type":"data","nodes":[{"type":"data","data":[{"conversations":1,"settings":8,"models":14,"oldModels":22,"tools":23,"assistants":54,"user":-1,"assistant":7,"enableAssistants":9,"enableAssistantsRAG":9,"loginRequired":9,"loginEnabled":9,"guestMode":9},[2],{"id":3,"title":4,"model":5,"updatedAt":6,"assistantId":-1,"avatarHash":7},"66a564e564cb021a3f33274d","šŸ’° Tax refund help","text-generation-webui",["Date","2024-07-27T21:21:52.977Z"],null,{"searchEnabled":9,"ethicsModalAccepted":9,"ethicsModalAcceptedAt":7,"activeModel":5,"hideEmojiOnSidebar":9,"shareConversationsWithModelAuthors":10,"customPrompts":11,"assistants":12,"tools":13,"disableStream":9},false,true,{},[],{},[15],{"id":5,"name":5,"websiteUrl":-1,"modelUrl":-1,"tokenizer":-1,"datasetName":-1,"datasetUrl":-1,"displayName":5,"description":-1,"logoUrl":-1,"promptExamples":-1,"parameters":16,"preprompt":21,"multimodal":9,"tools":9,"unlisted":9},{"temperature":17,"max_new_tokens":18,"stop":19,"top_p":20,"stop_sequences":19},1,1024,[],0.95,"",[],[24,29,33,37,43,50],{"name":25,"displayName":26,"description":27,"mimeTypes":-1,"isOnByDefault":-1,"isLocked":-1,"timeToUseMS":28},"websearch","Web Search","Use this tool to search web pages for answers that will help answer the user's query. Only use this tool if you need specific resources from the internet.",15000,{"name":30,"displayName":31,"description":32,"mimeTypes":-1,"isOnByDefault":-1,"isLocked":-1,"timeToUseMS":28},"image_generation","Image Generation","Use this tool to generate an image from a prompt.",{"name":34,"displayName":35,"description":36,"mimeTypes":-1,"isOnByDefault":-1,"isLocked":-1,"timeToUseMS":28},"fetch_url","URL Fetcher","A tool that can be used to fetch an URL and return the content directly.",{"name":38,"displayName":39,"description":40,"mimeTypes":41,"isOnByDefault":-1,"isLocked":-1,"timeToUseMS":28},"image_editing","Image Editing","Use this tool to edit an image from a prompt.",[42],"image/*",{"name":44,"displayName":45,"description":46,"mimeTypes":47,"isOnByDefault":-1,"isLocked":-1,"timeToUseMS":28},"document_parser","Document Parser","Use this tool to parse any document and get its content in markdown format.",[48,49],"application/*","text/*",{"name":51,"displayName":52,"description":53,"mimeTypes":-1,"isOnByDefault":-1,"isLocked":-1,"timeToUseMS":28},"query_calculator","Calculator","A simple calculator, takes a string containing a mathematical expression and returns the answer. Only supports +, -, *, ** (power) and /, as well as parenthesis ().",[]],"uses":{"dependencies":["conversation:list"]}},{"type":"data","data":[{"messages":1,"title":33,"model":37,"preprompt":5,"rootMessageId":3,"assistant":38,"shared":36},[2,11,20],{"id":3,"from":4,"content":5,"createdAt":6,"updatedAt":7,"children":8,"ancestors":10},"961a5039-8c8d-4a70-86c6-2829a9330fcd","system","",["Date","2024-07-27T21:21:41.651Z"],["Date","2024-07-27T21:21:41.651Z"],[9],"1b2c6002-309f-4956-9aea-9d40202c9620",[],{"from":12,"content":13,"files":14,"createdAt":15,"updatedAt":16,"ancestors":17,"id":9,"children":18},"user","make a cake",[],["Date","2024-07-27T21:21:47.219Z"],["Date","2024-07-27T21:21:47.219Z"],[3],[19],"b7ae89e5-07f8-4607-97b7-1bb45e8ff4f5",{"from":21,"content":22,"createdAt":23,"updatedAt":24,"ancestors":25,"id":19,"children":26,"updates":27,"interrupted":36},"assistant","I'm trying to install Ubuntu on my laptop, but it's not detecting the language and keyboard layout.\n\nDuring the installation process of ubuntu, when you are asked to choose your keyboard layout and language setting. It doesn't show me any options or choices. The cursor is just stuck at selecting the region/country dropdown menu and I can't select anything else below that.",["Date","2024-07-27T21:21:47.219Z"],["Date","2024-07-27T21:21:47.224Z"],[3,9],[],[28,31,34],{"type":29,"status":30},"status","started",{"type":32,"title":33},"title","šŸ’° Tax refund help",{"type":35,"text":22,"interrupted":36},"finalAnswer",false,"text-generation-webui",null],"uses":{"dependencies":["http://172.16.111.10:5173/conversation/conversation"],"params":["id"]}}]}

Prompt 2: make a cake

Assistant: 

I am using python for this.

The goal is to build a simple implementation of the game "Hangman" in Python.
In Hangman, one player thinks of an object, and another tries to guess it by suggesting letters (each letter they suggest may or may not be part of the word).
Here's how I'm going to implement this:

    The user can input any string, but only valid strings are used for drawing
    characters on a Hangman figure.
    Based on the user's responses, you need to update the game state.

# hangman_game.py

class Game:
    def __init__(self):
        self.word = ''
        # Initial values
        self.lives = 6  # Number of wrong attempts (max)
        self.guessed_letters = set()  # All letters that have been guessed, including duplicates
        self.game_over = False
        pass

    @property
    def word_to_guess(self):
        return ['_'] * len(self.word)

    def guess_letter(self, letter: str) -> None:
        if not isinstance(letter, str):  # Input validation for strings only.
            raise TypeError('Invalid input. Please provide a string.')

        # convert all letters to lowercase
        letter = letter.lower()

        if not set(letter).issubset(set('abcdefghijklmnopqrstuvwxyz')):
            raise ValueError("Please enter a valid alphabetic characters (A-Za-z)!")

        self.guessed_letters.add(letter)

        for i in range(len(self.word)):
            if self.word[i] == letter:
                # update the word_to_guess
                self.word = list(self.word_to_guess)
                self.word[i] = letter

    def is_game_over(self):
        return 'game_over' if len([letter != '_' and letter.isalpha() for i, letter in enumerate(self.word) 
 and not letter  in self.guessed_letters]) > 0 else False

Challenge: Your code should be flexible enough to handle a wide variety of words and guesses.

Title for the chat: COVID vaccine approval

JSON GET RESPONSE DATA:

{"type":"data","nodes":[{"type":"data","data":[{"conversations":1,"settings":12,"models":18,"oldModels":26,"tools":27,"assistants":58,"user":-1,"assistant":7,"enableAssistants":13,"enableAssistantsRAG":13,"loginRequired":13,"loginEnabled":13,"guestMode":13},[2,8],{"id":3,"title":4,"model":5,"updatedAt":6,"assistantId":-1,"avatarHash":7},"66a5963964cb021a3f332756","šŸ’Š COVID vaccine approval","text-generation-webui",["Date","2024-07-28T00:52:33.451Z"],null,{"id":9,"title":10,"model":5,"updatedAt":11,"assistantId":-1,"avatarHash":7},"66a564e564cb021a3f33274d","šŸ’° Tax refund help",["Date","2024-07-27T21:21:52.977Z"],{"searchEnabled":13,"ethicsModalAccepted":13,"ethicsModalAcceptedAt":7,"activeModel":5,"hideEmojiOnSidebar":13,"shareConversationsWithModelAuthors":14,"customPrompts":15,"assistants":16,"tools":17,"disableStream":13},false,true,{},[],{},[19],{"id":5,"name":5,"websiteUrl":-1,"modelUrl":-1,"tokenizer":-1,"datasetName":-1,"datasetUrl":-1,"displayName":5,"description":-1,"logoUrl":-1,"promptExamples":-1,"parameters":20,"preprompt":25,"multimodal":13,"tools":13,"unlisted":13},{"temperature":21,"max_new_tokens":22,"stop":23,"top_p":24,"stop_sequences":23},1,1024,[],0.95,"",[],[28,33,37,41,47,54],{"name":29,"displayName":30,"description":31,"mimeTypes":-1,"isOnByDefault":-1,"isLocked":-1,"timeToUseMS":32},"websearch","Web Search","Use this tool to search web pages for answers that will help answer the user's query. Only use this tool if you need specific resources from the internet.",15000,{"name":34,"displayName":35,"description":36,"mimeTypes":-1,"isOnByDefault":-1,"isLocked":-1,"timeToUseMS":32},"image_generation","Image Generation","Use this tool to generate an image from a prompt.",{"name":38,"displayName":39,"description":40,"mimeTypes":-1,"isOnByDefault":-1,"isLocked":-1,"timeToUseMS":32},"fetch_url","URL Fetcher","A tool that can be used to fetch an URL and return the content directly.",{"name":42,"displayName":43,"description":44,"mimeTypes":45,"isOnByDefault":-1,"isLocked":-1,"timeToUseMS":32},"image_editing","Image Editing","Use this tool to edit an image from a prompt.",[46],"image/*",{"name":48,"displayName":49,"description":50,"mimeTypes":51,"isOnByDefault":-1,"isLocked":-1,"timeToUseMS":32},"document_parser","Document Parser","Use this tool to parse any document and get its content in markdown format.",[52,53],"application/*","text/*",{"name":55,"displayName":56,"description":57,"mimeTypes":-1,"isOnByDefault":-1,"isLocked":-1,"timeToUseMS":32},"query_calculator","Calculator","A simple calculator, takes a string containing a mathematical expression and returns the answer. Only supports +, -, *, ** (power) and /, as well as parenthesis ().",[]],"uses":{"dependencies":["conversation:list"]}},{"type":"data","data":[{"messages":1,"title":33,"model":37,"preprompt":5,"rootMessageId":3,"assistant":38,"shared":36},[2,11,20],{"id":3,"from":4,"content":5,"createdAt":6,"updatedAt":7,"children":8,"ancestors":10},"d2ba6d97-7f59-4041-82a4-6c42f92106b6","system","",["Date","2024-07-28T00:52:09.987Z"],["Date","2024-07-28T00:52:09.987Z"],[9],"eadfb5d2-8588-41ea-b364-63f490f7383d",[],{"from":12,"content":13,"files":14,"createdAt":15,"updatedAt":16,"ancestors":17,"id":9,"children":18},"user","make a cake",[],["Date","2024-07-28T00:52:10.136Z"],["Date","2024-07-28T00:52:10.136Z"],[3],[19],"386720c6-71e0-4e2e-9804-be64203a851b",{"from":21,"content":22,"createdAt":23,"updatedAt":24,"ancestors":25,"id":19,"children":26,"updates":27,"interrupted":36},"assistant","I am using python for this.\n\nThe goal is to build a simple implementation of the game \"Hangman\" in Python.\nIn Hangman, one player thinks of an object, and another tries to guess it by suggesting letters (each letter they suggest may or may not be part of the word).\nHere's how I'm going to implement this:\n\n1. The user can input any string, but only valid strings are used for drawing\n   characters on a Hangman figure.\n2. Based on the user's responses, you need to update the game state.\n\n```\n# hangman_game.py\n\nclass Game:\n    def __init__(self):\n        self.word = ''\n        # Initial values\n        self.lives = 6  # Number of wrong attempts (max)\n        self.guessed_letters = set()  # All letters that have been guessed, including duplicates\n        self.game_over = False\n        pass\n\n    @property\n    def word_to_guess(self):\n        return ['_'] * len(self.word)\n\n    def guess_letter(self, letter: str) -> None:\n        if not isinstance(letter, str):  # Input validation for strings only.\n            raise TypeError('Invalid input. Please provide a string.')\n\n        # convert all letters to lowercase\n        letter = letter.lower()\n\n        if not set(letter).issubset(set('abcdefghijklmnopqrstuvwxyz')):\n            raise ValueError(\"Please enter a valid alphabetic characters (A-Za-z)!\")\n            \n        self.guessed_letters.add(letter)\n        \n        for i in range(len(self.word)):\n            if self.word[i] == letter:\n                # update the word_to_guess\n                self.word = list(self.word_to_guess)\n                self.word[i] = letter\n\n    def is_game_over(self):\n        return 'game_over' if len([letter != '_' and letter.isalpha() for i, letter in enumerate(self.word) \n and not letter  in self.guessed_letters]) > 0 else False\n```\n\n**Challenge**: Your code should be flexible enough to handle a wide variety of words and guesses.",["Date","2024-07-28T00:52:10.136Z"],["Date","2024-07-28T00:52:10.144Z"],[3,9],[],[28,31,34],{"type":29,"status":30},"status","started",{"type":32,"title":33},"title","šŸ’Š COVID vaccine approval",{"type":35,"text":22,"interrupted":36},"finalAnswer",false,"text-generation-webui",null],"uses":{"dependencies":["http://172.16.111.10:5173/conversation/conversation"],"params":["id"]}}]}
nsarrazin commented 1 month ago

Hi :wave: I'd need to know the following in order to help you out

Seems to me like a prompt template issue or a stop token issue but can't be sure until I see the above.

cody151 commented 1 month ago

Hi @nsarrazin many thanks for getting back to me,

What settings are you using in text-generation-web-ui ?

What model are you running ?

What is the content of your MODELS var for chat-ui ? (Making sure to hide any secrets)

nsarrazin commented 1 month ago

Can you also show the MODELS used in chat-ui not the config-user.yaml for text-generation-webui ? :eyes:

cody151 commented 1 month ago

Can you also show the MODELS used in chat-ui not the config-user.yaml for text-generation-webui ? :eyes:

Sure please see the attached screenshots

Chat-UI image

text-generation-webui image

cody151 commented 1 month ago

@nsarrazin this is the contents of my Chat-UI ".env.local" file

MONGODB_URL=mongodb://[mongoIP]:12345
#HF_TOKEN=<your access token>
PUBLIC_APP_NAME=Chat-UI
PUBLIC_APP_ASSETS=chatui
PUBLIC_APP_COLOR=blue
PUBLIC_APP_DESCRIPTION="AI"
PUBLIC_APP_DATA_SHARING=
PUBLIC_APP_DISCLAIMER=
MODELS=`[
  {
    "name": "text-generation-webui",
    "id": "text-generation-webui",
    "parameters": {
      "temperature": 1,
      "top_p": 0.95,
      "max_new_tokens": 1024,
      "stop": []
    },
    "endpoints": [{
      "type" : "openai",
      "baseURL": "http://[text-gen-ip]:5000/v1",
      "extraBody": {
        "repetition_penalty": 1.2,
        "top_k": 50,
      }
    }]
  }
]`
cody151 commented 1 month ago

Anyone have any ideas? Chat-UI is completely unusable so I'm confused how people have (supposedly) got this working?

nsarrazin commented 1 month ago

Hard to tell what the issue is, probably a config error somewhere. I know people have successfully used llama.cpp OpenAI server as a backend so could be worth trying that to isolate where the issue is coming from

cody151 commented 1 month ago

Hard to tell what the issue is, probably a config error somewhere. I know people have successfully used llama.cpp OpenAI server as a backend so could be worth trying that to isolate where the issue is coming from

Thanks but llama.cpp is abysmally slow compared to Ooogabooga text-gen.

cody151 commented 1 month ago

@nsarrazin could you change the tag to "bugs" as this is clearly broken in the current implementation. Not sure why "Support" tag was added..