search

hpcaitech / ColossalAI

Making large AI models cheaper, faster and more accessible
https://www.colossalai.org
Apache License 2.0
38.8k stars 4.35k forks source link

[BUG]: CololssalChat is really weird. #3345

Closed hyunwoongko closed 1 year ago

hyunwoongko commented 1 year ago

🐛 Describe the bug

image

image

What's the 'appropriate content'...?

Environment

No response

FrankLeeeee commented 1 year ago

We have collected a list of words which are offensive and check if the content contains these words as a simple safety checker. This occurs perhaps because the some words are accidentally hit.

This has been stated in the banner section at the bottom of the page as well.

FrankLeeeee commented 1 year ago

I guess it might be because that damn is included as part of the chat history. Every query contains the word damn and thus rejected by the service.

JThh commented 1 year ago

You may try asking some more involved things

Screenshot 2023-03-30 at 3 40 43 PM Screenshot 2023-03-30 at 3 40 52 PM

like below. It responded quite well.

hyunwoongko commented 1 year ago

My first question was 'who is the tallest person in the world'. I don't understand why this is inappropriate.

hyunwoongko commented 1 year ago

@JThh Hello bro. Responding to a user's bug report with 'it works well for me' is meaningless.

In any case, we should think about improving this system, remembering that there was a conversational episode where no answer was possible even though not a single unsafe question was asked. Thanks for answering my bug report.

FrankLeeeee commented 1 year ago

Hi @hyunwoongko , I understand your confusion. To be honest, I do not know what the response contains as I can't see the response without adding a print statement in the backend service since we don't record any user input and output. Absolutely we should improve on this system. The simple safety filter is not enough and we are indeed currently working on it, but that will be the easiest to implement for now.

FrankLeeeee commented 1 year ago

@hyunwoongko We would appreciate a lot if you have some better suggestions to handle harmful input and output :)

hyunwoongko commented 1 year ago

I've checked https://github.com/hpcaitech/ColossalAI/blob/main/applications/Chat/inference/server.py this file, and there is a parameter named 'profanity_file'. could you share this file are you using?

FrankLeeeee commented 1 year ago

We used this open-sourced word list https://github.com/camsdono/CensorshipAPI/blob/main/censor_datasheet.csv and damn happens to be one of them

FrankLeeeee commented 1 year ago

I think my answers should resolve this issue and thus I will close this issue for now. Do re-open it if you feel there is more to discuss and we'd love to exchange ideas to better improve our system. 😺

hyunwoongko commented 1 year ago

@FrankLeeeee I think that list is too extensive and covers several languages.

For example, git was registered as 'bad word' in this list. so if user or model input 'git' into the prompt, all the after respond will be blocked. So this system can not explain what is 'git'. image

For another example, queue could be bad word in French, but it's a word that means 'waitlist' in English. but in this system, there's no language detector. So this system can not explain what is 'queue'. image

I suggest you to re-organize the word list and use language detector like fasttext detector. https://fasttext.cc/docs/en/language-identification.html

This system can not explain who is Madonna, because there is a word 'Madonna' in the list. image

Issues-translate-bot commented 1 year ago

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

@FrankLeeeee

FrankLeeeee commented 1 year ago

@FrankLeeeee I think that list is too extensive and covers several languages.

For example, git was registered as 'bad word' in this list. so if user or model input 'git' into the prompt, all the after respond will be blocked. So this system can not explain what is 'git'. image

For another example, queue could be bad word in French, but it's a word that means 'waitlist' in English. but in this system, there's no language detector. So this system can not explain what is 'queue'. image

I suggest you to re-organize the word list and use language detector like fasttext detector. https://fasttext.cc/docs/en/language-identification.html

This system can not explain who is Madonna, because there is a word 'Madonna' in the list. image

Hi @hyunwoongko Yes, we noticed that some common words are mis-treated as offensive. We have manually removed some words as we looked through the list and we will continue to improve on the word list and look into other methods to better filter harmful content, e.g. OpenAI Moderation API https://platform.openai.com/docs/guides/moderation/overview

hyunwoongko commented 1 year ago

Yeah. OpenAI API seems great for this case. Thanks!

FrankLeeeee commented 1 year ago

Thanks for your feedback as well!