search

invoke-ai / InvokeAI

Invoke is a leading creative engine for Stable Diffusion models, empowering professionals, artists, and enthusiasts to generate and create visual media using the latest AI-driven technologies. The solution offers an industry leading WebUI, and serves as the foundation for multiple commercial products.
https://invoke-ai.github.io/InvokeAI/
Apache License 2.0
23.63k stars 2.43k forks source link

[enhancement]: localize prompts #2607

Open ParisNeo opened 1 year ago

ParisNeo commented 1 year ago

Is there an existing issue for this?

Contact Details

No response

What should this feature add?

Prompts are mostly written in english. Since the UI supports multiple languages, Why not add the possibility to write the prompt in a different language.

I can develop a python script that uses the mbart-large-50-many-to-many-mmt model from facebook which is available on hugging face servers to translate the prompt from the language to English. Thus people can write their prompts in their favorite language and still can have answers.

This should just add a translation step before starting the diffusion execution.

Alternatives

No response

Aditional Content

No response

hugodopradofernandes commented 1 year ago

Although it is a nice idea, it must be optional, as SD understands a little bit of other languages, and mixing languages in the prompt results in interesting images.

女性 for example will return a japanese woman without translation.

ParisNeo commented 1 year ago

Totally agree. I propose to add it in the options just like the super-resolution or face restauration

ParisNeo commented 1 year ago

I have already coded this in my personal UI for stable diffusion and I think this should be done in a more global tool like invoke ai

github-actions[bot] commented 1 year ago

There has been no activity in this issue for 14 days. If this issue is still being experienced, please reply with an updated confirmation that the issue is still being experienced with the latest release.

ParisNeo commented 1 year ago

Hi,

I want to grub this issue and add the translation tool to it if possible. But I am waiting for the team to move to the new nodes based UI before starting my development.

github-actions[bot] commented 1 year ago

There has been no activity in this issue for 14 days. If this issue is still being experienced, please reply with an updated confirmation that the issue is still being experienced with the latest release.

ParisNeo commented 1 year ago

I still would like to add the language localization. Still wait for moving to the new nodes based UI before starting my development.

github-actions[bot] commented 1 year ago

There has been no activity in this issue for 14 days. If this issue is still being experienced, please reply with an updated confirmation that the issue is still being experienced with the latest release.

ParisNeo commented 1 year ago

I have already created an extension called prompt_translator for AUTOMATIC1111's sd-webui. I can bring the same idea to InvokeAI

Millu commented 1 year ago

Hi @ParisNeo if you are still interested, the nodes backend is fully up and ready for contributions!

ParisNeo commented 1 year ago

Hello there and thank you for answering. It's been a long time since i've posted this. I have been developping my own ai tool the last months and it takes up all my free time as I am the lead dev. I'll try to checkout this and maybe integrate my artbot in there instead. Artbot is a prompt generation llm that can build very good prompts from text from any language. So basically the user writes few words and the Ai does the rest. It is one of more than 300 tools in my lord of large language models tool. If you are interested to check it out, it is on my github. It has a server in the backend, so with some simple python code you can integrate it anywhere. You can also look at my videos. If i have time, i'll check it out.

Millu commented 1 year ago

@ParisNeo of course :)

I'll take a look! Sounds very interesting