search

oobabooga / text-generation-webui

A Gradio web UI for Large Language Models.
GNU Affero General Public License v3.0
40.58k stars 5.31k forks source link

Translator doesn't work in the API (UPD: I found a solution) #6121

Open Shark-vil opened 5 months ago

Shark-vil commented 5 months ago

Describe the bug

I tried to use the API along with the translator, but the API always outputs the English version of the answer. The extension triggers during the API call, but does not give the desired response. In the UI version everything works.

Is there an existing issue for this?

Reproduction

  1. Enable API and translator extension
  2. Use OpenAI API

Screenshot

Postman_WolLwu9VwT

Logs

-

System Info

Windows 10
RTX 3060 TI (8GB VRAM)
32GB RAM
Shark-vil commented 4 months ago

My solution is to edit this feature: https://github.com/oobabooga/text-generation-webui/blob/4820ae9aef3b7716cb00a6519955db435b6dcb56/extensions/openai/completions.py#L335-L337

I found out that the edited response of the translator extension is stored in the "visible" array. I have rewritten the code so that it works:

    for a in generator:
        try:
            answer = a['visible'][-1][1]
            if answer is None or answer == '':
                answer = a['internal'][-1][1]
        except:
            answer = a['internal'][-1][1]

I'm not sure if all these checks are necessary, but I added them just in case. You can try this solution too if you need a translator in the API. Code_UzQPouAeQV

Postman_9ddHoc6rcm

UPD: I added this for a direct request. I haven't checked if it works with Stream.