Translate using a private model like llama

[allanbatista]

Hi. There exists any plan to implement a translation using a private model like llama?

[jb41]

No, not right now. GPT-4 is good enough, and it’s not that expensive when translating a full book - ~$5.

Feel free to issue a Pull Request with Llama.

[delta-whiplash]

Hello @allanbatista

you can use any openai compatible server like this :

in main.py :

if __name__ == "__main__":
    # ... (rest of the code)

    if args.mode == 'translate':
        config = read_config(args.config)
        openai_api_key = config['openai']['api_key']
        openai_base_url = config['openai'].get('base_url')  # Get base_url if it exists

        openai_client = OpenAI(
            api_key=openai_api_key,
            base_url=openai_base_url if openai_base_url else None  # use custom endpoint if it's not defined
        )

        # ... (rest of the code)

in the config.yaml file :

openai:
  api_key: "API KEY"
  base_url: "http://10.10.20.30:8000/v1"

[delta-whiplash]

@allanbatista do you have good "private" models for translation ? Am searching for some if you have any suggestions

[pixelfiller]

Good project, but this isn't about the $5 for a book. I am ready to pay $10 to run a GPU cluster, so I can use Llama3 on multiple GPUs and get translations in under 1 minute.

[delta-whiplash]

Good project, but this isn't about the $5 for a book. I am ready to pay $10 to run a GPU cluster, so I can use Llama3 on multiple GPUs and get translations in under 1 minute.

Do you have a good provider for this type of usecase ?