search

magpie-align / magpie

Official repository for "Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothing"
https://magpie-align.github.io/
MIT License
175 stars 22 forks source link

Use online LLM inference #8

Open mkaratayev opened 1 week ago

mkaratayev commented 1 week ago

Hi,

Thank you for the excellent project! I have a question: is it possible to use the online inference model with your pipeline?

Thanks in advance!

fly-dust commented 6 days ago

Hi,

Since the online inference service provider will automatically provide the template for inference, it cannot be used for generating the instruction (step 1). However, you can use the online inference platform to generate the response (step 2)! You can do this by setting api=True and providing a Together api key in gen_res.py.