Add my model or API into source code?

[Nduy2k]

I wan to to add my model into source code for comparison, how i can do it. Thank you!

[ralyodio]

you can search with huggingface api key and add models that way, otherwise I don't know.

[Nduy2k]

I mean is add my Language Model into the list of model is available. Then, i can compare the result of my model with different model.

[zainhuda]

you can take a look at the readme to see how to do it. you'll have to modify the models.json and add your own completion function and it should take care of itself on the frontend!

[keldenl]

My fork https://github.com/keldenl/openplayground adds support for gpt-llama.cpp (https://github.com/keldenl/gpt-llama.cpp#quickstart-installation), which makes adding local models pretty simple if you have gpt-llama.cpp set up already!

[DrewPear309]

you can take a look at the readme to see how to do it. you'll have to modify the models.json and add your own completion function and it should take care of itself on the frontend!

I'm stuck. Hoping you can help. I have Llama cpp running with a model and openplayground running in docker. i can edit the models.json file and app.py. editing models.json i understand i think. how to edit app.py to add my locally running llm i do not. after adding my model to models.json per instructions and restarting docker, i don't see it listed. i'm hoping it also has to be listed in app.py. anyways, can you be specific, because the guide doesn't make it clear to me. Thank you!

[Nduy2k]

you can take a look at the readme to see how to do it. you'll have to modify the models.json and add your own completion function and it should take care of itself on the frontend!

I'm stuck. Hoping you can help. I have Llama cpp running with a model and openplayground running in docker. i can edit the models.json file and app.py. editing models.json i understand i think. how to edit app.py to add my locally running llm i do not. after adding my model to models.json per instructions and restarting docker, i don't see it listed. i'm hoping it also has to be listed in app.py. anyways, can you be specific, because the guide doesn't make it clear to me. Thank you!

You need to edit file model.json with path: /username/.config/openplayground/model.json this is another file model.json.

[DrewPear309]

you can take a look at the readme to see how to do it. you'll have to modify the models.json and add your own completion function and it should take care of itself on the frontend!

I'm stuck. Hoping you can help. I have Llama cpp running with a model and openplayground running in docker. i can edit the models.json file and app.py. editing models.json i understand i think. how to edit app.py to add my locally running llm i do not. after adding my model to models.json per instructions and restarting docker, i don't see it listed. i'm hoping it also has to be listed in app.py. anyways, can you be specific, because the guide doesn't make it clear to me. Thank you!

You need to edit file model.json with path: /username/.config/openplayground/model.json this is another file model.json.

Wow. Thank you! That's a biggy. On Windows 11 i found it in C:\Users\joe\AppData\Roaming\openplayground. i hope this might be helpful for others. i made the addition to models.json using the example for a local llm with no changes. after restarting docker, i see no added option under settings. Since i'm not sure how to edit app.py, i'm not sure if that is significant. Thank you!

[zainhuda]

appreciate it @Nduy2k , we should probably update the documentation to make it clearer

[DrewPear309]

zainhuda

@zainhuda, could you please give me an example of how i add code to app.py to add my locally running Llama cpp? Thank you!

[zainhuda]

hey @DrewPear309 try taking a look at this here: https://github.com/nat/openplayground/pull/11 does a very good job of integrating llama.cpp

[DrewPear309]

hey @DrewPear309 try taking a look at this here: #11 does a very good job of integrating llama.cpp

Thank you!