search

PromtEngineer / localGPT

Chat with your documents on your local device using GPT models. No data leaves your device and 100% private.
Apache License 2.0
20.12k stars 2.25k forks source link

gradio demo #74

Open AK391 opened 1 year ago

AK391 commented 1 year ago

would be great to have a gradio UI for this similar to https://github.com/oobabooga/text-generation-webui

PromtEngineer commented 1 year ago

@AK391 Yes, it will be implemented in the near future. I am personally fimiliar with Streamlit but if someone can help with gradio UI or Panel, that would be great.

ashokrs commented 1 year ago

image

PromtEngineer commented 1 year ago

@ashokrs do you mind sharing the link, I can have a look at the code and improve it. I was going to implement it in streamlit but I am running into issues. If you don't mind, I can integrate it within LocalGPT.

ashokrs commented 1 year ago

@PromtEngineer I already shared. Please see the last line with the google drive share. Let me know if you can’t .

ashokrs commented 1 year ago

@PromtEngineer My comment went in as an image. I apologize. Here comes the link in textual form: https://drive.google.com/file/d/15uiXsEFd3QYsSRGqidkTLnh9TwXqqVub/view?usp=sharing

Hope it helps! Please drop a note as soon you grab it. I will delete/unshare the link.

ashokrs commented 1 year ago

@PromtEngineer I am sure u all know what to do to test execute this program. Here comes couple of simple inputs:

After setting your Python3 virtual env, activate it and run: pip3 install gradio

& Launch this program, python3 run_localGPT_WebUI.py

For the launch action, I set the "sharing with a public URL" turned on. In addition to launching the web ui available on local host http://localhost:8071, it will also make a public URL available and make it accessible for 72 hours as long as u keep the server runing. The url may be of the form: https://9214f72a5ac010750b.gradio.live (example url only; Your URL may vary).

!good luck to all of us!

PromtEngineer commented 1 year ago

@ashokrs Thank you, really appreciate the efforts here. I will go over the code and see if I can make any changes. If you are okay with it, I will add this in the next update to the repo. I already downloaded the file so you can unshare it. Thank you again.

ashokrs commented 1 year ago

@PromtEngineer Glad to know you are going to make the right modifications to get it working better. Please change as you see fit. Will be looking forward to see in the next repo update. People would appreciate it. I unshared the link. Please run it and let me know what you think as your first impression. Can't compare with oobabooga level. I get network error (exception kicks in) after asking two questions and their responses... The model is getting loaded into memory again and again taking the time for every invocation. I don't know why. May be due to bad design from my part. Anyway, it is with safe pair of hands to get better..

!good luck to all of us!

ashokrs commented 1 year ago

@PromtEngineer I rewrote the Web UI code myself to make it work for me. I removed the load model and predict function in a Model serving API server. This works and serves my purpose. Still, not a high quality code but works for me. The previous one was unusable more than 1 query. You can run as many queries on this web ui. I have a GitHUB account but I don't have any public repo. Please let me know how do I share two python files with you. BTW, my work is based off of the previous commit and may not have the updated version of other files. IS there any specific reason that I should retrofit to the new updates? Am i missing any major functionality?

PromtEngineer commented 1 year ago

@ashokrs that would be awesome if you can share, if you don't have a public github, you can email me. Not sure which commit you are using but the newest adds the ability to use quantized models along with now we have an API as well that can be used in your Apps. If possible, it would be great if you can retrofit it to the new updates.

ashokrs commented 1 year ago

@PromtEngineer please share your email or let me know where can I find it. yes. I saw the updated code. The support for GPT quantized model , the API, and the ability to handle the API via a simple web ui. I will do my best to retrofit my code but the quality of my code will not be good as I the ability for some of the updates are quite good. I will not be too embarrassed as my code works too :-)

Ananderz commented 1 year ago

@ashokrs would love to test it as well. The current UI is not the greatest. Got any screenshots as to what it looks like?

asubramania commented 1 year ago

@Ananderz @PromtEngineer As the localGPT project has been going thru some changes, I have been waiting to retrofit my extremely simple WebUI here. BTW, my WebUI doesn't have the ingestion capability yet. I have created my own API server which does one thing well (ie) execute the query and bring back the top_k=4 results back to the web ui. How I render the results on the web ui may be ugly.! but is is functional and few have been impressed as they are looking at the results only :-) @PromtEngineer Please confirm that the changes you have been making to localGPT main programs are done. then I will go ahead and retrofit the changes. 👍

asubramania commented 1 year ago

BTW both @asubramania and @ashokrs are the same person. I logged on as the wrong user above to respond the comment.

asubramania commented 1 year ago

Issue #48 refers to https://github.com/h2oai/h2ogpt This UI is absolutely awesome. I am embarrassed looking with my poor web ui. If someone is integrating with H2O.ai's h2oGPT Web UI, please let me know. I will liberate you and not trouble with my little try!

PromtEngineer commented 1 year ago

@asubramania I will be pushing one update today just to take all the model definitions etc to the constant.py file. Had a major update planned but will de delaying that. I think you can go ahead and create a PR for the WebUI. Let's integrate that and we can make changes as we move forward. Appreciate the contributions :)

ashokrs commented 1 year ago

@PromtEngineer Any luck pushing the updates to contant(s).py ? I am waiting for it to do my little retrofit.

PromtEngineer commented 1 year ago

@ashokrs yes just working on test it right now and will be pushing it tonight. Here is the branch if you want to work on it while I push the updates. https://github.com/PromtEngineer/localGPT/blob/api_update/constants.py

ashokrs commented 1 year ago

@PromtEngineer I managed to get the basic gradio based capability working. I am not used to following the best practices to update my new set of code in your "localGPT" repo. I have created a README.docx/pdf file along with content of all files in a zip file. How can I send it to you? For now, I have attached my zip file here. If for some reason, u can't download the file, please ping me here and suggest how else I can send to you. Please test it thoroughly and decide whether you want to merge to the main branch.

localGPT-try-API-n-WEBUI.zip

Good luck to all of us! Thanks again for this wonderful "localGPT" project.

asubramania commented 1 year ago

Updated the "run_localGPT_try_WEBUI.py" program and the "png" files for the Web UI. Attaching the whole ZIP file again. Please replace the previous version of "run_localGPT_try_WEBUI.py" with the one found in this zip. localGPT-try-API-n-WEBUI-v2.zip

PromtEngineer commented 1 year ago

@asubramania thanks for putting this together. I will have a look at this over the next few days and will get back to you if I need help with anything. Really appreciate this.

asubramania commented 1 year ago

@PromtEngineer sure. thanks.

asubramania commented 1 year ago

run_localGPT_try_WEBUI.zip @PromtEngineer I edited one of the files, **** to render a simple HTML accordion on the result part of it. This completes the cycle. I have attached the code here. Please take this version and overwrite the content from the zip file uploaded earlier..

asubramania commented 1 year ago

@PromtEngineer Please see the consolidated/cumulative zip file.

localGPT-try-API-n-WEBUI-v3.zip

PromtEngineer commented 1 year ago

Perfect, I will have some time over the weekend to test this out.

Get Outlook for iOShttps://aka.ms/o0ukef

From: Ashok R. Subramanian @.> Sent: Thursday, August 17, 2023 5:33:21 PM To: PromtEngineer/localGPT @.> Cc: PromptEngineer @.>; Mention @.> Subject: Re: [PromtEngineer/localGPT] gradio demo (Issue #74)

@PromtEngineerhttps://github.com/PromtEngineer Please see the consolidated/cumulative zip file.

localGPT-try-API-n-WEBUI-v3.ziphttps://github.com/PromtEngineer/localGPT/files/12374301/localGPT-try-API-n-WEBUI-v3.zip

— Reply to this email directly, view it on GitHubhttps://github.com/PromtEngineer/localGPT/issues/74#issuecomment-1683155903, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BAB6XLOP4GWCVUZRPRWUSZDXV2Z5DANCNFSM6AAAAAAYZW3U3Q. You are receiving this because you were mentioned.Message ID: @.***>

asubramania commented 1 year ago

I updated one of the files: run_localGPT_try_API.py localGPT-try-API-n-WEBUI-v4.zip

PromtEngineer commented 1 year ago

I was testing the code and am running into issues. I recreated the index for new files and the API code is not able to find the index. Trying to debug it. Will keep you posted. Thanks for all the help @asubramania

bp020108 commented 9 months ago

@PromtEngineer I am sure u all know what to do to test execute this program. Here comes couple of simple inputs:

After setting your Python3 virtual env, activate it and run: pip3 install gradio

& Launch this program, python3 run_localGPT_WebUI.py

For the launch action, I set the "sharing with a public URL" turned on. In addition to launching the web ui available on local host http://localhost:8071, it will also make a public URL available and make it accessible for 72 hours as long as u keep the server runing. The url may be of the form: https://9214f72a5ac010750b.gradio.live (example url only; Your URL may vary).

!good luck to all of us!

do you have updates for python files to use gradio as a webUI? What are the changes required and which python files?