openchatai / OpenChat

LLMs custom-chatbots console ⚡
https://open.cx
MIT License
5.17k stars 642 forks source link

Not working after uploading #8

Closed sanlega closed 1 year ago

sanlega commented 1 year ago

I tried a bunch of things already but it seems that the the files cant be uploaded to pinecone, maybe the chunk splitter is not working fine, here are some logs:

image image

I have all the keys well set and i've tested a bunch of methods, different OS's and everything. I think that it might be the pinecone client.

eduardosantosjr commented 1 year ago

Make sure you've put key values before run make install command. If you do this after, you'll have to do the same in .env file or even delete it before run make install again.

sanlega commented 1 year ago

Make sure you've put key values before run make install command. If you do this after, you'll have to do the same in .env file or even delete it before run make install again.

I did it a bunch of times no mater how i did it it doesn't work. Right now im trying it on a M1 mac but i also tried in a windows x64 and a mac x64.

eduardosantosjr commented 1 year ago

There is another problem after this step that I'm trying to figure out the reason. Check if you're in the same situation. By the way, I'm using a m1 mac.

image

sanlega commented 1 year ago

I had the same problem, it seems that the document "parser" in langchan in this code is deprecated i had to change some files that contained those imports and it was ok then but either way, pinecone didnt receive the data. I think that the token length counter for the vector creation is not working fine but idk

eduardosantosjr commented 1 year ago

So which files you had changed? I've tried fixing it using the commands below inside llm-server container.

npm update
npm install
node node_modules/node-sass/scripts/install.js
npm rebuild node-sass
sanlega commented 1 year ago

I changed them before running the containers, one was on the pages directory. You can use 'grep' in the terminal to look for the 2 files that have to be changed, be careful because in one file there are 2 documen loaders but only one is the deprecated one, i think.

grep -rnw '.' -e 'langchain/document_loaders'

eduardosantosjr commented 1 year ago

I was debugging the code and find out the error: In file ingest.ts, code are trying load a uncreated path file. So, its not about the deprecated dependence (despite its not a good thing). If I find something more I tell you guys.

Ps.: A hot-reload could be a good thing ⏩

image

gharbat commented 1 year ago

It seems there is an issue with the shared volume not syncing between containers, I'm working on a fix and will release a new update with many bugs resolved this weekend.

Also will add a validation when running the make command so it tells you if your apis keys are ok or not.


@eduardosantosjr agree, hot reload indeed is a good idea, I will ship it this weekend.

I'll drop a message on the discord server once the update is shipped :)

https://discord.gg/yjEgCgvefr

gharbat commented 1 year ago

Introduced windows support in https://github.com/openchatai/OpenChat/releases/tag/0.3.0 if you have any other issues, please feel free to open a new issue :)