search

LagPixelLOL / ChatGPTCLIBot

ChatGPT Bot in CLI with long term memory support using Embeddings.
MIT License
341 stars 38 forks source link

Ingesting a folder of text files #11

Closed AutoCBT closed 1 year ago

AutoCBT commented 1 year ago

Is it possible to ingest an entire folder of text files? Or does it have to be a single text file?

LagPixelLOL commented 1 year ago

ofc its possible, but i didn't code that yet😢. so currently with this program you cant do it, but you can merge your files' content into a single text file before parsing them with my program.

AutoCBT commented 1 year ago

Thanks for the feedback, I did end up doing that. Next I'll be trying to append folder structure before each file with a file name. Should be largely the same effect.

I was curious about one thing though, when the program is processing your text file, it will use one core, then switch to another one, then another one. Just thought that was odd behavior, doesn't make much of a difference if it's not intended to be multithreaded though so :man_shrugging:

As far as how the program worked, had a couple hiccups but otherwise worked fine:

Only two issues, pretty damn good for something this fresh.

LagPixelLOL commented 1 year ago

The SSL thing is pretty weird, I'm not good at Linux at all, cURL's default SSL path is incorrect, and it doesn't bundle it by itself, so i need to find the SSL certification file in the system, but i only have a centos vps, so i can't actually know other system's SSL cert path. Your issue with inserting the API key into the terminal when prompted did not work, because the program will try to connect to the OpenAI API, but when cURL can't find the SSL cert, it will just fail. Maybe you can help me with how to find the SSL cert tho...

AutoCBT commented 1 year ago

As far as I could find on the net you had the correct directory, I did a search of the system and it doesn't show up on my backups until I placed it there (Also it shows up in flatpaks, but that's unsurprising). Looks like Linux mint doesn't come preinstalled with it.

LagPixelLOL commented 1 year ago

Fixed SSL cert issue in v1.2.5. Closed as completed.