[feat] mysql or pgsql support

[Arvrairobo]

Hello, How are you today? let me know how can i contribute in this project?

I have successfully run the project, but it is based on sqlite only? how can i connect mysql or pgsql? does it support SAAS based architecture? @pkarw

Thanks bro

[pkarw]

Hey @Arvrairobo, yes, it's SQLite only at the moment, but I guess it can be pretty easily ported to pgsql - however, the question is, is it really needed as SQLite is really quite a powerful option?

what do you mean by the question of supporting SaaS architectures? I mean it already do - as you can register an account on openagentsbuilder.com which is kind of saas, so yes it's multitenant - and sqlite helps here a lot with the easy data separation

[Arvrairobo]

i would need one db one file system for saas, sqlite works as a file based system and for each tenant, if it creates one database file then it will be very problamatic to take backup plus for future updates how it will automatically upgrades to all the DB? i am also more comfortable with mysql and pgsql or even mongodb, can easily scale to multiple servers for horizontal as well as vertical upgrading, imagine, millions of users use this and millions of sqlite dbs, how to manage it?

[pkarw]

understood, well all DB has it's strong and weak sides :) maybe your contribution would be a pgsql support then? I think it could be pretty easily achieved as ALL db logic is encapsulated within the src/app/data/server/*-repository.ts + the db instance management inside the db-provider.ts - and all is using drizzle

I think a way to go would be to have a config setting (for example via ENV) setting the driver - then would migrate from the src/app/data/server/... structure to more complex one like src/app/data/server/sqlite and src/app/data/server/pgsql - where all the repositories would sit and their db-providers as well.

now the db-providers creates individual database file per tenat, in this pgsql architecture I think we could have a single database (for easier maintenance) and then some columnbased split - important thing will be to make sure all the statements (both selects, updates and deletes) uses this columnt to filter by the databseIdHash file (which is virtually a database name) - or maybe create the individual DBs inside postgres? I donkt know what do you think?

Please let me know about what kind of Saas do you think bc. we already offer this as a Saas :D https://openagentsbuilder.com is the registration page

[Arvrairobo]

Please let me know about what kind of Saas do you think bc. we already offer this as a Saas :D https://openagentsbuilder.com/ is the registration page ==> i guess right now the saas that you are offering is, one file system, multiple sqlite database? am i right? (i mean it has bottleneck while you are scaling) i would definitely go with one file system and one database that can host all the customers, they are differentiated with individual company id, so lets say there would be a company table, where each company is registered and each has unique company id, now that company id is referenced to other tables and in each of the queries, company id must be attached so it only bring the data related to that company.

i will go through with the db logic and try to see if i can migrate for the pgsql and can send you an PR, i would like to know the roadmap for this project, is it production friendly? seems right now it is only supporting openAI, i tried to integrate OLLAMA and it didnt work. so looking forward for the roadmap!

i would love to be part of this project and can contribute to the development.

[pkarw]

That's cool. Please feel welcome to OAB as a contributor! The roadmap is in the making. I had some issues with Ollama since it supported tools in pretty different ways than Openai did - I mean, choosing the different tools; probably, requires having different prompts for different LLMS, which is still doable but wasn't our priority. Also welcomed to improved support for Ollama

I'm just - me - coding it at the moment so any help is much appreciated :D

[Arvrairobo]

ok let me come up with pgsql and mysql migration and then get back to you, also regarding agents,how scalable/concurrency it can achieve? cause mostly code is based on node.js and it is single threaded so what about concurrency?

[Arvrairobo]

we have progressed further on this one, i will send PR request soon. thanks

[pkarw]

That sounds just terrific ❤️

[Arvrairobo]

what about the other features, i can see lots of open features are there? any major development done on those part? are we releasing 0.4.0 any soon?

[pkarw]

Sure Please feel free to pick up any interesting one form the open issues, ive just merged the Vector Search features into main and ready to release 0.4.0 Early next week (im on business trip until Saturday)

[pkarw]

Hey @Arvrairobo how are you doing? Do you need any support on that one?

[Arvrairobo]

@pkarw yes i am doing good, i have converted it into pgsql, 70% work is done, so soon will send you PR request, this week and next week i am travelling mostly so it will be slow from my side :( but overall looks great. i have seen 0.04 launched, will be merging on my local machine. thank you very much for the great work.

[pkarw]

Thanks! That's great! Let me know if you'd need any support

[Arvrairobo]

new version is broken @pkarw chat is not working and ending up with the error

[Arvrairobo]

[pkarw]

Is it from 'develop' branch? What kind of error is displayed in the console (server console)

[pkarw]

Hey! Just wanted to check up how things are going with this one?

[Arvrairobo]

successfully created with pgsql, just testing it with different options and whilte testing i am getting the errors, which i am correcting and also posting, so soon will send PR

[pkarw]

Cool it's great to hear!

[pkarw]

@Arvrairobo any progress on this one? :)

[Arvrairobo]

yes we are on it, so many features to test hence taking time. i am also occupied some other stuff, so taking more time then it should, sorry for late :(

[pkarw]

Thanks for heads up, no worries we're all busy and it's completely fine 🙌

[pkarw]

Hey @Arvrairobo just wanted to kindly ask how things are going and if you need any support here?