search

GerevAI / gerev

🧠 AI-powered enterprise search engine 🔎
https://app.klu.so/signup?utm_source=github_gerevai
MIT License
2.71k stars 178 forks source link

Suggestion: Change data source file structure #57

Closed Itaykal closed 1 year ago

Itaykal commented 1 year ago

TL;DR - Make each data source a package instead of a module.

Changing data_sources folder to contain foldersinstead of a file for each source.

It will make the code less "monolithic" without creating a meess, you could implement your own clients/wrappers for those clients which will make the code much more easy to maintain. add files for testing, etc etc.

Have a look at CloudQuery's solution. (although it's an extremely different product for a different use case..) I guess it's better to do that sooner than later ;/

rishi003 commented 1 year ago

I was looking at the data_sources folder too, and I completely agree. A structure would make it more maintainable overall. I would also recommend using a mono repo tooling for the entire project.

Roey7 commented 1 year ago

Done in https://github.com/GerevAI/gerev/pull/60