fix:dashboard:github import

louis030195 commented 1 year ago

Mixed changes mainly wanted to make JS SDK isomorphic but tested on the github fix in edge (unless we find a way to simulate edge runtime in tests maybe https://edge-runtime.vercel.app/packages/vm)

[x] JS SDK working on edge and node in a single import - no more ugly hacks
[x] Github import in edge runtime (supposed to solve the slowness and timeout issues)

went bulldozer a bit, side effect:

no more callback in splittext (personally would prefer an async generator if we want this)
splittext and merge needs to be async because of wasm async import required for edge runtime
different splitters (removed kinda dead code old splitters not used/documented)

vercel[bot] commented 1 year ago

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
embedbase	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	May 19, 2023 9:13am
embedbase-docs	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	May 19, 2023 9:13am

louis030195 commented 1 year ago

almost ready to merge imho, will do later

didnt test the .generate in vercel edge though!

different-ai / embedbase

fix:dashboard:github import #99