The new service structure makes notebooks hard. This makes them easy again. Notebooks now have a separate poetry environment that loads all the others.
Changes proposed in this pull request
Adds a notebook pyproject.toml with all other services as dependencies
Updates quickupload.ipynb to use them to show they work
Updates quickupload.ipynb to use the new embedding model
Adds a script to quickupload.ipynb to quickly embed and save a directory of raw files to .jsonl without using the vector store
Guidance to review
Confirm you're happy with the new pyproject
Things to check
[x] I have added any new ENV vars in all deployed environments
Context
The new service structure makes notebooks hard. This makes them easy again. Notebooks now have a separate poetry environment that loads all the others.
Changes proposed in this pull request
pyproject.tomlwith all other services as dependenciesquickupload.ipynbto use them to show they workquickupload.ipynbto use the new embedding modelquickupload.ipynbto quickly embed and save a directory of raw files to.jsonlwithout using the vector storeGuidance to review
Things to check