added llama hub connector for reading .MD files

dgarnitz / vectorflow

VectorFlow is a high volume vector embedding pipeline that ingests raw data, transforms it into vectors and writes it to a vector DB of your choice.

https://www.getvectorflow.com/

Apache License 2.0

676 stars 49 forks source link

Closed dgarnitz closed 1 year ago

dgarnitz commented 1 year ago

Added support for markdown, MD files, using Llama Hub Loaders.

This PR was to explore the possibility of replacing all loaders with Llama Hub loaders

Upload to local qdrant succeeds:

dgarnitz commented 1 year ago

Unit tests pass: