search

vectara / vectara-ingest

An open source framework to crawl data sources and ingest into Vectara
https://vectara.com
Apache License 2.0
147 stars 50 forks source link

Adding support for Ray #33

Closed ofermend closed 1 year ago

ofermend commented 1 year ago

Support ray in website_crawler to multi-process crawling of website. Refactored text extraction in indexer: moved to extract.py and cleaned it up.