ScrapeGraphAI / Scrapegraph-ai

Python scraper based on AI
https://scrapegraphai.com
MIT License
14.06k stars 1.12k forks source link

does it support custom embeddings #460

Closed riddle911 closed 1 week ago

riddle911 commented 1 month ago

Silicon Cloud offers a vast array of free models, including Large Language Models (LLMs) and vector models, with support for online invocation. I would like to know what methods can be used to support the invocation of their API. Below is the example of the api. https://docs.siliconflow.cn/reference/createembedding-1

VinciGit00 commented 1 month ago

Hi, we will add it if it is possible

f-aguzzi commented 1 month ago

The current implementation of ScrapeGraph uses LangChain as a wrapper to exchange data with external LLM providers. Silicon Cloud is not currently supported by LangChain, so adding it within ScrapeGraph will require a custom-made wrapper. I'll work on it (@VinciGit00 thinks it might actually be a good thing to bypass LangChain in the long run) but it might take a while. We'll keep you updated.