search

raid-guild / gaianet-rag-api-pipeline

Supercharge your Gaianet node by generating a vector knowledge base from any API. Demo slides: https://hackmd.io/@santteegt/ByoykY4nC#/ Link to Docs below
https://raid-guild.github.io/gaianet-rag-api-pipeline/
MIT License
1 stars 0 forks source link

2.2.3 Optimize Parsing Logic #22

Closed earth2travis closed 2 months ago

earth2travis commented 3 months ago

Refine and optimizing the parsing logic used in the data pipeline. Based on the results from testing, adjustments may be needed to improve the accuracy and efficiency of data extraction and transformation. Tweaking the parsing logic ensures that the pipeline produces high-quality, consistent data for embedding and storage.

earth2travis commented 3 months ago

Related to: https://github.com/raid-guild/gaianet-rag-api-pipeline/issues/21

santteegt commented 3 months ago

Introdced some optimizations in 55f5317f1261ded640a42ae1887dd082678d4ce5