Ohyeon5 / syn-search

Optimal Drug Synthesis Condition Searching Conversational AI, using LLMs

MIT License

2 stars 2 forks source link

[Vector DB] Convert USPTO to index database #10

Open Ohyeon5 opened 7 months ago

Ohyeon5 commented 7 months ago

Goal

Using Llama-index to convert USPTO to an index database. The main challenge here is to find a smart way to get over the token limit issue. Also, parallelization of the process.

Task list

[ ] Read potential possibilities to parallelize the process (connect multiple index db)
[x] Resolve the token limit issue
[ ] Start processing

Ohyeon5 commented 7 months ago

Nice tutorial to follow: https://howaibuildthis.substack.com/p/llamaindex-how-to-use-index-correctly Use ComposableGraph to combine different indices from llama_index.indices.composability import ComposableGraph