search

macrocosm-os / prompting

SN1: An incentive mechanism for internet-scale conversational intelligence
https://bittensor.com
MIT License
78 stars 43 forks source link

Increase efficiency of wiki context creation #113

Open steffencruz opened 8 months ago

steffencruz commented 8 months ago

At present, 40-50% of wiki articles are rejected because they do not contain sections with greater than our minimum number of words. This introduces extra latency as we sequentially download additional articles until finding a suitable one.

We should instead download batches of pages concurrently so that we don't need to wait for sequential processing times.

steffencruz commented 7 months ago

@p-ferreira related to your current work