Store the data to be crawled before it's indexed? Or at least store it. This can be the basis for distributing the index. Or rather the data that is indexed.
Should be stored in IndexedDB.
For every change + a certain amount of idle time, so you know nothing more is happening: Create a new webtorrent.
Store the data to be crawled before it's indexed? Or at least store it. This can be the basis for distributing the index. Or rather the data that is indexed.
Should be stored in IndexedDB.
For every change + a certain amount of idle time, so you know nothing more is happening: Create a new webtorrent.