search

buddycloud / channel-directory

The buddycloud search engine. Search and recommendations based on solr and mahaut.
https://buddycloud.org/wiki/Channel_Directory_Project
Apache License 2.0
21 stars 7 forks source link

Write crawler subscription logic #20

Open abmargb opened 13 years ago

abmargb commented 13 years ago

The crawler will subscribe to open nodes and receive their updated information. This should be the standard way of getting nodes information. Node crawlers would work when data cannot be sent via a subscription feed.

abmargb commented 13 years ago

Firehose node will summarize nodes notification. When crawling a new server, the crawler will try to subscribe to its firehose.

abmargb commented 13 years ago

Done. Must test with /firehose node.

abmargb commented 10 years ago

Subscribing means the crawler will eventually get event notifications. We need more testing on the notification-based approach before closing this.