search

storacha / w3name

IPNS client and service for generating, updating, and tracking immutable names with signed updates
Other
45 stars 12 forks source link

W3name publishing - Use a queueing system which can scale #66

Closed adamalton closed 1 year ago

adamalton commented 2 years ago

Currently all w3name record requests are stored in memory before they are processed by the POST api end point to publish on the DHT. At the current traffic this is perfectly acceptable, but at a certain scale or volume of requests, this might cause the service to fall over.

joshghent commented 2 years ago

@francois-potato to work on a proposal for this work. Suggested implementation is to use SQS with a FIFO queue. Using this tool: https://github.com/bbc/sqs-consumer

mbommerez commented 2 years ago

Proposal: https://www.notion.so/ipns-publisher-2-258f425cabf4444b826481918f7b4c1e

adamalton commented 1 year ago

This ticket is really in the wrong project. I've made a replacement ticket in ipns-publish, which is where it belongs: https://github.com/web3-storage/ipns-publisher/issues/8