feathersjs-ecosystem / feathers-sync

Synchronize service events between Feathers application instances
MIT License
222 stars 42 forks source link

[Feature Request] Add retry mechanism #143

Closed defiboy closed 3 years ago

defiboy commented 4 years ago

I'm wonder if would it be possible to add a retry mechanism to this package when we try to connect rabbitMQ or redis

What do you think?

daffl commented 4 years ago

Doesn't Redis/AMQP already have an option that can be passed to do that?

abudhia-oyo commented 4 years ago

My application server instance also stops intermittently due to the redis connection issue.

app.configure(sync({ uri: app.get("rediscloudurl") }));

Are there any params I can pass above so redis connection is retried?

apmcodes commented 4 years ago

Using amqp. As this relates to reconect attempt, so thought of adding to this issue.

The amqplib doesn't have reconnect logic build in. We need to listen connection close event and add reconnect logic. Also, need to catch error event, as currently FeathersJs app crashes on rabbitmq heartbeat timeout.

Example https://github.com/squaremo/amqp.node/issues/557#issuecomment-597116027

events.js:292
      throw er; // Unhandled 'error' event
      ^

Error: Heartbeat timeout
    at Heart.<anonymous> (/node_modules/amqplib/lib/connection.js:427:19)
    at Heart.emit (events.js:315:20)
    at Heart.EventEmitter.emit (domain.js:483:12)
    at Heart.runHeartbeat (/node_modules/amqplib/lib/heartbeat.js:88:17)
    at listOnTimeout (internal/timers.js:549:17)
    at processTimers (internal/timers.js:492:7)
Emitted 'error' event on ChannelModel instance at:
    at Connection.emit (events.js:315:20)
    at Connection.EventEmitter.emit (domain.js:483:12)
    at Heart.<anonymous> (/node_modules/amqplib/lib/connection.js:428:12)
    at Heart.emit (events.js:315:20)
    [... lines matching original stack trace ...]
    at processTimers (internal/timers.js:492:7)
[nodemon] app crashed - waiting for file changes before starting...
fragilehm commented 4 years ago

Having the same issue with Redis. So when the redis goes down the server crashes: image

Creiger commented 3 years ago

Hi i have made retry mechanism for amqp. It will try to connect every second and also reconnect if the connection fails later. I am not sure i it is ok for pull request https://github.com/Creiger/feathers-sync

daffl commented 3 years ago

Closed via #160