FEATURE: Consumer prefetch update?

[mortiy]

Hi @cressie176, Is there any possible/recommended way I can update existing subscription settings, like prefetch?

[cressie176]

Hi @mortiy,

I don't believe it is possible to change a consumer specific prefetch with the AMQP. The only way would be to cancel the consumer and re-consume, specifying the new prefetch value, e.g.

const subscription1 = await broker.subscribe('s1', { prefetch: 1 });
await subscription1.cancel();
const subscription2 = await broker.subscribe('s1', { prefetch: 2 });

If you use a channel level prefetch it is possible with AMQP, but not with Rascal since it uses a channel pool.

[mortiy]

Got it, thank you.

[cressie176]

Sorry @mortiy, I just realised I was wrong with the comment about channel pooling - this is only when publishing messages. Rascal users a single channel per consumer, so it might actually be possible to update it. Let me check

[cressie176]

I think it will be possible with the addition of a new method on the subscription and some other changes. e,g,

const subscription = await broker.subscribe('s1', { prefetch: 1 });
subscription.setPrefetch(2);

The catch is that for this to work, I will have to change Rascal's existing mode of consumer level prefetch to channel level prefetch. I think this will be OK since Rascal does not share channels between consumers.

I also don't think it makes sense to support changing any other subscriber options. Do you agree?

[mortiy]

I think this will be OK since Rascal does not share channels between consumers.

Sounds good for me.

I also don't think it makes sense to support changing any other subscriber options. Do you agree?

I don't have any other options that I'm willing to change, so yes, that should work.

[cressie176]

I've read up on Consumer Prefetch and things aren't quite as straight forward as I had hoped. According to the docs, setting a channel level prefetch causes consumption to be slow, especially in a clustered environment. I'm not sure if this is still true when using an explicit channel per consumer, but suspect it might be. Therefore it wouldn't be safe to change Rascal to set a channel level prefetch.

However, it is possible to specify both a channel and consumer level prefetch, in which case both will be honoured. So I could provide a subscription.setPrefetch(i); method which you could use to override the consumer level prefetch if you did not want to go to the trouble of cancelling the consumer, albeit at a potential performance cost. However, this could only ever be used to reduce the preset from the initial consumer level one, but never to increase it since both would remain in effect.

Overall, I'm not sure the risk of an unexpected (if someone didn't RTFM) performance penalty is worth the trouble, so will give it some more thought.

[cressie176]

Will be published shortly as v17. Docs here - https://github.com/onebeyond/rascal#channelprefetch