smrchy / rsmq

Redis Simple Message Queue
MIT License
1.76k stars 125 forks source link

Hitting 'message too long' error. At what size does this occur? #15

Closed 0xgeert closed 9 years ago

0xgeert commented 9 years ago

Hitting message too long as defined in https://github.com/smrchy/rsmq/blob/deb56ab82f6349b9578bc8d4e15286ca53fa058f/test/test.coffee (could't find other place it's defined in code)

It this a redis imposed threshold? Do you happen to know the max size?

mpneuried commented 9 years ago

see this line of code: https://github.com/smrchy/rsmq/blob/deb56ab82f6349b9578bc8d4e15286ca53fa058f/index.coffee#L152

The default maxsize is 65536 chars. But you can change the configuration by setting to option maxsize. See docs https://github.com/smrchy/rsmq#createqueue

smrchy commented 9 years ago

The maxsize setting is just a precaution so that a queue can not make a small Redis server run out of memory. Technically it's possible to store videos, images etc. in the queue but you might want to rethink your design and store these things in S3 or similar systems. Send a small message to RSMQ and let the worker retrieve the blob from your storage (e.g. S3).

briangruber commented 7 years ago

Looks like you can't set it over maxsize as there is a check for that range. I created this to help with that for those that need it. #45

zubair1024 commented 6 years ago

Seems like setting the maxsize to -1 does not work. I'm getting "maxsize must be between 1024 and 65536"

mathurs commented 5 years ago

It seems if you have created the queue before setting maxsize, it needs to be deleted from redis and recreated with maxsize. Otherwise you still get this error. Afrer recreating, maxsize -1 works well.